|
Size: 4635
Comment: clarify question, un-indent examples, add find -exec +
|
Size: 1770
Comment:
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 1: | Line 1: |
| #pragma section-numbers 3 | |
| Line 2: | Line 3: |
| == How can I replace a string with another string in a file, or in all the files in a directory? == `ed` is the standard UNIX command-based editor. Here are some commonly-used syntaxes for replacing the string `olddomain.com` by the string `newdomain.com` in a file named `file`. All four commands do the same thing, with varying degrees of portability and efficiency: |
== How can I replace a string with another string in a file? == We're trying to edit files, which means we need a file editor. Most systems come with a range of editors, but POSIX defines only two that we can use: `ed` and `ex`. '''Warning''': A lot of bad advice on the Internet will recommend `sed -i` or `perl -i` for this. While on the surface it may look like these tools modify your files, what they ''really'' do is recreate your file and delete the original. This has many downsides and is often very risky. Open handles are destroyed, certain metadata will get lost, a lot of excess disk I/O is involved, and depending on the implementation, interrupted operations may cause your file to get lost. Just use a file editor. `ed` is the standard UNIX command-based editor. `ex` is another POSIX command-based editor with syntax slightly similar to that of `sed`. Here are some commonly-used syntaxes for replacing the string `old` by the string `new` in a file named `file`. |
| Line 6: | Line 12: |
| # Bash ed -s file <<< $'g/olddomain\\.com/s//newdomain.com/g\nw\nq' # Bourne (with printf) printf '%s\n' 'g/olddomain\.com/s//newdomain.com/g' w q | ed -s file printf 'g/olddomain\\.com/s//newdomain.com/g\nw\nq' | ed -s file # Bourne (without printf) ed -s file <<! g/olddomain\\.com/s//newdomain.com/g w q ! |
ex -sc '%s/old/new/ge|x' file ed -s file <<< $'g/old/s//new/g\nw\nq' |
| Line 22: | Line 16: |
| To replace a string in all files of the current directory: | If you need to edit multiple files, `ex` can easily be combined with `find`. `ed` is hard to call from `find` since it takes commands as input. |
| Line 25: | Line 19: |
| for file in ./*; do [[ -f $file ]] && ed -s "$file" <<< $'g/old/s//new/g\nw\nq' done }}} To do this recursively, the easy way would be to enable globstar in bash 4 (`shopt -s globstar`, a good idea to put this in your `~/.bashrc`) and use: {{{ for file in ./**/*; do [[ -f $file ]] && ed -s "$file" <<< $'g/old/s//new/g\nw\nq' done }}} If you don't have bash 4, you can use [[UsingFind|find]]. Unfortunately, it's a bit tedious to feed `ed` stdin for each file hit: {{{ |
find . -type f -exec ex -sc '%s/old/new/ge|x' {} \; |
| Line 44: | Line 23: |
| `sed` is a '''Stream EDitor''', not a '''file''' editor. Nevertheless, people everywhere tend to abuse it for trying to edit files. It doesn't edit files. GNU `sed` (and some BSD `sed`s) have a `-i` option that makes a copy and replaces the original file with the copy. An expensive operation, but if you enjoy unportable code, I/O overhead and bad side effects (such as destroying symlinks), this would be an option: | You may notice that this causes an `ex`/`ed` process per file. If your `ex` is provided by `vim` (NOT POSIX), you can optimize that into: |
| Line 47: | Line 26: |
| sed -i 's/old/new/g' ./* # GNU sed -i '' 's/old/new/g' ./* # BSD for file in ./* # Other do [ -f "$file" ] && sed 's/old/new/g' "$file" > "$file~" && mv "$file~" "$file" done |
find . -type f -exec ex -sc 'argdo %s/old/new/ge|x' {} + |
| Line 57: | Line 29: |
| Those of you who have perl 5 can accomplish the same thing using this code: {{{ perl -pi -e 's/old/new/g' ./* }}} Recursively using `find`: {{{ find . -type f -exec perl -pi -e 's/old/new/g' {} \; # if your find doesn't have + yet find . -type f -exec perl -pi -e 's/old/new/g' {} + # if it does }}} If you want to delete lines instead of making substitutions: {{{ # Deletes any line containing the perl regex foo perl -ni -e 'print unless /foo/' ./* }}} To replace for example all "unsigned" with "unsigned long", if it is not "unsigned int" or "unsigned long" ...: {{{ find . -type f -exec perl -i.bak -pne \ 's/\bunsigned\b(?!\s+(int|short|long|char))/unsigned long/g' {} \; }}} Finally, for those of you who shun `ed` and have ''none'' of the useful things above, here's a script that may be useful: {{{ #!/bin/sh # chtext - change text in several files # neither string may contain '|' unquoted old='olddomain\.com' new='newdomain\.com' # if no files were specified on the command line, use all files: [ $# -lt 1 ] && set -- ./* for file do [ -f "$file" ] || continue # do not process e.g. directories [ -r "$file" ] || continue # cannot read file - ignore it # Replace string, write output to temporary file. Terminate script in case of errors sed "s|$old|$new|g" -- "$file" > "$file"-new || exit # If the file has changed, overwrite original file. Otherwise remove copy if cmp -- "$file" "$file"-new >/dev/null 2>&1 then rm -- "$file"-new # file has not changed else mv -- "$file"-new "$file" # file has changed: overwrite original file fi done }}} If the code above is put into a script file (e.g. `chtext`), the resulting script can be used to change a text e.g. in all HTML files of the current and all subdirectories: {{{ find . -type f -name '*.html' -exec chtext {} \; }}} Many optimizations are possible: * use another `sed` separator character than '|', e.g. ^A (ASCII 0x01) * the [[UsingFind|find]] command above could use `+` instead of `\;` if available * the `chtext` script could be replaced with an `ed` command Note: `set -- ./*` in the code above is safe with respect to files whose names contain spaces. The expansion of `./*` by `set` is the same as the expansion done by `for`, and filenames will be preserved properly as individual parameters, and not broken into words on whitespace. A more sophisticated example of `chtext` is here: http://www.shelldorado.com/scripts/cmds/chtext |
This will run a single `ex` for as many files as fit on the command line. You can't do this with `ed`. |
How can I replace a string with another string in a file?
We're trying to edit files, which means we need a file editor. Most systems come with a range of editors, but POSIX defines only two that we can use: ed and ex.
Warning: A lot of bad advice on the Internet will recommend sed -i or perl -i for this. While on the surface it may look like these tools modify your files, what they really do is recreate your file and delete the original. This has many downsides and is often very risky. Open handles are destroyed, certain metadata will get lost, a lot of excess disk I/O is involved, and depending on the implementation, interrupted operations may cause your file to get lost. Just use a file editor.
ed is the standard UNIX command-based editor. ex is another POSIX command-based editor with syntax slightly similar to that of sed. Here are some commonly-used syntaxes for replacing the string old by the string new in a file named file.
ex -sc '%s/old/new/ge|x' file ed -s file <<< $'g/old/s//new/g\nw\nq'
If you need to edit multiple files, ex can easily be combined with find. ed is hard to call from find since it takes commands as input.
find . -type f -exec ex -sc '%s/old/new/ge|x' {} \;
find . -type f -exec bash -c 'printf "%s\n" "g/old/s//new/g" w q | ed -s "$1"' _ {} \;You may notice that this causes an ex/ed process per file. If your ex is provided by vim (NOT POSIX), you can optimize that into:
find . -type f -exec ex -sc 'argdo %s/old/new/ge|x' {} +This will run a single ex for as many files as fit on the command line. You can't do this with ed.