Differences between revisions 22 and 23
Revision 22 as of 2009-12-30 18:06:39
Size: 3452
Editor: MatthiasPopp
Comment:
Revision 23 as of 2010-09-27 13:33:20
Size: 3807
Editor: Lhunath
Comment: ed > sed
Deletions are marked like this. Additions are marked like this.
Line 3: Line 3:
{{{sed}}} is a good command to replace strings, e.g. {{{ed}}} is the standard UNIX command-based editor. Here's three commonly-used syntaxes for replacing the string `olddomain.com` by the string `newdomain.com` in a file named `file`. All three commands do the same although the last two incur the minor additional overhead of a subshell.
Line 6: Line 6:
    sed 's/olddomain\.com/newdomain.com/g' input > output     # ed -s file <<< $'s/olddomain\.com/newdomain.com/g\nw'
    # printf '%s\n' 's/olddomain\.com/newdomain.com/g' w | ed -s file
    # printf 's/olddomain\.com/newdomain.com/g\nw' | ed -s file
Line 8: Line 10:
'''Note that input and output must differ.'''
Line 13: Line 14:
    for i in ./*; do
        sed 's/old/new/g' "$i" > atempfile && mv atempfile "$i"
    for file in ./*; do
        ed <<< $'s/old/new/g\nw' "$file"
Line 18: Line 19:
GNU sed 4.x has a special {{{-i}}} flag which makes the loop and temp file unnecessary: {{{sed}}} is a '''Stream EDitor''', not a '''file'' editor. Nevertheless, people everywhere tend to abuse it for trying to edit files. It doesn't edit files. GNU's `sed` (and some BSD `sed`'s) have a `-i` option that makes a copy and replaces the original file with the copy. An expensive operation, but if you enjoy unportable code, I/O overhead and bad side effects (such as destroying symlinks), this would be an option:
Line 21: Line 22:
      sed -i 's/old/new/g' ./*       sed -i 's/old/new/g' ./*   # GNU
      sed -i '' 's/old/new/g' ./* # BSD
Line 23: Line 25:

On some (but not all) BSD systems, sed has a {{{-i}}} flag as well, but it takes a mandatory argument. The above example then becomes

{{{
      sed -i '' 's/old/new/g' ./*
}}}

which in turn does not work with GNU sed. Effectively, whenever portability matters, {{{sed -i}}} should be avoided.
Line 38: Line 32:
Recursively (requires GNU or BSD {{{find}}}): Recursively using find:
Line 41: Line 35:
    find . -type f -print0 | xargs -0 perl -pi -e 's/old/new/g'     find . -type f -exec perl -pi -e 's/old/new/g' {} +
Line 54: Line 48:
    find . -type f -print0 | xargs -0 perl -i.bak -pne \
 's/\bunsigned\b(?!\s+(int|short|long|char))/unsigned long/g'
    find . -type f -exec perl -i.bak -pne \
 's/\bunsigned\b(?!\s+(int|short|long|char))/unsigned long/g' {} +

How can I replace a string with another string in all files?

ed is the standard UNIX command-based editor. Here's three commonly-used syntaxes for replacing the string olddomain.com by the string newdomain.com in a file named file. All three commands do the same although the last two incur the minor additional overhead of a subshell.

    # ed -s file <<< $'s/olddomain\.com/newdomain.com/g\nw'
    # printf '%s\n' 's/olddomain\.com/newdomain.com/g' w | ed -s file
    # printf 's/olddomain\.com/newdomain.com/g\nw' | ed -s file

To replace a string in all files of the current directory:

    for file in ./*; do
        ed <<< $'s/old/new/g\nw' "$file"
    done

sed is a Stream EDitor, not a file editor. Nevertheless, people everywhere tend to abuse it for trying to edit files. It doesn't edit files. GNU's sed (and some BSD sed's) have a -i option that makes a copy and replaces the original file with the copy. An expensive operation, but if you enjoy unportable code, I/O overhead and bad side effects (such as destroying symlinks), this would be an option:

      sed -i 's/old/new/g' ./*   # GNU
      sed -i '' 's/old/new/g' ./*  # BSD

Those of you who have perl 5 can accomplish the same thing using this code:

    perl -pi -e 's/old/new/g' ./*

Recursively using find:

    find . -type f -exec perl -pi -e 's/old/new/g' {} +

If you want to delete lines instead of making substitutions:

    perl -ni -e 'print unless /foo/' ./*
    # Deletes any line containing the perl regex foo

To replace for example all "unsigned" with "unsigned long", if it is not "unsigned int" or "unsigned long" ...:

    find . -type f -exec perl -i.bak -pne \
        's/\bunsigned\b(?!\s+(int|short|long|char))/unsigned long/g' {} +

Finally, for those of you with none of the useful things above, here's a script that may be useful:

    #!/bin/sh
    # chtext - change text in several files

    # neither string may contain '|' unquoted
    old='olddomain\.com'
    new='newdomain\.com'

    # if no files were specified on the command line, use all files:
    [ $# -lt 1 ] && set -- ./*

    for file
    do
        [ -f "$file" ] || continue # do not process e.g. directories
        [ -r "$file" ] || continue # cannot read file - ignore it
        # Replace string, write output to temporary file. Terminate script in case of errors
        sed "s|$old|$new|g" -- "$file" > "$file"-new || exit
        # If the file has changed, overwrite original file. Otherwise remove copy
        if cmp -- "$file" "$file"-new >/dev/null 2>&1
        then rm -- "$file"-new              # file has not changed
        else mv -- "$file"-new "$file"      # file has changed: overwrite original file
        fi
    done

If the code above is put into a script file (e.g. chtext), the resulting script can be used to change a text e.g. in all HTML files of the current and all subdirectories:

    find . -type f -name '*.html' -exec chtext {} \;

Many optimizations are possible:

  • use another sed separator character than '|', e.g. ^A (ASCII 0x01)

  • the find command above could use either xargs or the built-in xargs of POSIX find

Note: set -- ./* in the code above is safe with respect to files whose names contain spaces. The expansion of ./* by set is the same as the expansion done by for, and filenames will be preserved properly as individual parameters, and not broken into words on whitespace.

A more sophisticated example of chtext is here: http://www.shelldorado.com/scripts/cmds/chtext


CategoryShell

BashFAQ/021 (last edited 2022-11-03 23:42:27 by GreyCat)