Diff for "BashFAQ/021"

How can I replace a string with another string in all files?

ed is the standard UNIX command-based editor. Here are some commonly-used syntaxes for replacing the string olddomain.com by the string newdomain.com in a file named file. All four commands do the same thing, with varying degrees of portability and efficiency:

    # Bash
    ed -s file <<< $'g/olddomain\\.com/s//newdomain.com/g\nw\nq'

    # Bourne (with printf)
    printf '%s\n' 'g/olddomain\.com/s//newdomain.com/g' w q | ed -s file

    printf 'g/olddomain\\.com/s//newdomain.com/g\nw\nq' | ed -s file

    # Bourne (without printf)
    ed -s file <<!
    g/olddomain\\.com/s//newdomain.com/g
    w
    q
    !

To replace a string in all files of the current directory:

    for file in ./*; do
        [[ -f $file ]] && ed -s "$file" <<< $'g/old/s//new/g\nw\nq'
    done

To do this recursively, the easy way would be to enable globstar in bash 4 (shopt -s globstar, a good idea to put this in your ~/.bashrc) and use:

    for file in ./**/*; do
        [[ -f $file ]] && ed -s "$file" <<< $'g/old/s//new/g\nw\nq'
    done

If you don't have bash 4, you can use find. Unfortunately, it's a bit tedious to feed ed stdin for each file hit:

    find . -type f -exec bash -c 'printf "%s\n" "g/old/s//new/g" w q | ed -s "$1"' _ {} \;

sed is a Stream EDitor, not a file editor. Nevertheless, people everywhere tend to abuse it for trying to edit files. It doesn't edit files. GNU sed (and some BSD seds) have a -i option that makes a copy and replaces the original file with the copy. An expensive operation, but if you enjoy unportable code, I/O overhead and bad side effects (such as destroying symlinks), this would be an option:

    sed -i    's/old/new/g' ./*  # GNU
    sed -i '' 's/old/new/g' ./*  # BSD
    for file in ./*              # Other
    do
        [ -f "$file" ] &&
            sed 's/old/new/g' "$file" > "$file~" &&
            mv "$file~" "$file"
    done

Those of you who have perl 5 can accomplish the same thing using this code:

    perl -pi -e 's/old/new/g' ./*

Recursively using find:

    find . -type f -exec perl -pi -e 's/old/new/g' {} \;

If you want to delete lines instead of making substitutions:

    # Deletes any line containing the perl regex foo
    perl -ni -e 'print unless /foo/' ./*

To replace for example all "unsigned" with "unsigned long", if it is not "unsigned int" or "unsigned long" ...:

    find . -type f -exec perl -i.bak -pne \
        's/\bunsigned\b(?!\s+(int|short|long|char))/unsigned long/g' {} \;

Finally, for those of you who shun ed and have none of the useful things above, here's a script that may be useful:

    #!/bin/sh
    # chtext - change text in several files

    # neither string may contain '|' unquoted
    old='olddomain\.com'
    new='newdomain\.com'

    # if no files were specified on the command line, use all files:
    [ $# -lt 1 ] && set -- ./*

    for file
    do
        [ -f "$file" ] || continue # do not process e.g. directories
        [ -w "$file" ] || continue # cannot write to file - ignore it
        # Replace string, write output to temporary file. Terminate script in case of errors
        sed "s|$old|$new|g" -- "$file" > "$file"-new || exit
        # If the file has changed, overwrite original file. Otherwise remove copy
        if cmp -- "$file" "$file"-new >/dev/null 2>&1
        then rm -- "$file"-new              # file has not changed
        else mv -- "$file"-new "$file"      # file has changed: overwrite original file
        fi
    done

If the code above is put into a script file (e.g. chtext), the resulting script can be used to change a text e.g. in all HTML files of the current and all subdirectories:

    find . -type f -name '*.html' -exec chtext {} \;

Many optimizations are possible:

• use another sed separator character than '|', e.g. ^A (ASCII 0x01)

• the find command above could use + instead of \; if available

• the chtext script could be replaced with an ed command

Note: set -- ./* in the code above is safe with respect to files whose names contain spaces. The expansion of ./* by set is the same as the expansion done by for, and filenames will be preserved properly as individual parameters, and not broken into words on whitespace.

A more sophisticated example of chtext is here: http://www.shelldorado.com/scripts/cmds/chtext

CategoryShell