5452
Comment:
|
263
all gone
|
Deletions are marked like this. | Additions are marked like this. |
Line 2: | Line 2: |
== How can i perform a substitution (s/foo/bar/) safely, without treating either value as a regular expression? == Sed is not the right tool for this. At best, it will be an escaping nightmare, and extremely prone to bugs. |
== How can i perform a substitution with arbitrary values ("s/$foo/$bar/") safely, without treating either value as a regular expression or worrying about other special characters? == |
Line 5: | Line 4: |
First, what are we performing the substitution on? If it's a string, it can be done very simply with a parameter expansion. {{{ var='some string' printf '%s\n' "${var//some/another}" }}} This is discussed in more detail in [[BashFAQ/100|Faq #100]]. If it's a file or stream, things get a bit trickier. One way to accomplish this would be to combine the previous method with [[BashFAQ/001|Faq #1]]. {{{ # file while IFS= read -r line; do printf '%s\n' "${line//foo/bar}" done < file # command output while IFS= read -r line; do printf '%s\n' "${line//foo/bar}" done < <(my_command) my_command | while IFS= read -r line; do printf '%s\n' "${line//foo/bar}" done }}} The second of the two command examples there creates a subshell. See [[BashFAQ/024|Faq #24]] for more information on that. Both of the above examples print to stdout. Neither actually edits the file in place. Of course this could be resolved with something like: {{{ while IFS= read -r line; do printf '%s\n' "${line//foo/bar}" done < file > new_file && mv new_file file }}} On large data sets, you'll notice that this is quite slow. The following functions use awk, and are quite a bit faster: {{{ # usage: sub_literal STR REP # Replaces the first instance (on each line) of STR with REP, treating them as # literal strings and not regexes. Reads stdin and writes to stdout. # Similar to sed 's/STR/REP/' sub_literal() { # string manip needed to escape '\'s, so awk doesn't expand '\n' and such awk -v str="${1//\\/\\\\}" -v rep="${2//\\/\\\\}" ' # get the length of the search string BEGIN { len = length(str); } # if the search string is in the line (i = index($0, str)) { # replace the first occurance with rep $0 = substr($0, 1, i-1) rep substr($0, i + len); } # print each line 1 ' } # usage: sub_literal_f STR REP FILE # Replaces the first instance (on each line) of STR with REP in FILE, treating # them as literal strings and not regexes. # Similar to sed -i 's/STR/REP/' FILE sub_literal_f() { local tmp if ! [[ -f $3 && -r $3 && -w $3 ]]; then printf '%s does not exist or is not readable or writable\n' "$3" >&2 return 1 fi trap 'rm -rf "$tmp"' RETURN tmp=$(mktemp) && cp "$3" "$tmp" || return # string manip needed to escape '\'s, so awk doesn't expand '\n' and such awk -v str="${1//\\/\\\\}" -v rep="${2//\\/\\\\}" ' # get the length of the search string BEGIN { len = length(str); } # if the search string is in the line (i = index($0, str)) { # replace the first occurance with rep $0 = substr($0, 1, i-1) rep substr($0, i + len); } # print each line 1 ' "$tmp" > "$3" } # usage: gsub_literal STR REP # Replaces all instances of STR with REP, treating them as literal strings # and not regexes. Reads stdin and writes to stdout # Similar to sed 's/STR/REP/g' gsub_literal() { # string manip needed to escape '\'s, so awk doesn't expand '\n' and such awk -v str="${1//\\/\\\\}" -v rep="${2//\\/\\\\}" ' # get the length of the search string BEGIN { len = length(str); } { # empty the output string out = ""; # continue looping while the search string is in the line while (i = index($0, str)) { # append everything up to the search string, and the replacement string out = out substr($0, 1, i-1) rep; # remove everything up to and including the first instance of the # search string from the line $0 = substr($0, i + len); } # append whatever is left out = out $0; print out; } ' } # usage: gsub_literal_f STR REP FILE # Replaces all instances of STR with REP in FILE, treating them as literal # strings and not regexes. # Similar to sed -i 's/STR/REP/g' FILE gsub_literal_f() { local tmp if ! [[ -f $3 && -r $3 && -w $3 ]]; then printf '%s does not exist or is not readable or writable\n' "$3" >&2 return 1 fi trap 'rm -rf "$tmp"' RETURN tmp=$(mktemp) && cp "$3" "$tmp" || return # string manip needed to escape '\'s, so awk doesn't expand '\n' and such awk -v str="${1//\\/\\\\}" -v rep="${2//\\/\\\\}" ' # get the length of the search string BEGIN { len = length(str); } { # empty the output string out = ""; # continue looping while the search string is in the line while (i = index($0, str)) { # append everything up to the search string, and the replacement string out = out substr($0, 1, i-1) rep; # remove everything up to and including the first instance of the # search string from the line $0 = substr($0, i + len); } # append whatever is left out = out $0; print out; } ' "$tmp" > "$3" } }}} The mktemp(1) command used in the *_f functions above is not completely portable. While it will work on most systems, more information on safely creating temp files can be found in [[BashFAQ/062|Faq #62.]] |
This FAQ has been merged with [[BashFAQ/021|Faq #21]]. |
How can i perform a substitution with arbitrary values ("s/$foo/$bar/") safely, without treating either value as a regular expression or worrying about other special characters?
This FAQ has been merged with Faq #21.