2519
Comment: Added the second awk solution. Karolisl
|
3832
Use if for the grep case too
|
Deletions are marked like this. | Additions are marked like this. |
Line 1: | Line 1: |
[[Anchor(faq79)]] == How can I grep for lines containing foo AND bar, foo OR bar? Or for files containing foo AND bar, possibly on separate lines? == |
<<Anchor(faq79)>> == How can I grep for lines containing foo AND bar, foo OR bar? Or for files containing foo AND bar, possibly on separate lines? Or files containing foo but NOT bar? == This is really four different questions, so we'll break this answer into parts. === foo AND bar on the same line === |
Line 11: | Line 14: |
It can also be done with one {{{egrep}}}, although (as you can probably guess) this doesn't really scale well to more than two patterns: | It can also be done with one {{{grep}}}, although (as you can probably guess) this doesn't really scale well to more than two patterns: |
Line 14: | Line 17: |
egrep 'foo.*bar|bar.*foo' | grep -E 'foo.*bar|bar.*foo' |
Line 17: | Line 20: |
If you prefer, you can achieve this in one {{{sed}}} or {{{awk}}} statement. (The {{{awk}}} example is probably the most scalable.) | If you prefer, you can achieve this in one {{{sed}}} or {{{awk}}} statement: |
Line 20: | Line 23: |
sed -n '/foo/{/bar/p}' | sed '/foo/!d; /bar/!d' |
Line 24: | Line 27: |
To match lines containing foo OR bar, {{{egrep}}} is the natural choice, but it can also be done with {{{sed}}}, {{{awk}}}, etc. | If you need to scale the awk solution to an arbitrary number of patterns, you can write a function like this: |
Line 27: | Line 30: |
egrep 'foo|bar' # some people prefer grep -E 'foo|bar' |
# POSIX multimatch() { # usage: multimatch pattern... awk ' BEGIN { for ( i = 1; i < ARGC; i++ ) a[i] = ARGV[i] ARGC = 1 } { for (i in a) if ($0 !~ a[i]) next }' "$@" } }}} |
Line 30: | Line 47: |
# This is another option, some people prefer: | === foo OR bar on the same line === There are lots of ways to match lines containing foo OR bar. `grep` can be given multiple patterns with `-e`: {{{ |
Line 34: | Line 55: |
{{{egrep}}} is the oldest and most portable form of the {{{grep}}} command using Extended Regular Expressions (EREs). {{{-E}}} is a POSIX-required switch. | Or you can separate the patterns with newlines: {{{ grep 'foo bar' }}} Or you can construct one pattern with {{{grep -E}}}: {{{ grep -E 'foo|bar' }}} (You can't use the `|` union operator with plain `grep`. `|` is only available in [[RegularExpression|Extended Regular Expressions]].) It can also be done with {{{sed}}}, {{{awk}}}, etc. {{{ awk '/foo|bar/' }}} The `awk` approach has the advantage of letting you use `awk`'s other features on the matched lines, such as extracting only certain fields. To match lines that do not contain "foo" AND do not contain "bar": {{{ grep -E -v 'foo|bar' }}} === foo AND bar in the same file, not necessarily on the same line === |
Line 39: | Line 89: |
grep -q foo "$myfile" && grep -q bar "$myfile" && echo "Found both" | if grep -q foo "$myfile" && grep -q bar "$myfile"; then printf 'Found both\n' fi }}} The double {{{grep -q}}} solution has the advantage of stopping each read whenever it finds a match; so if you have a huge file, but the matched words are both near the top, it will only read the first part of the file. Unfortunately, if the matches are near the bottom (worst case: very last line of the file), you may read the whole file two times. Another approach is to read the file once, keeping track of what you've seen as you go along. In awk: {{{ if awk '/foo/{a=1} /bar/{b=1} a&&b{exit} END{if(a&&b){exit 0};exit 1}' "$myfile"; then printf 'Found both\n' fi }}} It reads the file one time, stopping when both patterns have been matched. No matter what happens, the END block is then executed, and the exit status is set accordingly. If you want to do additional checking of the file's contents, this awk solution can be adapted quite easily. A perl one-liner that scales to any number of patterns, while also reading each input file only once: {{{ perl -e '@pat=("foo","bar"); local $/; L: for $f (@ARGV){open(FH,,$f); $a=<FH>; for(@pat){next L unless $a =~ $_} print "$f\n"}' |
Line 42: | Line 112: |
Another approach is to read the file once, keeping track of what you've seen as you go along. There are several ways to do this in awk - the first example reads the whole file, and, after it reads the whole file, it checks if both were found: | === foo but NOT bar in the same file, possibly on different lines === This is a variant of the previous case. The advantage here is that if we find "bar", we can stop reading. Here's an awk solution: |
Line 45: | Line 117: |
awk '/foo/ { foo=1 } /bar/ { bar=1 } END { if (foo && bar) print "found both" }' | awk '/foo/{good=1} /bar/{good=0;exit} END{exit !good}' |
Line 47: | Line 119: |
The second, more efficient one avoids reading the whole file by checking if the other string was already matched, and, if so, exiting: {{{ awk 'function found() { print "Found both!"; exit } /foo/ { a=1; if (b) found() } /bar/ { b=1; if (a) found() }' }}} The double {{{grep -q}}} solution has the advantage of stopping each read whenever it finds a match; so if you have a huge file, but the matched words are both near the top, it will only read the first part of the file. The first awk solution reads the whole file one time, while the second one stops reading the file at the second match; if you want to do additional checking of the file contents, the awk solution can be adapted far more readily. |
How can I grep for lines containing foo AND bar, foo OR bar? Or for files containing foo AND bar, possibly on separate lines? Or files containing foo but NOT bar?
This is really four different questions, so we'll break this answer into parts.
foo AND bar on the same line
The easiest way to match lines that contain both foo AND bar is to use two grep commands:
grep foo | grep bar grep foo "$myfile" | grep bar # for those who need the hand-holding
It can also be done with one grep, although (as you can probably guess) this doesn't really scale well to more than two patterns:
grep -E 'foo.*bar|bar.*foo'
If you prefer, you can achieve this in one sed or awk statement:
sed '/foo/!d; /bar/!d' awk '/foo/ && /bar/'
If you need to scale the awk solution to an arbitrary number of patterns, you can write a function like this:
# POSIX multimatch() { # usage: multimatch pattern... awk ' BEGIN { for ( i = 1; i < ARGC; i++ ) a[i] = ARGV[i] ARGC = 1 } { for (i in a) if ($0 !~ a[i]) next print }' "$@" }
foo OR bar on the same line
There are lots of ways to match lines containing foo OR bar. grep can be given multiple patterns with -e:
grep -e 'foo' -e 'bar'
Or you can separate the patterns with newlines:
grep 'foo bar'
Or you can construct one pattern with grep -E:
grep -E 'foo|bar'
(You can't use the | union operator with plain grep. | is only available in Extended Regular Expressions.)
It can also be done with sed, awk, etc.
awk '/foo|bar/'
The awk approach has the advantage of letting you use awk's other features on the matched lines, such as extracting only certain fields.
To match lines that do not contain "foo" AND do not contain "bar":
grep -E -v 'foo|bar'
foo AND bar in the same file, not necessarily on the same line
If you want to match files (rather than lines) that contain both "foo" and "bar", there are several possible approaches. The simplest (although not necessarily the most efficient) is to read the file twice:
if grep -q foo "$myfile" && grep -q bar "$myfile"; then printf 'Found both\n' fi
The double grep -q solution has the advantage of stopping each read whenever it finds a match; so if you have a huge file, but the matched words are both near the top, it will only read the first part of the file. Unfortunately, if the matches are near the bottom (worst case: very last line of the file), you may read the whole file two times.
Another approach is to read the file once, keeping track of what you've seen as you go along. In awk:
if awk '/foo/{a=1} /bar/{b=1} a&&b{exit} END{if(a&&b){exit 0};exit 1}' "$myfile"; then printf 'Found both\n' fi
It reads the file one time, stopping when both patterns have been matched. No matter what happens, the END block is then executed, and the exit status is set accordingly.
If you want to do additional checking of the file's contents, this awk solution can be adapted quite easily.
A perl one-liner that scales to any number of patterns, while also reading each input file only once:
perl -e '@pat=("foo","bar"); local $/; L: for $f (@ARGV){open(FH,,$f); $a=<FH>; for(@pat){next L unless $a =~ $_} print "$f\n"}'
foo but NOT bar in the same file, possibly on different lines
This is a variant of the previous case. The advantage here is that if we find "bar", we can stop reading. Here's an awk solution:
awk '/foo/{good=1} /bar/{good=0;exit} END{exit !good}'