Differences between revisions 26 and 27
Revision 26 as of 2013-09-12 10:24:04
Size: 10918
Editor: geirha
Comment: no, `if` cannot do pattern matching. It really is case, though it's a keyword, not a builtin.
Revision 27 as of 2014-11-28 07:09:55
Size: 5
Editor: 148
Comment:
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
## page was renamed from BashGuide/04.Patterns
[[BashGuide/Parameters|<- Parameters]] | [[BashGuide/TestsAndConditionals|Tests and Conditionals ->]]

----
<<Anchor(StartOfContent)>>

= Patterns =
[[BASH]] offers three different kinds of ''pattern matching''. Pattern matching serves two roles in the shell: selecting filenames within a directory, or determining whether a string conforms to a desired format.

On the command line you will mostly use ''globs''. These are a fairly straight-forward form of patterns that can easily be used to match a range of files, or to check variables against simple rules.

The second type of pattern matching involves ''extended globs'', which allow more complicated expressions than regular globs.

Since version `3.0`, Bash also supports ''regular expression'' patterns. These will be useful mainly in scripts to test user input or parse data. (You can't use a regular expression to select filenames; only globs and extended globs can do that.)

--------
 . ''Pattern'': A pattern is a string with a special format designed to match filenames, or to check, classify or validate data strings.

--------
<<Anchor(Glob_Patterns)>>

== Glob Patterns ==
[[glob|Globs]] are a very important concept in Bash, if only for their incredible convenience. Properly understanding globs will benefit you in many ways. Globs are basically patterns that can be used to match filenames or other strings.

Globs are composed of normal characters and metacharacters. Metacharacters are characters that have a special meaning. These are the metacharacters that can be used in globs:

 * '''*''': Matches any string, including the null string.
 * '''?''': Matches any single character.
 * '''[...]''': Matches any one of the enclosed characters.

Globs are implicitly ''anchored'' at both ends. What this means is that a glob must match a ''whole'' string (filename or data string). A glob of `a*` will not match the string `cat`, because it only matches the `at`, not the whole string. A glob of `ca*`, however, would match `cat`.

Here's an example of how we can use glob patterns to expand to filenames:

{{{
$ ls
a abc b c
$ echo *
a abc b c
$ echo a*
a abc
}}}
Bash sees the glob, for example `a*`. It ''expands'' this glob, by looking in the current directory and matching it against all files there. Any filenames that match the glob are gathered up and sorted, and then the list of filenames is used in place of the glob. As a result, the statement `echo a*` is replaced by the statement `echo a abc`, which is then executed.

When a glob is used to match ''filenames'', the `*` and `?` characters cannot match a slash (`/`) character. So, for instance, the glob `*/bin` might match `foo/bin` but it cannot match `/usr/local/bin`. When globs match ''patterns'', the `/` restriction is removed.

Bash performs filename expansions ''after'' word splitting has already been done. Therefore, filenames generated by a glob will not be split; they will always be handled correctly. For example:

{{{
$ touch "a b.txt"
$ ls
a b.txt
$ rm *
$ ls
}}}
Here, `*` is expanded into the single filename "`a b.txt`". This filename will be passed as a single argument to `rm`. Using globs to enumerate files is '''always''' a better idea than using {{{`ls`}}} for that purpose. Here's an example with some more complex syntax which we will cover later on, but it will illustrate the reason very well:

{{{
$ ls
a b.txt
$ for file in `ls`; do rm "$file"; done
rm: cannot remove `a': No such file or directory
rm: cannot remove `b.txt': No such file or directory
$ for file in *; do rm "$file"; done
$ ls
}}}
Here we use the `for` command to go through the output of the `ls` command. The `ls` command prints the string `a b.txt`. The `for` command splits that string into words over which it iterates. As a result, `for` iterates over first `a`, and then `b.txt`. Naturally, this is '''not''' what we want. The glob, however, expands in the proper form. It results in the string "`a b.txt`", which `for` takes as a single argument.

In addition to filename expansion, globs may also be used to check whether data matches a specific format. For example, we might be given a filename, and need to take different actions depending on its extension:

{{{
$ filename="somefile.jpg"
$ if [[ $filename = *.jpg ]]; then
> echo "$filename is a jpeg"
> fi
somefile.jpg is a jpeg
}}}
The `[[` keyword and the `case` keyword (which we will discuss in more detail later) both offer the opportunity to check a string against a glob -- either regular globs, or extended globs, if the latter have been enabled.

--------
 . '''Good Practice: <<BR>> You should always use globs instead of `ls` (or similar) to enumerate files. Globs will always expand safely and minimize the risk for bugs. <<BR>> You can sometimes end up with some very weird filenames. Most scripts aren't tested against all the odd cases that they may end up being used with. Don't let your script be one of those!'''

----
 . '''In The Manual: [[http://www.gnu.org/software/bash/manual/bashref.html#Pattern-Matching|Pattern Matching]]'''

----
 . '''In the FAQ: <<BR>> [[BashFAQ/016|How can I use a logical AND/OR/NOT in a shell pattern (glob)?]]'''

----
 . ''Glob'': A glob is a string that can match certain strings or filenames.

--------

== Extended Globs ==
Bash also supports a feature called ''Extended Globs''. These globs are more powerful in nature; technically, they are equivalent to regular expressions, although the syntax looks different than most people are used to. This feature is turned off by default, but can be turned on with the `shopt` command, which is used to toggle '''sh'''ell '''opt'''ions:

{{{
$ shopt -s extglob
}}}
 * '''?(list)''': Matches zero or one occurrence of the given patterns.
 * '''*(list)''': Matches zero or more occurrences of the given patterns.
 * '''+(list)''': Matches one or more occurrences of the given patterns.
 * '''@(list)''': Matches one of the given patterns.
 * '''!(list)''': Matches anything except one of the given patterns.

The list inside the parentheses is a list of globs or extended globs separated by the `|` character. Here's an example:

{{{
$ ls
names.txt tokyo.jpg california.bmp
$ echo !(*jpg|*bmp)
names.txt
}}}
Our extended glob expands to anything that does not match the `*jpg` or the `*bmp` pattern. Only the text file passes for that, so it is expanded.

<<Anchor(Regular_Expressions)>>

== Regular Expressions ==
Regular expressions (regex) are similar to ''Glob Patterns'', but they can only be used for pattern matching, not for filename matching. Since 3.0, Bash supports the `=~` operator to the `[[` keyword. This operator matches the string that comes before it against the regex pattern that follows it. When the string matches the pattern, `[[` returns with an exit code of `0` ("true"). If the string does not match the pattern, an exit code of `1` ("false") is returned. In case the pattern's syntax is invalid, `[[` will abort the operation and return an exit code of `2`.

Bash uses the ''Extended Regular Expression'' (`ERE`) dialect. We will not cover regexes in depth in this guide, but if you are interested in this concept, please read up on RegularExpression, or [[http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap09.html#tag_09_04|Extended Regular Expressions]].

''Regular Expression'' patterns that use capturing groups (parentheses) will have their captured strings assigned to the `BASH_REMATCH` variable for later retrieval.

Let's illustrate how regex can be used in Bash:

{{{
$ langRegex='(..)_(..)'
$ if [[ $LANG =~ $langRegex ]]
> then
> echo "Your country code (ISO 3166-1-alpha-2) is ${BASH_REMATCH[2]}."
> echo "Your language code (ISO 639-1) is ${BASH_REMATCH[1]}."
> else
> echo "Your locale was not recognised"
> fi
}}}
Be aware that regex parsing in Bash has changed between releases `3.1` and `3.2`. Before `3.2` it was safe to wrap your regex pattern in quotes but this has changed in `3.2`. Since then, regex should always be unquoted. You should protect any special characters by escaping it using a backslash. The best way to always be compatible is to put your regex in a variable and expand that variable in `[[` without quotes, as we showed above.

--------
 . '''Good Practice: <<BR>> Since the way regex is used in `3.2` is also valid in `3.1` we ''highly'' recommend you just never quote your regex. Remember to keep special characters properly escaped!'''
 . ''' For cross-compatibility (to avoid having to escape parentheses, pipes and so on) use a variable to store your regex, e.g. {{{re='^\*( >| *Applying |.*\.diff|.*\.patch)'; [[ $var =~ $re ]]}}} This is much easier to maintain since you only write ERE syntax and avoid the need for shell-escaping, as well as being compatible with all 3.x BASH versions.'''
 . See also [[http://tiswww.case.edu/php/chet/bash/FAQ|Chet Ramey's Bash FAQ]], section E14.

----
 . '''In The Manual: [[http://www.daemon-systems.org/man/regex.3.html|Regex(3)]]'''

----
 . '''In the FAQ: <<BR>> [[BashFAQ/066|I want to check if [[ $var == foo or $var == bar or $var == more ... without repeating $var n times.]]'''

----
 . ''Regular Expression'': A regular expression is a more complex pattern that can be used to match specific strings (but unlike globs cannot expand to filenames).

--------

== Brace Expansion ==
Then, there is ''Brace Expansion''. Brace Expansion technically does not fit in the category of patterns, but it is similar. Globs only expand to actual filenames, but brace expansions will expand to any possible permutation of their contents. Here's how they work:

{{{
$ echo th{e,a}n
then than
$ echo {/home/*,/root}/.*profile
/home/axxo/.bash_profile /home/lhunath/.profile /root/.bash_profile /root/.profile
$ echo {1..9}
1 2 3 4 5 6 7 8 9
$ echo {0,1}{0..9}
00 01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19
}}}
The brace expansion is replaced by a list of words, just like a glob is. However, these words aren't necessarily filenames, and they are not sorted (`than` would have come before `then` if they were).

Brace expansion happens ''before'' filename expansion. In the second `echo` command above, we used a combination of brace expansion and globs. The brace expansion goes first, and we get:

{{{
$ echo /home/*/.*profile /root/.*profile
}}}
After the brace expansion, the globs are expanded, and we get the filenames as the final result.

Brace expansions can only be used to generate lists of words. They cannot be used for pattern matching.
<<Anchor(EndOfContent)>>

--------
[[BashGuide/Parameters|<- Parameters]] | [[BashGuide/TestsAndConditionals|Tests and Conditionals ->]]
cas

cas

BashGuide/Patterns (last edited 2016-01-15 10:08:43 by google-proxy-66-249-93-205)