31686
Comment:
|
38282
|
Deletions are marked like this. | Additions are marked like this. |
Line 1: | Line 1: |
[[Anchor(Tests_And_Conditionals)]] == Tests and Conditionals == |
#pragma section-numbers 2 [[BashGuide/Arrays|<- Arrays]] | [[BashGuide/InputAndOutput|Input and Output ->]] ---- = Tests and Conditionals = <<TableOfContents>> |
Line 7: | Line 10: |
[[Anchor(Exit_Status)]] === Exit Status === Every application results in an exit code whenever it terminates. This exit code is used by whatever application started it to evaluate whether everything went OK. This exit code is like a return value from functions. It's an integer between 0 and 255 (inclusive). Convention dictates that we use 0 to denote success, and any other number to denote failure of some sort. The specific number is entirely application-specific, and is used to hint as to what exactly went wrong. |
<<Anchor(Exit_Status)>> == Exit Status == Every command results in an exit code whenever it terminates. This exit code is used by whatever application started it to evaluate whether everything went OK. This exit code is like a return value from functions. It's an integer between 0 and 255 (inclusive). Convention dictates that we use 0 to denote success, and any other number to denote failure of some sort. The specific number is entirely application-specific, and is used to hint as to what exactly went wrong. |
Line 16: | Line 17: |
'''From the `ping` manual:''' . '''If ping does not receive any reply packets at all it will exit with code 1. If a packet count and deadline are both specified, and fewer than count packets are received by the time the deadline has arrived, it will also exit with code 1. On other error it exits with code 2. Otherwise it exits with code 0. This makes it possible to use the exit code to see if a host is alive or not.''' The parameter `?` shows us the exit code of the last foreground process that terminated. Let's play around a little with `ping` to see its exit codes: |
'''From the Linux `ping` manual:''' . '''If ping does not receive any reply packets at all it will exit with code 1. If a packet count and deadline are both specified, and fewer than count packets are received by the time the deadline has arrived, it will also exit with code 1. On other error it exits with code 2. Otherwise it exits with code 0. This makes it possible to use the exit code to see if a host is alive or not.''' The special parameter `?` shows us the exit code of the last foreground process that terminated. Let's play around a little with `ping` to see its exit codes: |
Line 34: | Line 36: |
. '''Good Practice: [[BR]] You should make sure that your scripts always return a non-zero exit code if something unexpected happened in their execution. You can do this with the `exit` builtin:''' | . '''Good Practice: <<BR>> You should make sure that your scripts always return a non-zero exit code if something unexpected happened in their execution. You can do this with the `exit` builtin:''' |
Line 39: | Line 41: |
. '''In The Manual: [http://www.gnu.org/software/bash/manual/bashref.html#SEC52 Exit Status]''' | . '''In The Manual: [[http://www.gnu.org/software/bash/manual/bashref.html#Exit-Status|Exit Status]]''' |
Line 44: | Line 46: |
[[Anchor(Control_Operators)]] === Control Operators === |
<<Anchor(Control_Operators)>> == Control Operators (&& and ||) == |
Line 56: | Line 56: |
This simple example has two commands, `mkdir d` and `cd d`. You could easily just use a semi-colon there to separate both commands and execute them sequentially; but we want something more. In the above example, ["BASH"] will execute `mkdir d`, then `&&` will check the result of the `mkdir` application as it finishes. If the `mkdir` application resulted in a success (exit code 0), then `&&` will execute the next command, `cd d`. If `mkdir d` failed, and returned a non-0 exit code, `&&` will skip the next command, and we will stay in the current directory. | This simple example has two commands, `mkdir d` and `cd d`. You could easily just use a semi-colon there to separate both commands and execute them sequentially; but we want something more. In the above example, [[BASH]] will execute `mkdir d`, then `&&` will check the result of the `mkdir` application as it finishes. If the `mkdir` application resulted in a success (exit code 0), then `&&` will execute the next command, `cd d`. If `mkdir d` failed, and returned a non-0 exit code, `&&` will skip the next command, and we will stay in the current directory. |
Line 86: | Line 86: |
# There is a problem here.... | |
Line 92: | Line 93: |
The next section will show a safer way to put both "AND" and "OR" conditions after a command. -------- . '''Good Practice: [[BR]] It's best not to get overcourageous when dealing with conditional operators. They can make your script hard to understand, especially for a person that's assigned to maintain it and didn't write it himself.''' ---- . '''In The Manual: [http://www.gnu.org/software/bash/manual/bashref.html#SEC18 List of Commands]''' |
See the next two sections for two ways of dealing with this. -------- . '''Good Practice: <<BR>> It's best not to get overzealous when dealing with conditional operators. They can make your script hard to understand, especially for a person that's assigned to maintain it and didn't write it himself.''' ---- . '''In The Manual: [[http://www.gnu.org/software/bash/manual/bashref.html#Lists|Lists of Commands]]''' |
Line 103: | Line 104: |
[[Anchor(Conditional_Blocks)]] === Conditional Blocks === `if` is a shell builtin that executes a command, and checks that command's exit code to see whether its execution was successful. Depending on that exit code, `if` executes a specific block of code. |
<<Anchor(Grouping)>> == Grouping Statements == Using conditional operators is easy and terse if we want to run a statement on the condition that another succeeds. Things get a bit more dangerous, though, when we want to run multiple statements if a condition holds true, or if we need to evaluate multiple conditions. Suppose you want to delete a file if it contains a certain word but also doesn't contain another word. Using grep, we translate these conditions to: {{{ grep -q word "$file" # exit status 0 (success) if "$file" contains 'word' ! grep -q "another word" "$file" # exit status 0 (success) if "$file" does not contain 'another word' }}} We use `-q` (quiet) on grep because we don't want it to output the lines that match, we just want the exit code to be set. Now, to put these conditions together and delete the file as a result of both holding true, we could use '''Conditional Operators''': {{{ $ grep -q word "$file" && ! grep -q "another word" "$file" && rm "$file" }}} This works great. Now, imagine we want to show an error message in case the deletion of the file failed: {{{ $ grep -q word "$file" && ! grep -q "another word" "$file" && rm "$file" || echo "Couldn't delete: $file" >&2 }}} This looks OK, at first sight. If `rm`'s exit code is not `0` (success), then the `||` operator will trigger the next command and `echo` the error message (`>&2`: to standard error). ''However'', as pointed out at the end of the previous section on ''Conditional Operators'', should either `grep` command fail, the `&&` operator that follows it will only skip the ''next'' statement, but not the one after that. Imagine the first `grep` failing. The `&&` that follows it will make sure the second grep is not tried. [[BASH]] will try the statement after that, though. The next statement is the `rm` statement, but since it is also preceded by a `&&` operator, it will also not be tried (remember, our exit code still indicates failure since the last command that was executed was our failed `grep`). [[BASH]] will move on to the next statement. This time, we see the `echo` statement, and it's preceded by a `||` operator. Since our exit status indicates failure, the `||` operator lets us through, and the `echo` statement is executed. As a result, the failed grep causes the error message `Couldn't delete: $file` to be shown on the screen. That's not what we want. Doesn't sound lethal when it's just a wrong error message you receive, but if you're not careful, this '''will''' eventually happen on more dangerous code. You wouldn't want to accidentally delete files or overwrite files as a result of a failure in your logic! The failure in our logic is in the fact that we '''want''' the `rm` and the `echo` statements to belong together. The `echo` is related to the `rm`, not to the `grep`s. So what we need, is to group them. Grouping is done using curly braces: {{{ $ grep -q word "$file" && ! grep -q "another word" "$file" && { rm "$file" || echo "Couldn't delete: $file" >&2; } }}} (Note: don't forget that you need a semicolon or newline before the closing curly brace!) Now we've grouped the `rm` and `echo` command together. That effectively means: before [[BASH]] has gone into this group, it is considered '''one statement''' instead of several. Going back to our situation of the first `grep` failing, instead of [[BASH]] trying the `&& rm "$file"` statement, it will now try the `&& { ... }` statement. Since it is preceded by a `&&` and the last command it ran failed (the failed `grep`), it will skip this statement and move on. Command grouping can be used for more things than just grouping statements for ''Conditional Operators''. We may also want to group them so that we can redirect input to a group of statements instead of just one: {{{ { read firstLine read secondLine while read otherLine; do something done } < file }}} Here we're redirecting `file` to a group of commands that read input. The file will be opened when the curly brace opens, stay open for the duration of it, and be closed when the curly brace closes. This way, we can keep sequentially reading lines from it with multiple commands. -------- <<Anchor(Conditional_Blocks)>> == Conditional Blocks (if, test and [[) == `if` is a shell keyword that executes a command, and checks that command's exit code to see whether its execution was successful. Depending on that exit code, `if` executes a specific block of code. |
Line 116: | Line 176: |
Here you see the basic outline of an ''if-statement''. We start by calling `if` with the argument `true`. `true` is a built-in command that always ends successfully. `if` runs that command, and once the command is done, `if` checks the exit code. Since `true` always exits successfully, `if` continues to the `then`-block, and executes that code. Should the `true` application have failed somehow, and returned an unsuccessful exit code, the `if` statement would have skipped the `then` code, and executed the `else` code block instead. There are commands that can help us a lot in doing conditional checks. They are `[` (also named `test`) and `[[`. `[` is a normal application that reads its arguments and does some checks with them. `[[` is much like `[`, but it's special, and it offers far more versatility. Let's get practical: |
Here you see the basic outline of an ''if-statement''. We start by calling `if` with the command `true`. `true` is a built-in command that always ends successfully. `if` runs that command, and once the command is done, `if` checks the exit code. Since `true` always exits successfully, `if` continues to the `then`-block, and executes that code. Should the `true` command have failed somehow, and returned an unsuccessful exit code, the `if` statement would have skipped the `then` code, and executed the `else` code block instead. Different people have different preferred styles for writing `if` statements. Here are some of the common styles: {{{ if COMMANDS then OTHER COMMANDS fi if COMMANDS then OTHER COMMANDS fi if COMMANDS; then OTHER COMMANDS fi }}} There are commands that can help us a lot in doing conditional checks. They are `[` (also named `test`) and `[[`. `[` is a normal command that reads its arguments and does some checks with them. `[[` is much like `[`, but it's special (a shell keyword), and it offers far more versatility. Let's get practical: |
Line 127: | Line 204: |
`if` executes the command `[` with the arguments '`a`', '`=`', '`b`' and '`]`'. `[` uses these arguments to determine what must be checked. In this case, it checks whether the string 'a' is identical to the string 'b', and if this is the case, it will exit successfully. However, since we know this is not the case, `[` will not exit successfully (its exit code will be 1). `if` sees that `[` terminated unsuccessfully and executes the code in the `else` block. | `if` executes the command `[` (you don't '''need''' `if` to run the `[` command!) with the arguments '`a`', '`=`', '`b`' and '`]`'. `[` uses these arguments to determine what must be checked. In this case, it checks whether the string `a` (the first argument) is equal (the second argument) to the string `b` (the third argument), and if this is the case, it will exit successfully. However, since we know this is not the case, `[` will not exit successfully (its exit code will be 1). `if` sees that `[` terminated unsuccessfully and executes the code in the `else` block. |
Line 132: | Line 210: |
$ if [ my dad = my dog ] > then echo "I have a problem." > fi |
$ [ my name = your name ] |
Line 137: | Line 213: |
Can you guess what caused the problem? [[BR]] `[` was executed with the arguments '`my`', '`dad`', '`=`', '`my`', '`dog`' and '`]`'. `[` doesn't understand what test it's supposed to execute, because it expects the second argument to be the operator. In our case, the operator is the third argument. Yet another reason why '''quotes''' are so terribly important. Whenever we type whitespace in bash that belongs together with the words before or after it, '''we need to quote the whole string''': {{{ $ if [ 'my dad' = 'my dog' ] > then echo "I have a problem." > fi |
Can you guess what caused the problem? `[` was executed with the arguments '`my`', '`name`', '`=`', '`your`', '`name`' and '`]`'. That is 6 arguments, not 4! `[` doesn't understand what test it's supposed to execute, because it expects either the first or second argument to be the operator. In our case, the operator is the third argument. Yet another reason why '''quotes''' are so terribly important. Whenever we type whitespace in bash that belongs together with the words before or after it, '''we need to quote the whole string''': {{{ $ [ 'my name' = 'your name' ] |
Line 147: | Line 223: |
$ dad='my dad'; dog='my dog' $ if [ $dad = $dog ] > then echo "I have a problem." > fi |
$ me='my name'; you='your name' $ [ $me = $you ] |
Line 153: | Line 227: |
How did we mess up this time? [[BR]] Here's a hint: ["BASH"] takes our ''if-statement'' and expands all the parameters in it. The result is `if [ my dad = my dog ]`. Boom, game over. Here's how it's supposed to look like: {{{ $ if [ "$dad" = "$dog" ] > then echo "I have a problem." > fi }}} To help us out a little, ["BASH"] introduced a new style of conditional test. Original as the ["BASH"] authors are, they called it `[[`. `[[` is loaded with several very interesting features which are missing from `[`. One of them helps us in dealing with parameter expansions: {{{ $ if [[ $dad = $dog ]] > then echo "I have a problem." > fi $ if [[ I want $dad = I want $dog ]] > then echo "I want too much." > fi |
How did we mess up this time? Here's a hint: [[BASH]] takes our ''if-statement'' and expands all the parameters in it. The result is `if [ my name = your name ]`, again. Boom, game over. Here's how you would fix this command using `[`: {{{ $ [ "$me" = "$you" ] }}} To help us out a little, the Korn shell introduced (and [[BASH]] adopted) a new style of conditional test. Original as the Korn shell authors are, they called it `[[`. `[[` is loaded with several very interesting features which are missing from `[`. One of them helps us in dealing with parameter expansions: {{{ $ [[ $me = $you ]] # Fine. $ [[ I am $me = I am $you ]] # Not fine! |
Line 172: | Line 240: |
-bash: syntax error near `want' }}} This time, `$dad` and `$dog` did not need quotes. Since `[[` isn't an application (while `[` is), but a ''shell keyword'', it has special magical powers. It parses its arguments before they are expanded by bash and does the expansion itself, taking the result as a single argument, even if that result contains whitespace. ''However'', be aware that simple strings still have to be quoted properly. `[[` can't know whether your literal whitespace in the statement is intentional or not; so it splits it up just like ["BASH"] normally would. Let's fix our last example: {{{ $ if [[ "I want $dad" = "I want $dog" ]] > then echo "I want too much." > fi }}} You could also combine several `if` statements into one using `elif` instead of `else`; where each test indicates another possibility: |
-bash: syntax error near `am' }}} This time, `$me` and `$you` did not need quotes. Since `[[` isn't a normal command (while `[` is), but a ''shell keyword'', it has special magical powers. It parses its arguments before they are expanded by bash and does the expansion itself, taking the result as a single argument, even if that result contains whitespace. ''However'', be aware that simple strings still have to be quoted properly. `[[` can't know whether your literal whitespace in the statement is intentional or not; so it splits it up just like [[BASH]] normally would. Let's fix our last example: {{{ $ [[ "I am $me" = "I am $you" ]] }}} Also; there is a subtle difference between quoting and not quoting the '''right-hand side''' of the comparison in `[[`. Unlike `[`, `[[` can do ''Pattern Matching''. That means, it can see whether a string is like a certain type of strings (conforms to a certain pattern). The `=` operator does this by default, whenever the ''right-hand side'' is '''not''' quoted: {{{ $ foo=[a-z]* name=lhunath $ [[ $name = $foo ]] && echo "Name $name matches pattern $foo" Name lhunath matches pattern [a-z]* $ [[ $name = "$foo" ]] || echo "Name $name is not equal to the string $foo" Name lhunath is not equal to the string [a-z]* }}} The first test checks whether the first argument (`name`) matches (because the right hand side is '''not''' quoted) the pattern in `foo`. The second test checks whether the first argument (`name`) is equal to (because the right hand side '''is''' quoted) the string in `foo`. A subtile syntax worth noting. '''Remember:''' Always quote stuff if you are unsure. If `foo` '''really''' contains a pattern instead of a string (a '''rare''' thing to want! You would normally write the pattern out literally: `[[ $name = [a-z]* ]]`), you will get a safe error here and you can come and fix it. If you neglect to quote, bugs can become very hard to find, since they aren't always easily reproduced! You could also combine several `if` statements into one using `elif` instead of `else`, where each test indicates another possibility: |
Line 192: | Line 272: |
> echo "You're neither George, Hans or Jack. Who the hell are you, $name?" | > echo "You're not George, Hans or Jack. Who the hell are you, $name?" |
Line 232: | Line 312: |
You want some examples? Sure: |
Some examples? Sure: |
Line 247: | Line 327: |
. '''Good Practice: [[BR]] Whenever you're making a ["BASH"] script, you should always use `[[`. [[BR]] Whenever you're making a Shell script, which may end up being used in an environment where ["BASH"] is not available, you should use `[`, because it is far more portable. (While being built in to ["BASH"] and some other shells, `[` should be available as an external application as well; meaning it will work as argument to, for example, find's -exec and xargs.)''' ---- . '''In The Manual: [http://www.gnu.org/software/bash/manual/bashref.html#SEC20 Conditional Constructs]''' ---- . '''In the FAQ: [[BR]] [http://wooledge.org/mywiki/BashFAQ/017 How can I group expressions, e.g. (A AND B) OR C?] [[BR]] [http://wooledge.org/mywiki/BashFAQ/031 What is the difference between the old and new test commands ([ and [[)?] [[BR]] [http://wooledge.org/mywiki/BashFAQ/041 How do I determine whether a variable contains a substring?] [[BR]] [http://wooledge.org/mywiki/BashFAQ/054 How can I tell whether a variable contains a valid number?]''' ---- . ''If (builtin)'': This command executes the command that is given on its arguments and depending on that command's exit code executes the code in the following `then` (or optionally `else`) block. -------- [[Anchor(Conditional_Loops)]] === Conditional Loops === |
. '''Good Practice: <<BR>> Whenever you're making a [[BASH]] script, you should always use `[[`. <<BR>> Whenever you're making a Shell script, which may end up being used in an environment where [[BASH]] is not available, you should use `[`, because it is far more portable. (While being built in to [[BASH]] and some other shells, `[` should be available as an external application as well; meaning it will work as argument to, for example, find's -exec and xargs.) <<BR>> Don't ever use the -a or -o tests of the `[` command. Use multiple `[` commands instead (or use `[[` if you can). POSIX doesn't define the behavior of `[` with complex sets of tests, so you never know what you'll get.''' ---- . '''In The Manual: [[http://www.gnu.org/software/bash/manual/bashref.html#Conditional-Constructs|Conditional Constructs]]''' ---- . '''In the FAQ: . [[BashFAQ/017|How can I group expressions, e.g. (A AND B) OR C?]] . [[BashFAQ/031|What is the difference between the old and new test commands ([ and [[)?]] . [[BashFAQ/041|How do I determine whether a variable contains a substring?]] . [[BashFAQ/054|How can I tell whether a variable contains a valid number?]]''' ---- . ''If (builtin)'': This command executes a command and depending on that command's exit code executes the code in the following `then` (or optionally `else`) block. -------- <<Anchor(Conditional_Loops)>> == Conditional Loops (while, until and for) == |
Line 263: | Line 345: |
["BASH"] scripts, much like any other kind of scripts, should never be overrated. Although they have great potential once you fully understand the shell's features, they aren't always the right tool for the job. At the same time, when you make scripts, you should remember to keep them light, both in length and in complexity. Very long and/or very complex scripts are most often also very bad scripts. Those that aren't yet soon will be, because they are always very difficult to maintain and adapt/extend. A technique that we can use to try and keep code length and complexity down is the ''loop''. There are two kinds of loops. Using the correct kind of loop will help you keep your scripts readable and healthy. ["BASH"] supports `while` loops and `for` loops. The `for` loops can appear in three different forms. Here's a summary: |
[[BASH]] scripts, much like any other kind of scripts, should never be overrated. Although they have great potential once you fully understand the shell's features, they aren't always the right tool for the job. At the same time, when you make scripts, you should remember to keep them light, both in length and in complexity. Very long and/or very complex scripts are most often also very bad scripts. Those that aren't yet soon will be, because they are always very difficult to maintain and adapt/extend. A technique that we can use to try to keep code length and complexity down is the ''loop''. There are two kinds of loops. Using the correct kind of loop will help you keep your scripts readable and healthy. [[BASH]] supports `while` loops and `for` loops. The `for` loops can appear in three different forms. Here's a summary: |
Line 271: | Line 353: |
* '''`for` ''variable'' `in` ''words''''': Repeat the loop for each word, putting it into the variable. * '''`for ((` ''expression''`;` ''expression''`;` ''expression'' `))`''': Starts by evaluating the first expression, repeats the loop so long as the second expression is valid and at the end of each loop evaluates the third expression. |
* '''`for` ''variable'' `in` ''words''''': Repeat the loop for each word, setting ''variable'' to each word in turn. * '''`for ((` ''expression''`;` ''expression''`;` ''expression'' `))`''': Starts by evaluating the first arithmetic expression; repeats the loop so long as the second arithmetic expression is valid; and at the end of each loop evaluates the third arithmetic expression. Each loop form is followed by the key word `do`, then one or more commands in the body, then the key word `done`. The `do` and `done` are similar to the `then` and `fi` (and possible `elif` and/or `else`) from the `if` statement we saw earlier. |
Line 274: | Line 359: |
Line 286: | Line 370: |
$ for i in {10..0} | $ for i in {10..1} |
Line 294: | Line 378: |
As I mentioned before: `for` runs through a list of words and puts each in the variable, one at a time, then loops through the body with it. The tricky part is how ["BASH"] decides what the words are. Let me explain myself by expanding the braces from that previous example: {{{ $ for i in 10 9 8 7 6 5 4 3 2 1 0 |
As I mentioned before: `for` runs through a list of words and puts each in the variable, one at a time, then loops through the body with it. The tricky part is how [[BASH]] decides what the words are. Let me explain myself by expanding the braces from that previous example: {{{ $ for i in 10 9 8 7 6 5 4 3 2 1 |
Line 301: | Line 385: |
["BASH"] takes the characters between `in` and the end of the line, and splits them up into words. You shouldn't confuse the splitting that happens here with the splitting that happens with ''Commandline Arguments'', even though they look exactly the same at first sight. Commandline arguments are split at ''spaces'', ''tabs'' and ''newlines''. The splitting in this `for` statement happens at ''spaces'' by default, but this default behaviour can be changed. The way `for` determines what delimiter to use for the splitting is by the contents of the `IFS` variable. `IFS` is an acronym for ''Internal Field Separator''; and by default it contains a ''space'', a ''tab'' and a ''newline''. Since the ''space'' is one of the characters there, `for` uses it to split up the words in our sequence, and feeds each word to the variable `i`, one at a time. '''As a result; be VERY careful not to make the following mistake:''' |
[[BASH]] takes the characters between `in` and the end of the line, and splits them up into words. This splitting is done on spaces and tabs, just like argument splitting. However, if there are any unquoted substitutions in there, they will be word-split as well; it is these split-up words that become the iteration elements. '''As a result, be VERY careful not to make the following mistake:''' |
Line 318: | Line 403: |
You should already know to quote the `$file` in the `rm` statement; but what's going wrong here? ["BASH"] expands the command substitution (`$(ls *.mp3)`), replaces it by its output, and as a result executes `for file in The best song in the world.mp3`. ["BASH"] splits that up into words by using ''spaces'' and tries to `rm` each word. ''Boom, you are dead''. | You should already know to quote the `$file` in the `rm` statement; but what's going wrong here? [[BASH]] expands the command substitution (`$(ls *.mp3)`), replaces it by its output, and as a result executes `for file in The best song in the world.mp3`. [[BASH]] splits that up into words by using ''spaces'' and tries to `rm` each word. ''Boom, you are dead''. |
Line 330: | Line 415: |
Quotes will indeed protect the whitespace in your filenames; but they will do more than that. The quotes will protect '''all the whitespace''' from the output of `ls`. There is no way ["BASH"] can know which parts of the output of `ls` represent filenames; it's not psychic. The output of `ls` is a simple string, and ["BASH"] treats it as such. The `for` puts the whole quoted output in `i` and runs the `rm` command with it. ''Damn, dead again''. | Quotes will indeed protect the whitespace in your filenames; but they will do more than that. The quotes will protect '''all the whitespace''' from the output of `ls`. There is no way [[BASH]] can know which parts of the output of `ls` represent filenames; it's not psychic. The output of `ls` is a simple string, and [[BASH]] treats it as such. The `for` puts the whole quoted output in `i` and runs the `rm` command with it. ''Damn, dead again''. |
Line 339: | Line 424: |
This time, ["BASH"] '''does''' know that it's dealing with filenames, and it '''does''' know what the filenames are, and as such it can split them up nicely. The result of expanding the glob is this: `for file in "The best song in the world.mp3" "The worst song in the world.mp3"`. Problem resolved. Let's talk about changing that delimiter. Say you've got yourself a nice cooking recipe, and you want to write a script that tells you how to use it. Sure, let's get right at it: |
This time, [[BASH]] '''does''' know that it's dealing with filenames, and it '''does''' know what the filenames are, and as such it can split them up nicely. The result of expanding the glob is this: `for file in "The best song in the world.mp3" "The worst song in the world.mp3"`. Problem resolved. Let's talk about changing that behavior. Say you've got yourself a nice cooking recipe, and you want to write a script that tells you how to use it. Sure, let's get right at it: |
Line 355: | Line 440: |
Yes, as explained earlier, `for` splits its stuff up into words by using the delimiters from `IFS`. To read the recipe correctly, we need to split it up by newlines alone, instead of by spaces and newlines. Here's how we do that: |
Yes, as explained earlier, [[BASH]] splits the contents of the `recipe` variable into words using the characters of the Input Field Separator (`IFS`). The `for` loop iterates over those words. To read the recipe correctly, we want to split it up by newlines alone, instead of by spaces and tabs and newlines. Here's how we do that: |
Line 374: | Line 459: |
}}} Beautiful. |
$ unset IFS }}} Excellent. Two special notes: * The syntax `$'\n'` represents a literal newline, and therefore `IFS=$'\n'` puts a literal newline into the `IFS` variable. * We unset the `IFS` variable at the end, so we don't get a nasty shock later, since this is our working shell and IFS is used for a lot of other things, too. We should either put it back to its default value or unset it which has the same effect. |
Line 397: | Line 485: |
$ unset IFS | |
Line 422: | Line 511: |
-------- . '''In The Manual: [http://www.gnu.org/software/bash/manual/bashref.html#SEC19 Looping Constructs]''' ---- . '''In the FAQ: [[BR]] [http://wooledge.org/mywiki/BashFAQ/015 How can I run a command on all files with the extention .gz?] [[BR]] [http://wooledge.org/mywiki/BashFAQ/018 How can I use numbers with leading zeros in a loop, e.g. 01, 02?] [[BR]] [http://wooledge.org/mywiki/BashFAQ/020 How can I find and deal with file names containing newlines, spaces or both?] [[BR]] [http://wooledge.org/mywiki/BashFAQ/030 How can I rename all my *.foo files to *.bar, or convert spaces to underscores, or convert upper-case file names to lower case?] [[BR]] [http://wooledge.org/mywiki/BashFAQ/034 Can I do a spinner in Bash?] [[BR]] [http://wooledge.org/mywiki/BashFAQ/046 I want to check to see whether a word is in a list (or an element is a member of a set).]''' ---- . ''Loop'': A loop is a structure that is designed to repeat the code within until a certain condition has been fulfilled. At that point, the loop stops and the code beyond it is executed. [[BR]] ''For (builtin)'': A `for`-loop is a type of loop that sets a variable to each of a list of values in turn, and repeats until the list is exhausted. [[BR]] ''While (builtin)'': A `while`-loop is a type of loop that continues to run its code so long as a certain command (run before each iteration) executes successfully. [[BR]] ''Until (builtin)'': An `until`-loop is a type of loop that continues to run its code so long as a certain command (run before each iteration) executes unsuccessfully. -------- [[Anchor(Choices)]] === Choices === Sometimes you just want to built application logic depending on the content of a variable but you already know all of the possible choices of content the variable can contain. You no longer need `if`'s two ''yes or no'' blocks, but you want a block of code for every possible choice of content a variable can have. You could obviously attempt to emulate this behaviour using several `if`-statements: |
Lastly, you can use the `continue` builtin to skip ahead to the next iteration of a loop without executing the rest of the iteration body, and the `break` builtin to jump out of the loop and continue with the script after it. -------- . '''In The Manual: [[http://www.gnu.org/software/bash/manual/bashref.html#Looping-Constructs|Looping Constructs]]''' ---- . '''In the FAQ: . [[BashFAQ/015|How can I run a command on all files with the extention .gz?]] . [[BashFAQ/018|How can I use numbers with leading zeros in a loop, e.g. 01, 02?]] . [[BashFAQ/020|How can I find and deal with file names containing newlines, spaces or both?]] . [[BashFAQ/030|How can I rename all my *.foo files to *.bar, or convert spaces to underscores, or convert upper-case file names to lower case?]] . [[BashFAQ/034|Can I do a spinner in Bash?]] . [[BashFAQ/046|I want to check to see whether a word is in a list (or an element is a member of a set).]]''' ---- . ''Loop'': A loop is a structure that is designed to repeat the code within until a certain condition has been fulfilled. At that point, the loop stops and the code beyond it is executed. . ''For (builtin)'': A `for`-loop is a type of loop that sets a variable to each of a list of values in turn, and repeats until the list is exhausted. . ''While (builtin)'': A `while`-loop is a type of loop that continues to run its code so long as a certain command (run before each iteration) executes successfully. . ''Until (builtin)'': An `until`-loop is a type of loop that continues to run its code so long as a certain command (run before each iteration) executes unsuccessfully. -------- <<Anchor(Choices)>> == Choices (case and select) == Sometimes you just want to build application logic depending on the content of a variable but you already know all of the possible choices of content the variable can contain. You no longer need `if`'s two ''yes or no'' blocks, but you want a block of code for every possible choice of content a variable can have. You could obviously attempt to emulate this behaviour using several `if`-statements: |
Line 449: | Line 548: |
But let's say we like to actually make clean code, shall we? ["BASH"] provides a construct called `case` exactly for this kind of situation. A `case` construct basically enumerates several possible ''Glob Patterns'' and checks the content of your parameter against these: | But let's say we like to actually make clean code, shall we? [[BASH]] provides a construct called `case` exactly for this kind of situation. A `case` construct basically enumerates several possible ''Glob Patterns'' and checks the content of your parameter against these: |
Line 461: | Line 560: |
Each choice in a `case` statement starts with a pattern followed by a block of code that is to be executed if the parameter's content matches that pattern, which in turn is followed by two semi-colons. `case` stops matching patterns as soon as one was successful. Therefore, we can use the `*` pattern in the end to match any case that has not been caught by the other choices. | Each choice in a `case` statement starts with a pattern followed by a block of code that is to be executed if the string to be checked matches that pattern, which in turn is followed by two semi-colons. `case` stops matching patterns as soon as one was successful. Therefore, we can use the `*` pattern in the end to match any case that has not been caught by the other choices. |
Line 488: | Line 587: |
. '''In The Manual: [http://www.gnu.org/software/bash/manual/bashref.html#SEC20 Conditional Constructs]''' ---- . '''In the FAQ: [[BR]] [http://wooledge.org/mywiki/BashFAQ/066 I want to check if [[ $var == foo || $var == bar || $var = more ... without repeating $var n times.]''' ---- . ''Case (builtin)'': This statement evaluates a parameter's value based on several given patterns (choices). [[BR]] ''Select (builtin)'': The `select` statement offers the user the choice between several options and executes a block of code for the user's choice. -------- |
. '''In The Manual: [[http://www.gnu.org/software/bash/manual/bashref.html#Conditional-Constructs|Conditional Constructs]]''' ---- . '''In the FAQ: . [[BashFAQ/066|I want to check if [[ $var == foo or $var == bar or $var = more ... without repeating $var n times.]]''' ---- . ''Case (builtin)'': This statement evaluates a parameter's value based on several given patterns (choices). . ''Select (builtin)'': The `select` statement offers the user the choice between several options and executes a block of code for the user's choice. -------- [[BashGuide/Arrays|<- Arrays]] | [[BashGuide/InputAndOutput|Input and Output ->]] |
<- Arrays | Input and Output ->
Tests and Conditionals
Contents
Sequential execution of applications is one thing, but to achieve a sort of logic in your scripts or your command line one-liners, you'll need variables and conditionals. Conditionals are used to make decisions which determine the execution flow of a script.
1. Exit Status
Every command results in an exit code whenever it terminates. This exit code is used by whatever application started it to evaluate whether everything went OK. This exit code is like a return value from functions. It's an integer between 0 and 255 (inclusive). Convention dictates that we use 0 to denote success, and any other number to denote failure of some sort. The specific number is entirely application-specific, and is used to hint as to what exactly went wrong.
For example, the ping command sends ICMP packets over the network to a certain host. That host normally responds to this packet by sending the exact same one right back. This way, we can check whether we can communicate with a remote host. ping has a range of exit codes which can tell us what went wrong, if anything did:
From the Linux ping manual:
If ping does not receive any reply packets at all it will exit with code 1. If a packet count and deadline are both specified, and fewer than count packets are received by the time the deadline has arrived, it will also exit with code 1. On other error it exits with code 2. Otherwise it exits with code 0. This makes it possible to use the exit code to see if a host is alive or not.
The special parameter ? shows us the exit code of the last foreground process that terminated. Let's play around a little with ping to see its exit codes:
$ ping God ping: unknown host God $ echo $? 2 $ ping -c 1 -W 1 1.1.1.1 PING 1.1.1.1 (1.1.1.1) 56(84) bytes of data. --- 1.1.1.1 ping statistics --- 1 packets transmitted, 0 received, 100% packet loss, time 0ms $ echo $? 1
Good Practice:
You should make sure that your scripts always return a non-zero exit code if something unexpected happened in their execution. You can do this with the exit builtin:
rm file || { echo "Could not delete file!"; exit 1; }
In The Manual: Exit Status
Exit Code / Exit Status: Whenever a command ends it notifies its parent (which in our case will always be the shell that started it) of its exit status. This is represented by a number ranging from 0 to 255. This code is a hint as to the success of the command's execution.
2. Control Operators (&& and ||)
Now that we know what exit codes are, and that an exit code of '0' means the command's execution was successful, we'll learn to use this information. The easiest way of performing a certain action depending on the success of a previous command is through the use of 'control operators'. These operators are && and ||, which respectively represent a logical AND and a logical OR. These operators are used between two commands, and they are used to control whether the second command should be executed depending on the success of the first.
Let's put that theory in practice:
$ mkdir d && cd d
This simple example has two commands, mkdir d and cd d. You could easily just use a semi-colon there to separate both commands and execute them sequentially; but we want something more. In the above example, BASH will execute mkdir d, then && will check the result of the mkdir application as it finishes. If the mkdir application resulted in a success (exit code 0), then && will execute the next command, cd d. If mkdir d failed, and returned a non-0 exit code, && will skip the next command, and we will stay in the current directory.
Another example:
$ rm /etc/some_file.conf || echo "I couldn't remove the file!" rm: cannot remove `/etc/some_file.conf': No such file or directory I couldn't remove the file!
|| is much like &&, but it does the exact opposite. It only executes the next command if the first failed. As such, the message is only echoed if the rm command was unsuccessful.
You can make a sequence with these operators, but you have to be very careful when you do. Remember what exit code the operator is really going to be checking against! Here's an example that might cause confusion:
$ false && true || echo "Riddle, riddle?" Riddle, riddle? $ true && false || echo "Riddle, riddle?" Riddle, riddle?
true is obviously always going to be successful. false is obviously always going to be unsuccessful. Can you guess why the echo statement is executed in both occasions?
The key to understanding how to sequence these operators properly is by evaluating exit codes from left to right.
In the first example, false is unsuccessful, so && does not execute the next command (which is true), but the next || gets a shot too. || still sees that the last exit code was that from false, and || executes the next command when the previous was unsuccessful. As a result, the echo statement is executed.
The same for the second statement again. true is successful, so the && executes the next statement. That is false; the last exit code now becomes unsuccessful. After that, || is evaluated, which sees the unsuccessful exit code from false and executes the echo statement.
It's all easy with trues and falses; but how about real commands?
# There is a problem here.... $ rm file && touch file || echo "File not found!"
All seems well with this piece of code, and when you test it, I'm sure you'll see that it actually does what it's supposed to. It tries to delete a file, and if it succeeds, it creates it again as a new and empty file; if something goes wrong we get the error message. What's the catch?
Perhaps you guessed, perhaps not, but here's a hint: Imagine the file system fills up (or some other disaster occurs) in between rm and the touch; the rm will succeed in deleting our file, but touch will fail to create it anew. As a result, we get a strange error message saying that the file wasn't found while we were actually trying to create it. What's up with that?
See the next two sections for two ways of dealing with this.
Good Practice:
It's best not to get overzealous when dealing with conditional operators. They can make your script hard to understand, especially for a person that's assigned to maintain it and didn't write it himself.
In The Manual: Lists of Commands
Control Operators: These operators are used to link commands together. They check the exit code of the command that was previously ran to determine whether or not to execute the next command in the sequence.
3. Grouping Statements
Using conditional operators is easy and terse if we want to run a statement on the condition that another succeeds. Things get a bit more dangerous, though, when we want to run multiple statements if a condition holds true, or if we need to evaluate multiple conditions.
Suppose you want to delete a file if it contains a certain word but also doesn't contain another word. Using grep, we translate these conditions to:
grep -q word "$file" # exit status 0 (success) if "$file" contains 'word' ! grep -q "another word" "$file" # exit status 0 (success) if "$file" does not contain 'another word'
We use -q (quiet) on grep because we don't want it to output the lines that match, we just want the exit code to be set.
Now, to put these conditions together and delete the file as a result of both holding true, we could use Conditional Operators:
$ grep -q word "$file" && ! grep -q "another word" "$file" && rm "$file"
This works great. Now, imagine we want to show an error message in case the deletion of the file failed:
$ grep -q word "$file" && ! grep -q "another word" "$file" && rm "$file" || echo "Couldn't delete: $file" >&2
This looks OK, at first sight. If rm's exit code is not 0 (success), then the || operator will trigger the next command and echo the error message (>&2: to standard error).
However, as pointed out at the end of the previous section on Conditional Operators, should either grep command fail, the && operator that follows it will only skip the next statement, but not the one after that. Imagine the first grep failing. The && that follows it will make sure the second grep is not tried. BASH will try the statement after that, though. The next statement is the rm statement, but since it is also preceded by a && operator, it will also not be tried (remember, our exit code still indicates failure since the last command that was executed was our failed grep). BASH will move on to the next statement. This time, we see the echo statement, and it's preceded by a || operator. Since our exit status indicates failure, the || operator lets us through, and the echo statement is executed. As a result, the failed grep causes the error message Couldn't delete: $file to be shown on the screen. That's not what we want.
Doesn't sound lethal when it's just a wrong error message you receive, but if you're not careful, this will eventually happen on more dangerous code. You wouldn't want to accidentally delete files or overwrite files as a result of a failure in your logic!
The failure in our logic is in the fact that we want the rm and the echo statements to belong together. The echo is related to the rm, not to the greps. So what we need, is to group them. Grouping is done using curly braces:
$ grep -q word "$file" && ! grep -q "another word" "$file" && { rm "$file" || echo "Couldn't delete: $file" >&2; }
(Note: don't forget that you need a semicolon or newline before the closing curly brace!)
Now we've grouped the rm and echo command together. That effectively means: before BASH has gone into this group, it is considered one statement instead of several. Going back to our situation of the first grep failing, instead of BASH trying the && rm "$file" statement, it will now try the && { ... } statement. Since it is preceded by a && and the last command it ran failed (the failed grep), it will skip this statement and move on.
Command grouping can be used for more things than just grouping statements for Conditional Operators. We may also want to group them so that we can redirect input to a group of statements instead of just one:
{ read firstLine read secondLine while read otherLine; do something done } < file
Here we're redirecting file to a group of commands that read input. The file will be opened when the curly brace opens, stay open for the duration of it, and be closed when the curly brace closes. This way, we can keep sequentially reading lines from it with multiple commands.
4. Conditional Blocks (if, test and [[)
if is a shell keyword that executes a command, and checks that command's exit code to see whether its execution was successful. Depending on that exit code, if executes a specific block of code.
$ if true > then echo "It was true." > else echo "It was false!" > fi It was true.
Here you see the basic outline of an if-statement. We start by calling if with the command true. true is a built-in command that always ends successfully. if runs that command, and once the command is done, if checks the exit code. Since true always exits successfully, if continues to the then-block, and executes that code. Should the true command have failed somehow, and returned an unsuccessful exit code, the if statement would have skipped the then code, and executed the else code block instead.
Different people have different preferred styles for writing if statements. Here are some of the common styles:
if COMMANDS then OTHER COMMANDS fi if COMMANDS then OTHER COMMANDS fi if COMMANDS; then OTHER COMMANDS fi
There are commands that can help us a lot in doing conditional checks. They are [ (also named test) and [[. [ is a normal command that reads its arguments and does some checks with them. [[ is much like [, but it's special (a shell keyword), and it offers far more versatility. Let's get practical:
$ if [ a = b ] > then echo "a is the same as b." > else echo "a is not the same as b." > fi a is not the same as b.
if executes the command [ (you don't need if to run the [ command!) with the arguments 'a', '=', 'b' and ']'. [ uses these arguments to determine what must be checked. In this case, it checks whether the string a (the first argument) is equal (the second argument) to the string b (the third argument), and if this is the case, it will exit successfully. However, since we know this is not the case, [ will not exit successfully (its exit code will be 1). if sees that [ terminated unsuccessfully and executes the code in the else block.
Now, to see why [[ is so much more interesting and trustworthy than [, let us highlight some possible problems with [:
$ [ my name = your name ] -bash: [: too many arguments
Can you guess what caused the problem?
[ was executed with the arguments 'my', 'name', '=', 'your', 'name' and ']'. That is 6 arguments, not 4! [ doesn't understand what test it's supposed to execute, because it expects either the first or second argument to be the operator. In our case, the operator is the third argument. Yet another reason why quotes are so terribly important. Whenever we type whitespace in bash that belongs together with the words before or after it, we need to quote the whole string:
$ [ 'my name' = 'your name' ]
This time, [ sees an operator (=) in the second argument and it can continue with its work. Now, this may be easy to see and avoid, but it gets just a little trickier when we put the strings in variables, rather than literally in the statement:
$ me='my name'; you='your name' $ [ $me = $you ] -bash: [: too many arguments
How did we mess up this time? Here's a hint: BASH takes our if-statement and expands all the parameters in it. The result is if [ my name = your name ], again. Boom, game over.
Here's how you would fix this command using [:
$ [ "$me" = "$you" ]
To help us out a little, the Korn shell introduced (and BASH adopted) a new style of conditional test. Original as the Korn shell authors are, they called it [[. [[ is loaded with several very interesting features which are missing from [. One of them helps us in dealing with parameter expansions:
$ [[ $me = $you ]] # Fine. $ [[ I am $me = I am $you ]] # Not fine! -bash: conditional binary operator expected -bash: syntax error near `am'
This time, $me and $you did not need quotes. Since [[ isn't a normal command (while [ is), but a shell keyword, it has special magical powers. It parses its arguments before they are expanded by bash and does the expansion itself, taking the result as a single argument, even if that result contains whitespace. However, be aware that simple strings still have to be quoted properly. [[ can't know whether your literal whitespace in the statement is intentional or not; so it splits it up just like BASH normally would. Let's fix our last example:
$ [[ "I am $me" = "I am $you" ]]
Also; there is a subtle difference between quoting and not quoting the right-hand side of the comparison in [[. Unlike [, [[ can do Pattern Matching. That means, it can see whether a string is like a certain type of strings (conforms to a certain pattern). The = operator does this by default, whenever the right-hand side is not quoted:
$ foo=[a-z]* name=lhunath $ [[ $name = $foo ]] && echo "Name $name matches pattern $foo" Name lhunath matches pattern [a-z]* $ [[ $name = "$foo" ]] || echo "Name $name is not equal to the string $foo" Name lhunath is not equal to the string [a-z]*
The first test checks whether the first argument (name) matches (because the right hand side is not quoted) the pattern in foo. The second test checks whether the first argument (name) is equal to (because the right hand side is quoted) the string in foo. A subtile syntax worth noting.
Remember: Always quote stuff if you are unsure. If foo really contains a pattern instead of a string (a rare thing to want! You would normally write the pattern out literally: [[ $name = [a-z]* ]]), you will get a safe error here and you can come and fix it. If you neglect to quote, bugs can become very hard to find, since they aren't always easily reproduced!
You could also combine several if statements into one using elif instead of else, where each test indicates another possibility:
$ name=lhunath $ if [[ $name = "George" ]] > then echo "Bonjour, $name!" > elif [[ $name = "Hans" ]] > then echo "Goeie dag, $name!" > elif [[ $name = "Jack" ]] > then echo "Good day, $name!" > else > echo "You're not George, Hans or Jack. Who the hell are you, $name?" > fi
Now that you've got a decent understanding of quoting issues that may arise, let's have a look at some of the other features that [ and [[ were blessed with:
Tests supported by [ (also known as test):
-e FILE: True if file exists.
-f FILE: True if file is a regular file.
-d FILE: True if file is a directory.
-h FILE: True if file is a symbolic link.
-r FILE: True if file is readable by you.
-s FILE: True if file exists and is not empty.
-t FD : True if FD is opened on a terminal.
-w FILE: True if the file is writable by you.
-x FILE: True if the file is executable by you.
-O FILE: True if the file is effectively owned by you.
-G FILE: True if the file is effectively owned by your group.
FILE -nt FILE: True if the first file is newer than the second.
FILE -ot FILE: True if the first file is older than the second.
-z STRING: True if the string is empty (it's length is zero).
-n STRING: True if the string is not empty (it's length is not zero).
STRING = STRING: True if the first string is identical to the second.
STRING != STRING: True if the first string is not identical to the second.
STRING < STRING: True if the first string sorts before the second.
STRING > STRING: True if the first string sorts after the second.
EXPR -a EXPR: True if both expressions are true (logical AND).
EXPR -o EXPR: True if either expression is true (logical OR).
! EXPR: Inverts the result of the expression (logical NOT).
INT -eq INT: True if both integers are identical.
INT -ne INT: True if the integers are not identical.
INT -lt INT: True if the first integer is less than the second.
INT -gt INT: True if the first integer is greater than the second.
INT -le INT: True if the first integer is less than or equal to the second.
INT -ge INT: True if the first integer is greater than or equal to the second.
Additional tests supported only by [[:
STRING = (or ==) PATTERN: Not string comparison like with [ (or test), but pattern matching is performed. True if the string matches the glob pattern.
STRING =~ REGEX: True if the string matches the regex pattern.
( EXPR ): Parantheses can be used to change the evaluation precedence.
EXPR && EXPR: Much like the '-a' operator of test, but does not evaluate the second expression if the first already turns out to be false.
EXPR || EXPR: Much like the '-o' operator of test, but does not evaluate the second expression if the first already turns out to be true.
Some examples? Sure:
$ test -e /etc/X11/xorg.conf && echo "Your Xorg is configured!" Your Xorg is configured! $ test -n "$HOME" && echo "Your homedir is set!" Your homedir is set! $ [[ boar != bear ]] && echo "Boars aren't bears!" Boars aren't bears! $ [[ boar != b?ar ]] && echo "Boars don't look like bears!" $ [[ $DISPLAY ]] && echo "Your DISPLAY variable is not empty, you probably have Xorg running." Your DISPLAY variable is not empty, you probably have Xorg running. $ [[ ! $DISPLAY ]] && echo "Your DISPLAY variable is not not empty, you probably don't have Xorg running."
Good Practice:
Whenever you're making a BASH script, you should always use [[.
Whenever you're making a Shell script, which may end up being used in an environment where BASH is not available, you should use [, because it is far more portable. (While being built in to BASH and some other shells, [ should be available as an external application as well; meaning it will work as argument to, for example, find's -exec and xargs.)
Don't ever use the -a or -o tests of the [ command. Use multiple [ commands instead (or use [[ if you can). POSIX doesn't define the behavior of [ with complex sets of tests, so you never know what you'll get.
In The Manual: Conditional Constructs
If (builtin): This command executes a command and depending on that command's exit code executes the code in the following then (or optionally else) block.
5. Conditional Loops (while, until and for)
You've learned how to code some basic logic flow for your scripts. It's important that you understand a thing or two about keeping scripts healthy before we start introducing more complexity.
BASH scripts, much like any other kind of scripts, should never be overrated. Although they have great potential once you fully understand the shell's features, they aren't always the right tool for the job. At the same time, when you make scripts, you should remember to keep them light, both in length and in complexity. Very long and/or very complex scripts are most often also very bad scripts. Those that aren't yet soon will be, because they are always very difficult to maintain and adapt/extend.
A technique that we can use to try to keep code length and complexity down is the loop. There are two kinds of loops. Using the correct kind of loop will help you keep your scripts readable and healthy.
BASH supports while loops and for loops. The for loops can appear in three different forms. Here's a summary:
while command: Repeat so long as command is executed successfully (exit code: 0).
until command: Repeat so long as command is executed unsuccessfully (exit code: >0).
for variable in words: Repeat the loop for each word, setting variable to each word in turn.
for (( expression; expression; expression )): Starts by evaluating the first arithmetic expression; repeats the loop so long as the second arithmetic expression is valid; and at the end of each loop evaluates the third arithmetic expression.
Each loop form is followed by the key word do, then one or more commands in the body, then the key word done. The do and done are similar to the then and fi (and possible elif and/or else) from the if statement we saw earlier.
Let's put that in practice; here are some examples to illustrate the differences but also the similarities between the loops:
$ while true > do echo "Infinite loop!" > done $ (( i=10 )); while (( i > 0 )) > do echo "$i empty cans of beer." > (( i-- )) > done $ for (( i=10; i > 0; i-- )) > do echo "$i empty cans of beer." > done $ for i in {10..1} > do echo "$i empty cans of beer." > done
The last three loops achieve exactly the same result, using different syntax. You'll encounter this many times in your shell scripting experience. There will nearly always be multiple approaches to solving a problem. The test of your skill soon won't be about solving a problem as much as about how best to solve it. You need to learn to pick the best angle of approach for the job. Usually, the main factors to keep into account will be the simplicity and flexibility of the resulting code. My personal favorite is the last of the examples. In that example I used Brace Expansion to generate the words; but there are other ways, too.
Let's take a closer look at that last example, because although it looks the easier of the two fors, it can often be the trickier, if you don't know exactly how it works.
As I mentioned before: for runs through a list of words and puts each in the variable, one at a time, then loops through the body with it. The tricky part is how BASH decides what the words are. Let me explain myself by expanding the braces from that previous example:
$ for i in 10 9 8 7 6 5 4 3 2 1 > do echo "$i empty cans of beer." > done
BASH takes the characters between in and the end of the line, and splits them up into words. This splitting is done on spaces and tabs, just like argument splitting. However, if there are any unquoted substitutions in there, they will be word-split as well; it is these split-up words that become the iteration elements.
As a result, be VERY careful not to make the following mistake:
$ ls The best song in the world.mp3 $ for file in $(ls *.mp3) > do rm "$file" > done rm: cannot remove `The': No such file or directory rm: cannot remove `best': No such file or directory rm: cannot remove `song': No such file or directory rm: cannot remove `in': No such file or directory rm: cannot remove `the': No such file or directory rm: cannot remove `world.mp3': No such file or directory
You should already know to quote the $file in the rm statement; but what's going wrong here? BASH expands the command substitution ($(ls *.mp3)), replaces it by its output, and as a result executes for file in The best song in the world.mp3. BASH splits that up into words by using spaces and tries to rm each word. Boom, you are dead.
You want to quote it, you say? Let's add another song:
$ ls The best song in the world.mp3 The worst song in the world.mp3 $ for file in "$(ls *.mp3)" > do rm "$file" > done rm: cannot remove `The best song in the world.mp3 The worst song in the world.mp3': No such file or directory
Quotes will indeed protect the whitespace in your filenames; but they will do more than that. The quotes will protect all the whitespace from the output of ls. There is no way BASH can know which parts of the output of ls represent filenames; it's not psychic. The output of ls is a simple string, and BASH treats it as such. The for puts the whole quoted output in i and runs the rm command with it. Damn, dead again.
So what do we do? As suggested earlier, globs are your best friend:
$ for file in *.mp3 > do rm "$file" > done
This time, BASH does know that it's dealing with filenames, and it does know what the filenames are, and as such it can split them up nicely. The result of expanding the glob is this: for file in "The best song in the world.mp3" "The worst song in the world.mp3". Problem resolved.
Let's talk about changing that behavior. Say you've got yourself a nice cooking recipe, and you want to write a script that tells you how to use it. Sure, let's get right at it:
$ recipe='2 c. all purpose flour > 6 tsp. baking powder > 2 eggs > 2 c. milk > 1/3 c. oil' $ for ingredient in $recipe > do echo "Take $ingredient; mix well." > done
Can you guess what the result will look like? I recommend you run the code if you can't and ponder the reason first. It will help you understand things.
Yes, as explained earlier, BASH splits the contents of the recipe variable into words using the characters of the Input Field Separator (IFS). The for loop iterates over those words.
To read the recipe correctly, we want to split it up by newlines alone, instead of by spaces and tabs and newlines. Here's how we do that:
$ recipe='2 c. all purpose flour > 6 tsp. baking powder > 2 eggs > 2 c. milk > 1/3 c. oil' $ IFS=$'\n' $ for ingredient in $recipe > do echo "Take $ingredient; mix well." > done Take 2 c. all purpose flour; mix well. Take 6 tsp. baking powder; mix well. Take 2 eggs; mix well. Take 2 c. milk; mix well. Take 1/3 c. oil; mix well. $ unset IFS
Excellent. Two special notes:
The syntax $'\n' represents a literal newline, and therefore IFS=$'\n' puts a literal newline into the IFS variable.
We unset the IFS variable at the end, so we don't get a nasty shock later, since this is our working shell and IFS is used for a lot of other things, too. We should either put it back to its default value or unset it which has the same effect.
Note: This delimiter is only used when the words consist of an expansion. Not when they're literal. Literal words are always split at spaces:
$ PATH=/bin:/usr/bin $ IFS=: $ for i in $PATH > do echo "$i" > done /bin /usr/bin $ for i in $PATH:/usr/local/bin > do echo "$i" > done /bin /usr/bin /usr/local/bin $ for i in /bin:/usr/bin:/usr/local/bin > do echo "$i" > done /bin:/usr/bin:/usr/local/bin $ unset IFS
Lets focus a little more on the while loop. It promises even more simplicity than this for loop, so long as you don't need any for specific features. The while loop is very interesting for its capacity of executing commands and basing the loop's progress on the result of them. Here are a few examples of how while loops are very often used:
$ # The sweet machine; hand out sweets for a cute price. $ while read -p $'The sweet machine.\nInsert 20c and enter your name: ' name > do echo "The machine spits out three lollipops at $name." > done $ # Check your email every five minutes. $ while sleep 5m > do kmail --check > done $ # Wait for a host to come back online. $ while ! ping -c 1 -W 1 "$host" > do echo "$host is still unavailable." > done; echo -e "$host is available again!\a"
The until loop is barely ever used, if only because it is pretty much exactly the same as the while loop except for the fact that its command is not checked for an exit status of zero but for an exit status of non-zero. As a result, we could rewrite our last example using an until loop as such:
$ # Wait for a host to come back online. $ until ping -c 1 -W 1 "$host" > do echo "$host is still unavailable." > done; echo -e "$host is available again!\a"
Lastly, you can use the continue builtin to skip ahead to the next iteration of a loop without executing the rest of the iteration body, and the break builtin to jump out of the loop and continue with the script after it.
In The Manual: Looping Constructs
In the FAQ:
How can I run a command on all files with the extention .gz?
How can I use numbers with leading zeros in a loop, e.g. 01, 02?
How can I find and deal with file names containing newlines, spaces or both?
I want to check to see whether a word is in a list (or an element is a member of a set).
Loop: A loop is a structure that is designed to repeat the code within until a certain condition has been fulfilled. At that point, the loop stops and the code beyond it is executed.
For (builtin): A for-loop is a type of loop that sets a variable to each of a list of values in turn, and repeats until the list is exhausted.
While (builtin): A while-loop is a type of loop that continues to run its code so long as a certain command (run before each iteration) executes successfully.
Until (builtin): An until-loop is a type of loop that continues to run its code so long as a certain command (run before each iteration) executes unsuccessfully.
6. Choices (case and select)
Sometimes you just want to build application logic depending on the content of a variable but you already know all of the possible choices of content the variable can contain. You no longer need if's two yes or no blocks, but you want a block of code for every possible choice of content a variable can have. You could obviously attempt to emulate this behaviour using several if-statements:
$ if [[ $LANG = en* ]] > then echo "Hello!"; fi $ if [[ $LANG = fr* ]] > then echo "Salut!"; fi $ if [[ $LANG = de* ]] > then echo "Guten Tag!"; fi $ if [[ $LANG = nl* ]] > then echo "Hallo!"; fi $ if [[ $LANG = it* ]] > then echo "Ciao!"; fi
But let's say we like to actually make clean code, shall we? BASH provides a construct called case exactly for this kind of situation. A case construct basically enumerates several possible Glob Patterns and checks the content of your parameter against these:
$ case $LANG in > en*) echo "Hello!" ;; > fr*) echo "Salut!" ;; > de*) echo "Guten Tag!" ;; > nl*) echo "Hallo!" ;; > it*) echo "Ciao!" ;; > *) echo "I do not speak your language." ;; > esac
Each choice in a case statement starts with a pattern followed by a block of code that is to be executed if the string to be checked matches that pattern, which in turn is followed by two semi-colons. case stops matching patterns as soon as one was successful. Therefore, we can use the * pattern in the end to match any case that has not been caught by the other choices.
Another construct of choice is the select construct. This statement smells like a loop and is a convenience statement for generating a menu of choices that the user can choose from.
The user is presented by choices and asked to enter a number reflecting his choice. The code in the select block is then executed with a variable set to the choice the user made. If the user's choice was invalid, the variable is made empty:
$ echo "Which of these does not belong in the group?"; \ > select choice in Apples Pears Crisps Lemons Kiwis; do > if [[ $choice = Crisps ]] > then echo "Correct! Crisps are not fruit."; break; fi > echo "Errr... no. Try again." > done
The menu reappears so long as the break statement is not executed. In the example the break statement is only executed when the user makes the correct choice.
We can also use the PS3 variable to define the prompt the user replies on. Instead of showing the question before executing the select statement, we could choose to set the question as our prompt:
$ PS3="Which of these does not belong in the group (#)? " \ > select choice in Apples Pears Crisps Lemons Kiwis; do > if [[ $choice = Crisps ]] > then echo "Correct! Crisps are not fruit."; break; fi > echo "Errr... no. Try again." > done
In The Manual: Conditional Constructs
In the FAQ:
I want to check if [[ $var == foo or $var == bar or $var = more ... without repeating $var n times.
Case (builtin): This statement evaluates a parameter's value based on several given patterns (choices).
Select (builtin): The select statement offers the user the choice between several options and executes a block of code for the user's choice.