Bash Pitfalls

TableOfContents

1. for i in `ls *.mp3`

One of the most common mistakes ["BASH"] programmers make is to write a loop like this:

•  for i in `ls *.mp3`; do     # Wrong!
      some command $i          # Wrong!
   done

This breaks when the user has a file with a space in its name. Why? Because the output of the ls *.mp3 command substitution undergoes word splitting. Assuming we have a file named 01 - Don't Eat the Yellow Snow.mp3 in the current directory, the for loop will iterate over each word in the resulting file name (namely: "01", "-", "Don't", "Eat", and so on).

You can't double-quote the substitution either:

•  for i in "`ls *.mp3`"; do   # Wrong!
   ...

This causes the entire output of the ls command to be treated as a single word, and instead of iterating over each file name in the output list, the loop will only execute once, with i taking on a value which is the concatenation of all the file names (with spaces between them).

In addition to this, the use of ls is just plain unnecessary. It's an external command, which simply isn't needed to do the job. So, what's the right way to do it?

•  for i in *.mp3; do         # Right!
     some command "$i"
   done

Let Bash expand the list of filenames for you. The expansion will not be subject to word splitting. Each filename that's matched by the *.mp3 pattern will be treated as a separate word and, the loop will iterate once per file name.

The astute reader will notice the double quotes in the second line. This leads to our second common pitfall.

2. cp $file $target

What's wrong with the command shown above? Well, nothing, if you happen to know in advance that $file and $target have no white space in them.

But if you don't know that in advance, or if you're paranoid, or if you're just trying to develop good habits, then you should quote your variable references to avoid having them undergo word splitting.

•  mv "$file" "$target"

Without the double quotes, you'll get a command like mv 01 - Don't Eat the Yellow Snow.mp3 /mnt/usb and then you'll get errors like mv: cannot stat `01': No such file or directory. With the double quotes, all's well.

3. [ $foo = "bar" ]

This is really the same as the previous pitfall, but I repeat it because it's so important. In the example above, the quotes are in the wrong place. You do not need to quote a string literal in bash. But you should quote your variables if you aren't sure whether they could contain white space.

•  [ "$foo" = bar ]       # Right!

Another way you could write this in bash involves the [[ keyword, which extends and embraces the old test command (also known as [).

•  [[ $foo = bar ]]       # Also right!

You don't need to quote variable references within [[ ]] because they don't undergo word splitting in that context. On the other hand, quoting them won't hurt anything either.

4. [ "$foo" = bar && "$bar" = foo ]

You can't use && inside the old test (or [) command. The Bash parser sees && outside of [[ ]] or (( )) and breaks your command into two commands, before and after the &&. Use one of these instead:

•  [ "$foo" = bar -a "$bar" = foo ]       # Right!
   [ "$foo" = bar ] && [ "$bar" = foo ]   # Also right!
   [[ $foo = bar && $bar = foo ]]         # Also right!

5. [[ $foo > 7 ]]

The [[ ]] operator is not used for an ArithmeticExpression. It's used for strings only. If you want to do a numeric comparison against the constant 7, you must use (( )) instead:

•  ((foo > 7))                            # Right!

6. grep foo bar | while read line; do ((count++)); done

The code above looks OK at first glance, doesn't it? Sure, it's just a poor implementation of grep -c, but it's intended as a simplistic example. So why doesn't it work? The variable count will be unchanged after the loop terminates, much to the surprise of Bash developers everywhere.

The reason this code does not work as expected is because each command in a pipeline is executed in a separate subshell. The changes to the count variable within the loop's subshell aren't reflected within the parent shell (the script in which the code occurs).

For solutions to this, please see [wiki:BashFaq Bash FAQ #24].