Differences between revisions 12 and 21 (spanning 9 versions)
Revision 12 as of 2014-12-30 21:57:03
Size: 3683
Editor: GreyCat
Comment: exit status masked by export, declare, typeset, local
Revision 21 as of 2016-12-09 23:53:20
Size: 5239
Editor: AladW
Comment: put output as comments
Deletions are marked like this. Additions are marked like this.
Line 2: Line 2:
Line 10: Line 11:
Line 24: Line 24:
Line 34: Line 33:
''Exercise 3: why aren't these two scripts identical?''
Line 35: Line 35:
''Exercise 3: why aren't these two scripts identical?''
Line 42: Line 41:
Line 50: Line 48:
''Exercise 4: why aren't '''these''' two scripts identical?''
Line 51: Line 50:
''Exercise 4: why aren't '''these''' two scripts identical?''
Line 58: Line 56:
Line 65: Line 62:
''Exercise 5: under what conditions will this fail?''
Line 66: Line 64:
''Exercise 5: under what conditions will this fail?''
Line 71: Line 68:

([[/Answers|Answers]])
([[BashFAQ/105/Answers|Answers]])
Line 82: Line 78:
Subshells from command substitution unset `set -e`, however (unless `inherit_errexit` is set with Bash 4.4):
Line 83: Line 80:
{{{#!highlight bash
set -e
foo=$(expr 1 - 1; true)
# Will run:
echo survived
}}}
Note that set -e is '''not''' unset for commands that are run asynchronously, for example with process substitution:

{{{#!highlight bash
set -e
mapfile foo < <(true; echo foo)
echo ${foo[-1]} # foo
mapfile foo < <(false; echo foo)
echo ${foo[-1]} # bash: foo: bad array subscript
}}}
Line 93: Line 105:
Line 96: Line 107:
Using [[ProcessSubstitution|Process substitution]], the exit code is also discarded as it is not visible from the main script:

{{{#!highlight bash
set -e
cat <(somecommand that fails)
echo survived
}}}
Using a pipe makes no difference, as only the ''rightmost'' process is considered:

{{{#!highlight bash
set -e
somecommand that fails | cat -
echo survived
}}}
`set -o pipefail` is a workaround by returning the exit code of the ''first'' failed process:

{{{#!highlight bash
set -e -o pipefail
failcmd1 | failcmd2 | cat -
# The following command will not be executed:
echo survived
}}}
though with pipefail in effect, code like this will sometimes cause an error, depending on whether the output of somecmd exceeds the size of the pipe buffer or not:

{{{#!highlight bash
set -e -o pipefail
somecmd | head -n1
# The following command will sometimes be executed, depending on how much output somecmd writes:
echo survived
}}}
Line 99: Line 140:

geirha's personal recommendation is to handle errors properly and not rely on the unreliable `set -e`.

Why doesn't set -e (or set -o errexit, or trap ERR) do what I expected?

set -e was an attempt to add "automatic error detection" to the shell. Its goal was to cause the shell to abort any time an error occurred, so you don't have to put || exit 1 after each important command.

That goal is non-trivial, because many commands intentionally return non-zero. For example,

  if [ -d /foo ]; then ...; else ...; fi

Clearly we don't want to abort when the conditional, [ -d /foo ], returns non-zero (because the directory does not exist) -- our script wants to handle that in the else part. So the implementors decided to make a bunch of special rules, like "commands that are part of an if test are immune", or "commands in a pipeline, other than the last one, are immune".

These rules are extremely convoluted, and they still fail to catch even some remarkably simple cases. Even worse, the rules change from one Bash version to another, as Bash attempts to track the extremely slippery POSIX definition of this "feature". When a SubShell is involved, it gets worse still -- the behavior changes depending on whether Bash is invoked in POSIX mode. Another wiki has a page that covers this in more detail. Be sure to check the caveats.

Exercise for the reader: why doesn't this example print anything?

   1 #!/usr/bin/env bash
   2 set -e
   3 i=0
   4 let i++
   5 echo "i is $i"

Exercise 2: why does this one sometimes appear to work? In which versions of bash does it work, and in which versions does it fail?

   1 #!/usr/bin/env bash
   2 set -e
   3 i=0
   4 ((i++))
   5 echo "i is $i"

Exercise 3: why aren't these two scripts identical?

   1 #!/usr/bin/env bash
   2 set -e
   3 test -d nosuchdir && echo no dir
   4 echo survived

   1 #!/usr/bin/env bash
   2 set -e
   3 f() { test -d nosuchdir && echo no dir; }
   4 f
   5 echo survived

Exercise 4: why aren't these two scripts identical?

   1 set -e
   2 f() { test -d nosuchdir && echo no dir; }
   3 f
   4 echo survived

   1 set -e
   2 f() { if test -d nosuchdir; then echo no dir; fi; }
   3 f
   4 echo survived

Exercise 5: under what conditions will this fail?

   1 set -e
   2 read -r foo < configfile

(Answers)

Even if you use expr(1) (which we do not recommend -- use arithmetic expressions instead), you still run into the same problem:

   1 set -e
   2 foo=$(expr 1 - 1)
   3 # The following command will not be executed:
   4 echo survived

Subshells from command substitution unset set -e, however (unless inherit_errexit is set with Bash 4.4):

   1 set -e
   2 foo=$(expr 1 - 1; true)
   3 # Will run:
   4 echo survived

Note that set -e is not unset for commands that are run asynchronously, for example with process substitution:

   1 set -e
   2 mapfile foo < <(true; echo foo)
   3 echo ${foo[-1]} # foo
   4 mapfile foo < <(false; echo foo)
   5 echo ${foo[-1]} # bash: foo: bad array subscript

Another pitfall associated with set -e occurs when you use commands that look like assignments but aren't, such as export, declare, typeset or local.

   1 set -e
   2 f() { local var=$(somecommand that fails); }
   3 f    # will not exit
   4 
   5 g() { local var; var=$(somecommand that fails); }
   6 g    # will exit

In function f, the exit status of somecommand is discarded. It won't trigger the set -e because the exit status of local masks it (the assignment to the variable succeeds, so local returns status 0). In function g, the set -e is triggered because it uses a real assignment which returns the exit status of somecommand.

Using Process substitution, the exit code is also discarded as it is not visible from the main script:

   1 set -e
   2 cat <(somecommand that fails)
   3 echo survived

Using a pipe makes no difference, as only the rightmost process is considered:

   1 set -e
   2 somecommand that fails | cat -
   3 echo survived

set -o pipefail is a workaround by returning the exit code of the first failed process:

   1 set -e -o pipefail
   2 failcmd1 | failcmd2 | cat -
   3 # The following command will not be executed:
   4 echo survived

though with pipefail in effect, code like this will sometimes cause an error, depending on whether the output of somecmd exceeds the size of the pipe buffer or not:

   1 set -e -o pipefail
   2 somecmd | head -n1
   3 # The following command will sometimes be executed, depending on how much output somecmd writes:
   4 echo survived

GreyCat's personal recommendation is simple: don't use set -e. Add your own error checking instead.

rking's personal recommendation is to go ahead and use set -e, but beware of possible gotchas. It has useful semantics, so to exclude it from the toolbox is to give into FUD.

geirha's personal recommendation is to handle errors properly and not rely on the unreliable set -e.

BashFAQ/105 (last edited 2021-03-11 06:07:25 by dsl-66-36-156-249)