I set variables in a loop that's in a pipeline. Why do they disappear after the loop terminates? Or, why can't I pipe data to read?

In most shells, each command of a pipeline is executed in a separate SubShell. Non-working example:

# Works only in ksh88/ksh93, or zsh or bash 4.2 with lastpipe enabled
# In other shells, this will print 0
linecount=0

printf '%s\n' foo bar |
while IFS= read -r line
do
    linecount=$((linecount + 1))
done

echo "total number of lines: $linecount"

The reason for this potentially surprising behaviour, as described above, is that each SubShell introduces a new variable context and environment. The while loop above is executed in a new subshell with its own copy of the variable linecount created with the initial value of '0' taken from the parent shell. This copy then is used for counting. When the while loop is finished, the subshell copy is discarded, and the original variable linecount of the parent (whose value hasn't changed) is used in the echo command.

Different shells exhibit different behaviors in this situation:

More broken stuff:

# Bash 4
# The problem also occurs without a loop
printf '%s\n' foo bar | mapfile -t line
printf 'total number of lines: %s\n' "${#line[@]}" # prints 0

f() {
    if [[ -t 0 ]]; then
        echo "$1"
    else
        read -r var
    fi
}

f 'hello' | f
echo "$var" # prints nothing

Again, in both cases the pipeline causes read or some containing command to run in a subshell, so its effect is never witnessed in the parent process.

It should be stressed that this issue isn't specific to loops. It's a general property of all pipes, though the while/read loop might be considered the canonical example that crops up over and over when people read the help or manpage description of the read builtin and notice that it accepts data on stdin. They might recall that data redirected into a compound command is available throughout that command, but not understand why all the fancy process substitutions and redirects they run across in places like FAQ #1 are necessary. Naturally they proceed to put their funstuff directly into a pipeline, and confusion ensues.

Workarounds

For more related examples of how to read input and break it into words, see FAQ #1.


CategoryShell

BashFAQ/024 (last edited 2023-12-12 13:15:33 by 195)