1961
Comment: link to FileDescriptor
|
2005
include example with `read -u` and `{var}<file`
|
Deletions are marked like this. | Additions are marked like this. |
Line 2: | Line 2: |
== I'm reading a file line by line and running ssh or ffmpeg, but everything after the first line is eaten! == | == I'm reading a file line by line and running ssh or ffmpeg, only the first line gets processed! == |
Line 6: | Line 6: |
# Non-working example while IFS= read -r file; do ffmpeg -i "$file" -vcodec libxvid -acodec libfaac -ar 32000 \ "${file%.avi}".mkv done < <(find . -name '*.avi') |
# Non-working example while IFS= read -r file; do ffmpeg -i "$file" -vcodec libxvid -acodec libfaac -ar 32000 "${file%.avi}".mkv done < <(find . -name '*.avi') |
Line 14: | Line 13: |
# Non-working example while read host; do ssh "$host" some command done <hostslist |
# Non-working example while read host; do ssh "$host" some command done <hostslist |
Line 22: | Line 21: |
Here's how you make it work: | Here's one way to make it work: |
Line 24: | Line 23: |
while IFS= read -r file; do ffmpeg </dev/null -i "$file" -vcodec libxvid -acodec libfaac -ar 32000 \ "${file%.avi}".mkv done < <(find . -name '*.avi') |
while IFS= read -r file; do ffmpeg -i "$file" -vcodec libxvid -acodec libfaac -ar 32000 "${file%.avi}".mkv </dev/null done < <(find . -name '*.avi') |
Line 30: | Line 28: |
Notice the redirection on the ffmpeg line: `</dev/null`. See the [[BashGuide/InputAndOutput#Redirection|redirection section]] of the BashGuide for more information on this. | Notice the [[BashGuide/InputAndOutput#Redirection|redirection]] on the ffmpeg line: `</dev/null`. The ssh example can be fixed the same way, or with the `-n` switch (at least with [[http://www.openssh.org/|OpenSSH]]). |
Line 32: | Line 30: |
The ssh example can be fixed the same way, or with the `-n` switch (at least with [[http://www.openssh.org/|OpenSSH]]). Sometimes with large loops it might be difficult to work out what's reading from stdin; or a program might change its behaviour when you add `</dev/null` to it. In this case you can make read use a different FileDescriptor that a random program is less likely to read from: |
Sometimes with large loops it might be difficult to work out what's reading from stdin, or a program might change its behaviour when you add `</dev/null` to it. In this case you can make read use a different FileDescriptor that a random program is less likely to read from: |
Line 36: | Line 32: |
while read <&3 line; do ...... done 3< file |
while read -r line <&3; do ... done 3<file |
Line 41: | Line 37: |
or use read's `-u` option (Bash only): | In bash, the `read` builtin can also be told to read directly from an fd (`-u fd`) without redirection, and since bash 4.1, an available fd can be assigned (`{var}<file`) instead of hard coding a file descriptor. |
Line 44: | Line 40: |
# Bash while read -u 3 line; do ...... done 3< file |
# bash 4.1+ while read -r -u "$fd" line; do ... done {fd}<file |
I'm reading a file line by line and running ssh or ffmpeg, only the first line gets processed!
When reading a file line by line, if a command inside the loop also reads stdin, it can exhaust the input file. For example:
# Non-working example while IFS= read -r file; do ffmpeg -i "$file" -vcodec libxvid -acodec libfaac -ar 32000 "${file%.avi}".mkv done < <(find . -name '*.avi')
# Non-working example while read host; do ssh "$host" some command done <hostslist
What's happening here? Let's take the first example. read reads a line from standard input (FD 0), puts it in the file parameter, and then ffmpeg is executed. Like any program you execute from BASH, ffmpeg inherits standard input, which for some reason it reads. I don't know why. But in any case, when ffmpeg reads stdin, it sucks up all the input from the find command, starving the loop.
Here's one way to make it work:
while IFS= read -r file; do ffmpeg -i "$file" -vcodec libxvid -acodec libfaac -ar 32000 "${file%.avi}".mkv </dev/null done < <(find . -name '*.avi')
Notice the redirection on the ffmpeg line: </dev/null. The ssh example can be fixed the same way, or with the -n switch (at least with OpenSSH).
Sometimes with large loops it might be difficult to work out what's reading from stdin, or a program might change its behaviour when you add </dev/null to it. In this case you can make read use a different FileDescriptor that a random program is less likely to read from:
while read -r line <&3; do ... done 3<file
In bash, the read builtin can also be told to read directly from an fd (-u fd) without redirection, and since bash 4.1, an available fd can be assigned ({var}<file) instead of hard coding a file descriptor.
# bash 4.1+ while read -r -u "$fd" line; do ... done {fd}<file