Tell me all about 2>&1 -- what's the difference between 2>&1 >foo and >foo 2>&1, and when do I use which?
Bash processes all redirections from left to right, in order. And the order is significant. Moving them around within a command may change the results of that command.
If all you want is to send both standard output and standard error to the same file, use this:
# Bourne foo >file 2>&1 # Sends both stdout and stderr to file.
Here's a simple demonstration of what's happening:
# POSIX
foo() {
echo "This is stdout"
echo "This is stderr" 1>&2
}
foo >/dev/null 2>&1 # produces no output
foo 2>&1 >/dev/null # writes "This is stderr" on the screenWhy do the results differ? In the first case, >/dev/null is performed first, and therefore the standard output of the command is sent to /dev/null. Then, the 2>&1 is performed, which causes standard error to be sent to the same place that standard output is already going. So both of them are discarded.
In the second example, 2>&1 is performed first. This means standard error is sent to wherever standard output happens to be going -- in this case, the user's terminal. Then, standard output is sent to /dev/null and is therefore discarded. So when we run foo the second time, we see only its standard error, not its standard output.
The redirection chapter in the guide explains why we use a duplicate file descriptor rather than opening /dev/null twice. In the specific case of /dev/null it doesn't actually matter because all writes are discarded, but when we write to a log file, it matters very much indeed.
There are times when we really do want 2>&1 to appear first -- for one example of this, see FAQ #40.
There are other times when we may use 2>&1 without any other redirections. Consider:
# Bourne find ... 2>&1 | grep "some error"
In this example, we want to search find's standard error (as well as its standard output) for the string "some error". The 2>&1 in the piped command forces standard error to go into the pipe along with standard output. (When pipes and redirections are mixed in this way, remember: the pipe is done first, before any redirections. So find's standard output is already set to point to the pipe before we process the 2>&1 redirection.)
If we wanted to read only standard error in the pipe, and discard standard output, we could do it like this:
# Bourne find ... 2>&1 >/dev/null | grep "some error"
The redirections in that example are processed thus:
First, the pipe is created. find's output is sent to it.
Next, 2>&1 causes find's standard error to go to the pipe as well.
Finally, >/dev/null causes find's standard output to be discarded, leaving only stderr going into the pipe.
A related question is FAQ #47, which discusses how to send stderr to a pipeline.
See Making sense of the copy descriptor operator for a more graphical explanation.
If you're still confused...
If you're still confused at this point, it's probably because you started out with a misconception about how FDs work, and you haven't been able to drop that misconception yet. Don't worry -- it's an extremely common misconception, and you're not alone. Let me try to explain....
Many people think that 2>&1 somehow "unites" or "ties together" or "marries" the two FDs, so that any change to one of them becomes a change to the other. This is not the case. And this is where the confusion comes from, for many people.
2>&1 only changes FD2 to point to "that which FD1 points to at the moment"; it does not actually make FD2 point to FD1 itself. Note that "2" and "1" have different meanings due to the way they are used: "2", which occurs before ">&" means the actual FD2, but "1", which occurs after ">&", means "that which FD1 currently points to", rather than FD1 itself. (If reversed, as in "1>&2", then 1 means FD1 itself, and 2 means "that which FD2 currently points to".)
Analogies may help. One analogy is to think of FDs as being like C pointers.
int some_actual_integer;
int *fd1, *fd2;
fd1 = &some_actual_integer; /* Analogous to 1>file */
fd2 = fd1; /* Analogous to 2>&1 */
fd1 = NULL; /* Analogous to 1>&- */
/* At this point, fd2 is still pointing to the actual memory location.
The fact that fd1 and fd2 both *used to* point to the same place is
not relevant. We can close or repoint one of them, without affecting
the other. */Another analogy is to think of them like hardlinks in a file system.
touch some_real_file
ln some_real_file fd1 # Make fd1 a link to our file
ln fd1 fd2 # Make fd2 another link to our file
rm fd1 # Remove the fd1 link, but fd2 is not
# affected
# At this point we still have a file with two links: "some_real_file"
# and "fd2".Or like symbolic links -- but we have to be careful with this analogy.
touch some_real_file
ln -s some_real_file fd1 # Make fd1 a SYMlink to our file
ln -s "$(readlink fd1)" fd2 # Make fd2 symlink to the same thing that
# fd1 is a symlink to.
rm fd1 # Remove fd1, but fd2 is untouched.
# Requires the nonstandard "readlink" program.
# Result is:
lrwxrwxrwx 1 wooledg wooledg 14 Mar 25 09:19 fd2 -> some_real_file
-rw-r--r-- 1 wooledg wooledg 0 Mar 25 09:19 some_real_file
# If we had attempted to use "ln -s fd1 fd2" in this analogy, we would have
# FAILED badly. This isn't how FDs work; rather, it's how some people
# THINK they work. And it's wrong.Other analogies include thinking of FDs as hoses. Think of files as barrels full of water (or empty, or half full). You can put a hose in a barrel in order to dump more water into it. You can put two hoses into the same barrel, and they can both dump water into the same barrel. You can then remove one of those hoses, and that doesn't cause the other hose to go away. It's still there.