Diff for "BashFAQ/063"

My ssh client hangs when I try to run a remote background job!

The following will not do what you expect:

   ssh me@remotehost 'sleep 120 &'
   # Client hangs for 120 seconds

This is a "feature" of OpenSSH. The client will not close the connection as long as the remote end's terminal still is still in use -- and in the case of sleep 120 &, stdout and stderr are still connected to the terminal.

The immediate answer to your question -- "How do I get the client to disconnect so I can get my shell back?" -- is to kill the ssh client. You can do this with the kill or pkill commands, of course; or by sending the INT signal (usually Ctrl-C) for a non-interactive ssh session (as above); or by pressing <Enter><~><.> (Enter, Tilde, Period) in the client's terminal window for an interactive remote shell.

The long-term workaround for this is to ensure that all the file descriptors are redirected to a log file (or /dev/null) on the remote side:

   ssh me@remotehost 'sleep 120 >/dev/null 2>&1 &'
   # Client should return immediately

This also applies to restarting daemons on some legacy Unix systems.

   ssh root@hp-ux-box   # Interactive shell
   ...                  # Discover that the problem is stale NFS handles
   /sbin/init.d/nfs.client stop   # autofs is managed by this script and
   /sbin/init.d/nfs.client start  # killing it on HP-UX is OK (unlike Linux)
   exit
   # Client hangs -- use Enter ~ . to kill it.

Please note that allowing root to log in over SSH is a very bad security practice. If you must do this, then create a single script that does all the commands you want, with no command line options, and then configure the sudoers file to grant a single user the right to run the mentioned script with no password required. This will ensure that you know which commands need run regularly, and that if the regular account is compromised, the damage which can be incurred is quantified.

The legacy Unix /sbin/init.d/nfs.client script runs daemons in the background but leaves their stdout and stderr attached to the terminal (and they don't fully self-daemonize). The solution is either to fix the Unix vendor's broken init script, or to kill the ssh client process after this happens. The author of this article uses the latter approach.

Another approach, which may work for your operating system and SSH server is to use nohup and redirect the stdin/stdout file descriptors.

   ssh me@remotehost 'nohup sleep 120 >/dev/null </dev/null 2>&1 &'

If you were hoping to see the output of the command you were running, then run two ssh sessions, one to run your command and write to a log file, and the second to review the log file ( tail -f ).