Getting ssh to execute a command in the background on target machine

This is a follow-on question to the How do you use ssh in a shell script? question. If I want to execute a command on the remote machine that runs in the background on that machine, how do I get the ssh command to return? When I try to just include the ampersand (&) at the end of the command it just hangs. The exact form of the command looks like this:

ssh user@target "cd /some/directory; program-to-execute &"

Any ideas? One thing to note is that logins to the target machine always produce a text banner and I have SSH keys set up so no password is required.

I had this problem in a program I wrote a year ago -- turns out the answer is rather complicated. You'll need to use nohup as well as output redirection, as explained in the wikipedia artcle on nohup, copied here for your convenience.

> Nohuping backgrounded jobs is for > example useful when logged in via SSH, > since backgrounded jobs can cause the > shell to hang on logout due to a race > condition [2]. This problem can also > be overcome by redirecting all three > I/O streams: > > nohup myprogram > foo.out 2> foo.err < /dev/null &

This has been the cleanest way to do it for me:-

ssh -n -f user@host "sh -c 'cd /whereever; nohup ./whatever > /dev/null 2>&1 &'"

The only thing running after this is the actual command on the remote machine

Redirect fd's

Output needs to be redirected with &>/dev/null which redirects both stderr and stdout to /dev/null and is a synonym of >/dev/null 2>/dev/null or >/dev/null 2>&1.

Parantheses

The best way is to use sh -c '( ( command ) & )' where command is anything.

ssh askapache 'sh -c "( ( nohup chown -R ask:ask /www/askapache.com &>/dev/null ) & )"'

Nohup Shell

You can also use nohup directly to launch the shell:

ssh askapache 'nohup sh -c "( ( chown -R ask:ask /www/askapache.com &>/dev/null ) & )"'

Nice Launch

Another trick is to use nice to launch the command/shell:

ssh askapache 'nice -n 19 sh -c "( ( nohup chown -R ask:ask /www/askapache.com &>/dev/null ) & )"'

If you don't/can't keep the connection open you could use screen, if you have the rights to install it.

user@localhost $ screen -t remote-command
user@localhost $ ssh user@target # now inside of a screen session
user@remotehost $ cd /some/directory; program-to-execute &

To detach the screen session: ctrl-a d

To list screen sessions:

screen -ls

To reattach a session:

screen -d -r remote-command

Note that screen can also create multiple shells within each session. A similar effect can be achieved with tmux.

user@localhost $ tmux
user@localhost $ ssh user@target # now inside of a tmux session
user@remotehost $ cd /some/directory; program-to-execute &

To detach the tmux session: ctrl-b d

To list screen sessions:

tmux list-sessions

To reattach a session:

tmux attach <session number>

The default tmux control key, 'ctrl-b', is somewhat difficult to use but there are several example tmux configs that ship with tmux that you can try.

I just wanted to show a working example that you can cut and paste:

ssh REMOTE "sh -c \"(nohup sleep 30; touch nohup-exit) > /dev/null &\""

I think you'll have to combine a couple of these answers to get what you want. If you use nohup in conjunction with the semicolon, and wrap the whole thing in quotes, then you get:

ssh user@target "cd /some/directory; nohup myprogram > foo.out 2> foo.err < /dev/null"

which seems to work for me. With nohup, you don't need to append the & to the command to be run. Also, if you don't need to read any of the output of the command, you can use

ssh user@target "cd /some/directory; nohup myprogram > /dev/null 2>&1"

to redirect all output to /dev/null.

Quickest and easiest way is to use the 'at' command:

ssh user@target "at now -f /home/foo.sh"

You can do this without nohup:

ssh user@host 'myprogram >out.log 2>err.log &'

This worked for me may times:

ssh -x remoteServer "cd yourRemoteDir; ./yourRemoteScript.sh </dev/null >/dev/null 2>&1 & " 

You can do it like this...

sudo /home/script.sh -opt1 > /tmp/script.out &

It appeared quite convenient for me to have a remote tmux session using the tmux new -d <shell cmd> syntax like this:

ssh someone@elsewhere 'tmux new -d sleep 600'

This will launch new session on elsewhere host and ssh command on local machine will return to shell almost instantly. You can then ssh to the remote host and tmux attach to that session. Note that there's nothing about local tmux running, only remote!

Also, if you want your session to persist after the job is done, simply add a shell launcher after your command, but don't forget to enclose in quotes:

ssh someone@elsewhere 'tmux new -d "~/myscript.sh; bash"'

Actually, whenever I need to run a command on a remote machine that's complicated, I like to put the command in a script on the destination machine, and just run that script using ssh.

For example:

# simple_script.sh (located on remote server)

#!/bin/bash

cat /var/log/messages | grep <some value> | awk -F " " '{print $8}'

And then I just run this command on the source machine:

ssh user@ip "/path/to/simple_script.sh"

I was trying to do the same thing, but with the added complexity that I was trying to do it from Java. So on one machine running java, I was trying to run a script on another machine, in the background (with nohup).

From the command line, here is what worked: (you may not need the "-i keyFile" if you don't need it to ssh to the host)

ssh -i keyFile user@host bash -c "\"nohup ./script arg1 arg2 > output.txt 2>&1 &\""

Note that to my command line, there is one argument after the "-c", which is all in quotes. But for it to work on the other end, it still needs the quotes, so I had to put escaped quotes within it.

From java, here is what worked:

ProcessBuilder b = new ProcessBuilder("ssh", "-i", "keyFile", "bash", "-c",
 "\"nohup ./script arg1 arg2 > output.txt 2>&1 &\"");
Process process = b.start();
// then read from process.getInputStream() and close it.

It took a bit of trial & error to get this working, but it seems to work well now.

YOUR-COMMAND &> YOUR-LOG.log &    

This should run the command and assign a process id you can simply tail -f YOUR-LOG.log to see results written to it as they happen. you can log out anytime and the process will carry on

If you are using zsh then use program-to-execute &! is a zsh-specific shortcut to both background and disown the process, such that exiting the shell will leave it running.

First follow this procedure:

Log in on A as user a and generate a pair of authentication keys. Do not enter a passphrase:

a@A:~> ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/a/.ssh/id_rsa): 
Created directory '/home/a/.ssh'.
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /home/a/.ssh/id_rsa.
Your public key has been saved in /home/a/.ssh/id_rsa.pub.
The key fingerprint is:
3e:4f:05:79:3a:9f:96:7c:3b:ad:e9:58:37:bc:37:e4 a@A

Now use ssh to create a directory ~/.ssh as user b on B. (The directory may already exist, which is fine):

a@A:~> ssh b@B mkdir -p .ssh
b@B's password: 

Finally append a's new public key to b@B:.ssh/authorized_keys and enter b's password one last time:

a@A:~> cat .ssh/id_rsa.pub | ssh b@B 'cat >> .ssh/authorized_keys'
b@B's password: 

From now on you can log into B as b from A as a without password:

a@A:~> ssh b@B

then this will work without entering a password

ssh b@B "cd /some/directory; program-to-execute &"

I think this is what you need: At first you need to install sshpass on your machine. then you can write your own script:

while read pass port user ip; do
sshpass -p$pass ssh -p $port $user@$ip <<ENDSSH1
    COMMAND 1
    .
    .
    .
    COMMAND n
ENDSSH1
done <<____HERE
    PASS    PORT    USER    IP
      .      .       .       .
      .      .       .       .
      .      .       .       .
    PASS    PORT    USER    IP    
____HERE