ssh breaks out of while-loop in bash

BashSsh

Bash Problem Overview


I use this bash-code to upload files to a remote server, for normal files this works fine:

for i in `find devel/ -newer $UPLOAD_FILE`
do
	echo "Upload:" $i
	if [ -d $i ]
	then
		echo "Creating directory" $i
		ssh $USER@$SERVER "cd ${REMOTE_PATH}; mkdir -p $i"
		continue
	fi
	if scp -Cp $i $USER@$SERVER:$REMOTE_PATH/$i
	then
		echo "$i OK"
	else
		echo "$i NOK"
		rm ${UPLOAD_FILE}_tmp
	fi
done

The only problem is that for files with a space in the name, the for-loop fails, so I replaced the first line like this:

find devel/ -newer $UPLOAD_FILE | while read i
do
	echo "Upload:" $i
	if [ -d $i ]
	then
		echo "Creating directory" $i
		ssh $USER@$SERVER "cd ${REMOTE_PATH}; mkdir -p $i"
		continue
	fi
	if scp -Cp $i $USER@$SERVER:$REMOTE_PATH/$i
	then
		echo "$i OK"
	else
		echo "$i NOK"
		rm ${UPLOAD_FILE}_tmp
	fi
done

For some strange reason, the ssh-command breaks out of the while-loop, therefore the first missing directory is created fine, but all subsequent missing files/directories are ignored.

I guess this has something to do with ssh writing something to stdout which confuses the "read" command. Commenting out the ssh-command makes the loop work as it should.

Does anybody know why this happens and how one can prevent ssh from breaking the while-loop?

Bash Solutions


Solution 1 - Bash

The problem is that ssh reads from standard input, therefore it eats all your remaining lines. You can just connect its standard input to nowhere:

ssh $USER@$SERVER "cd ${REMOTE_PATH}; mkdir -p $i" < /dev/null

You can also use ssh -n instead of the redirection.

Solution 2 - Bash

Another approach is to loop over a FD other than stdin:

while IFS= read -u 3 -r -d '' filename; do
  if [[ -d $filename ]]; then
    printf -v cmd_str 'cd %q; mkdir -p %q' "$REMOTE_PATH" "$filename"
    ssh "$USER@$SERVER" "$cmd_str"
  else
    printf -v remote_path_str '%q@%q:%q/%q' "$USER" "$SERVER" "$REMOTE_PATH" "$filename"
    scp -Cp "$filename" "$remote_path_str"
  fi
done 3< <(find devel/ -newer "$UPLOAD_FILE" -print0)

The -u 3 and 3< operators are critical here, using FD 3 rather than the default FD 0 (stdin).

The approach given here -- using -print0, a cleared IFS value, and the like -- is also less buggy than the original code and the existing answer, which can't handle interesting filenames correctly. (Glenn Jackman's answer is close, but even that can't deal with filenames with newlines or filenames with trailing whitespace).

The use of printf %q is critical to generate commands which can't be used to attack the remote machine. Consider what would happen with a file named devel/$(rm -rf /)/hello with code which didn't have this paranoia.

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionRobby75View Question on Stackoverflow
Solution 1 - BashchorobaView Answer on Stackoverflow
Solution 2 - BashCharles DuffyView Answer on Stackoverflow