Ssh breaks out of while loop in bash

Question

I use this bash code to upload files to a remote server, for normal files this works great:

for i in `find devel/ -newer $UPLOAD_FILE` do echo "Upload:" $i if [ -d $i ] then echo "Creating directory" $i ssh $USER@$SERVER "cd ${REMOTE_PATH}; mkdir -p $i" continue fi if scp -Cp $i $USER@$SERVER:$REMOTE_PATH/$i then echo "$i OK" else echo "$i NOK" rm ${UPLOAD_FILE}_tmp fi done 

The only problem is that for files with a space in the name, for-loop fails, so I replaced the first line as follows:

 find devel/ -newer $UPLOAD_FILE | while read i do echo "Upload:" $i if [ -d $i ] then echo "Creating directory" $i ssh $USER@$SERVER "cd ${REMOTE_PATH}; mkdir -p $i" continue fi if scp -Cp $i $USER@$SERVER:$REMOTE_PATH/$i then echo "$i OK" else echo "$i NOK" rm ${UPLOAD_FILE}_tmp fi done 

For some strange reason, the ssh command breaks out of the while loop, so the first missing directory is created perfectly, but all subsequent missing files / directories are ignored.

I assume this is due to the fact that ssh writes something to stdout, which confuses the read command. Commenting on the ssh command, the loop works as it should.

Does anyone know why this is happening, and how can ssh be prevented from breaking a while loop?

Answer 1

The problem is that ssh is read from standard input, so it eats all the remaining lines. You can simply connect your standard input to nowhere:

 ssh $USER@$SERVER "cd ${REMOTE_PATH}; mkdir -p $i" < /dev/null 

You can also use ssh -n instead of redirecting.

Answer 2

Another approach is to loop over FD, other than stdin:

 while IFS= read -u 3 -r -d '' filename; do if [[ -d $filename ]]; then printf -v cmd_str 'cd %q; mkdir -p %q' "$REMOTE_PATH" "$filename" ssh "$USER@$SERVER" "$cmd_str" else printf -v remote_path_str '%q@%q:%q/%q' "$USER" "$SERVER" "$REMOTE_PATH" "$filename" scp -Cp "$filename" "$remote_path_str" fi done 3< <(find devel/ -newer "$UPLOAD_FILE" -print0) 

The -u 3 and 3< operators are crucial here, using FD 3 rather than the standard FD 0 (stdin).

The approach given here is using -print0 , a cleared IFS value, etc. - also less erroneous than the source code and the existing answer, which cannot correctly handle interesting file names. (Glenn Jackman's answer is close, but even this cannot deal with file names with new characters or file names with a trailing space).

Using printf %q is critical to creating commands that cannot be used to attack a remote machine. Think about what happens with a file called devel/$(rm -rf /)/hello with code that did not have this paranoia.

Answer 3

In addition to choroba answer , do not use a for loop to read file names:

 find devel/ -newer $UPLOAD_FILE | while read -ri do ...