Bring me a long description.
In perl, I want to execute a simple remote command and capture output using perl :: Expect. However, I found that my result array is sometimes empty. I have experimented a lot and believe that the problem is that my remote ssh connection ends successfully before the remote command can shift to a thick one.
- the behavior is the same regardless of whether perl :: Expect or ssh is executed directly from the bash command line.
- All hosts in my data center have this behavior. The more the host loads, the more often the error occurs. For heavy hosts, I do not get the result in about 15% of cases. The error rate is close to 0 on lighter hosts.
- The remote command does not matter.
- I know that I am passing a password prompt and to a remote host. Using 'ssh -vvv' I can see that the remote command is being run every time.
- I experimented by executing a dummy code command and checking the remote return code in the hope of flushing the stdout buffer. Nothing was reliable.
In the following examples, the username / group / host name has been changed. In addition, the output is an unedited cut and paste.
> while true; do echo "start"; ssh -l user host 'date; ls -l abc; echo $? '; echo "end"; done
start
Password:
Mon Oct 6 13:06:51 PDT 2014
-rw-r - r-- 1 user group 34538 Dec 6 2012 abc
0
end
start
Password:
end
start
Password:
Mon Oct 6 13:06:54 PDT 2014
end
If I use a remote command for a dummy file, stdout will also be reset, and I will surely get output from my command every time. Unfortunately, this is a hacker job.
while true; do echo "start"; ssh -l user host 'date | tee -a /tmp/date.txt '; echo "end"; done
Don't dwell on the perl details in this example. An interesting bit is the @output returned array. In perl, I have the following command:
local :: rc :: Utils :: executeRemoteCommand ($ cmd, $ pass, 0, undef, undef, \ @ output)
which eventually drills it to trigger this:
my $ ssh = Expect-> spawn ($ command);
The result is a $ command:
ssh -l user host "ls -l abc; echo 'DONE'; echo $?"
Successful 'print Dumper \ @output':
$VAR1 = [ #0 " ", #1 "-rw-r--r-- 1 user group 34538 Dec 6 2012 abc", #2 "DONE", #3 0 ];
'Print Dumper \ @output' failed:
$VAR1 = [ #0 " ", #1 "DONE", #2 0 ];
Note that the result from 'ls' was not captured, but the echo of "DONE" was. I also saw where the output array has the result ls, but not an echo. I also saw where both results were missing.
Any insight would be very helpful. I'm currently at a dead end and should now program these remote calls into a loop that checks for the expected result. This is a terrible hack. I would rather fix the root cause.
thanks a lot