Best way to pass local () output to stdin remote start () commands in Fabric?

Question

Is there an easy way to output output from local commands to remote (and vice versa)?

I always just loaded into the file, moved the file and read it ... but it seems like it could be easier.

For simpler situations, just grab the output and use string interpolation:

ip = local('hostname -i') run('Script was run from ip: %s' % ip) 

But when the output either needs to be escaped in order to be safe on the command line, and / or must come from stdin, it is a bit more complicated.

If the output of bash is -safe, then something like run('echo "%s" | mycmd' % ip) will do what I am looking for (which I think implies the equivalent question would be "there is a simple bash way - escape strings? "), but it looks like there must be a" correct path "to provide the remote stdin.

Edit:

To clarify using long-necked inputs, there are a number of potential problems with simple string interpolation: classic shell problems (for example, the output may contain "; rm -rf / ), but also (and more realistically, in my case) the output may contain quotation marks ( both single and double).

I think just executing run("echo '%s' | cmd" % output.replace("'", "'\\''") should work, but there may be cases of edges that are skipped.

As I mentioned above, this looks like the type of thing that the fabric can handle more elegantly for me by directly sending the string to run () stdin (although maybe I was just spoiled by the fact that it handled everything else so beautifully :)

Answer 1

You can send remote stdin with fexpect, an extension of my fabric . this also sends the file, but hides it behind the api. You should have slipped away anyway.

Answer 2

I did this once to send a (binary) stream to a remote server.

This is a bit hacky as it goes deep into the paramiko fabric and channels, and there may be unchecked edges, but basically it does the job

 def remote_pipe(local_command, remote_command, buf_size=1024*1024): '''executes a local command and a remote command (with fabric), and sends the local stdout to the remote stdin''' local_p= subprocess.Popen(local_command, shell=True, stdout=subprocess.PIPE) channel= default_channel() #fabric function channel.set_combine_stderr(True) channel.settimeout(2) channel.exec_command( remote_command ) try: read_bytes= local_p.stdout.read(buf_size) while read_bytes: channel.sendall(read_bytes) read_bytes= local_p.stdout.read(buf_size) except socket.error: local_p.kill() #fail to send data, let see the return codes and received data... local_ret= local_p.wait() received= channel.recv(buf_size) channel.shutdown_write() channel.shutdown_read() remote_ret= channel.recv_exit_status() if local_ret!=0 or remote_ret!=0: raise Exception("remote_pipe failed. Local retcode: {0} Remote retcode: {1} output: {2}".format(local_ret, remote_ret, received)) 

If someone wants to make a change, this is part of btrfs-send-snapshot

Answer 3

This is a slightly improved version of @goncalopp's answer:

 def remote_pipe(local_command, remote_command, buffer_size=1024*1024, channel_timeout=60): '''executes a local command and a remote command (with fabric), and sends the local stdout to the remote stdin''' local_process = Popen(local_command, shell=True, stdout=PIPE) channel = default_channel() # Fabric function channel.set_combine_stderr(True) channel.settimeout(channel_timeout) channel.exec_command(remote_command) try: bytes_to_send = local_process.stdout.read(buffer_size) while bytes_to_send: channel.sendall(bytes_to_send) bytes_to_send = local_process.stdout.read(buffer_size) except socket.error: # Failed to send data, let see the return codes and received data... local_process.kill() local_returncode = local_process.wait() channel.shutdown_write() remote_output = "" try: bytes_received = channel.recv(buffer_size) while bytes_received: remote_output += bytes_received bytes_received = channel.recv(buffer_size) except socket.error: pass channel.shutdown_read() remote_returncode = channel.recv_exit_status() print(remote_output) if local_returncode != 0 or remote_returncode != 0: raise Exception("remote_pipe() failed, local return code: {0}, remote return code: {1}".format(local_returncode, remote_returncode, remote_output)) 

In addition to readability, the improvement lies in the fact that it does not interrupt with the socket timeout if the remote command produces less buffer_size bytes and that it prints the full output of the remote command.