How can I capture the output and show it at the same time as Python?

Question

I have a rather long job that lasts several minutes and then restarts. The task displays various information, which I write as follows:

output = subprocess.Popen(cmd,stdout=subprocess.PIPE).communicate() 

The fact is that I get only all the output at a time. I would like to show the output as the program sends it to stdout, but still pushes it back to the buffer (I need to check the output for some lines). In Ruby, I would do it like this:

 IO.popen(cmd) do |io| io.each_line do |line| puts line buffer << line end end 

Answer 1

You can try something like this:

 cmd = ["./my_program.sh"] p = subprocess.Popen( cmd, shell=False, stdout=subprocess.PIPE) # launch the process while p.poll() is None: # check if the process is still alive out = p.stdout.readline() # if it is still alive, grab the output do_something_with(out) # do what you want with it 

Answer 2

You can read it one line at a time:

 from subprocess import Popen, PIPE p = Popen('grep -ir graph .', stdout=PIPE) while not p.returncode: s = p.stdout.readline() print s p.poll() 

Thus, you only block the time spent processing one line.

Answer 3

You can use the "tee" command. He does exactly what you need from him.
http://www.computerhope.com/unix/utee.htm