Python.Popen subprocess hangs while collecting child output when child process does not exit

Question

When the process crashes abnormally or does not work at all, I still want to be able to collect what result he could create to this point.

The obvious solution to this code example is to kill the child process with os.kill, but in my real code the child is waiting for NFS and not responding to SIGKILL.

#!/usr/bin/python
import subprocess
import os
import time
import signal
import sys
child_script = """
#!/bin/bash
i=0
while [ 1 ]; do
    echo "output line $i"
    i=$(expr $i \+ 1)
    sleep 1
done
"""
childFile = open("/tmp/childProc.sh", 'w')
childFile.write(child_script)
childFile.close()

cmd = ["bash", "/tmp/childProc.sh"]
finish = time.time() + 3
p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, stdin=subprocess.PIPE)
while p.poll() is None:
    time.sleep(0.05)
    if finish < time.time():
        print "timed out and killed child, collecting what output exists so far"
        out, err = p.communicate()
        print "got it"
        sys.exit(0)

In this case, a print expression about the timing appears and the python script never exits or progresses. Does anyone know how I can do this differently and still get output from my child process

Answer 1

, bash CTRL-C, . SIGHUP SIGTERM :

cmd = ["bash", 'childProc.sh']
p = subprocess.Popen(cmd, stdout=subprocess.PIPE, 
                          stderr=subprocess.STDOUT, 
                          close_fds=True)
time.sleep(3)
print 'killing pid', p.pid
os.kill(p.pid, signal.SIGTERM)
print "timed out and killed child, collecting what output exists so far"
out  = p.communicate()[0]
print "got it", out

:

killing pid 5844
timed out and killed child, collecting what output exists so far
got it output line 0
output line 1
output line 2

Answer 2

POSIX . , , ...

import subprocess
import os
import time
import signal
import sys

pr, pw = os.pipe()
pid = os.fork () 

if pid: #parent
    os.close(pw)
    cmd = ["bash"]
    finish = time.time() + 3
    p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, stdin=pr, close_fds=True)
    while p.poll() is None:
        time.sleep(0.05)
        if finish < time.time():
            os.kill(p.pid, signal.SIGTERM)
            print "timed out and killed child, collecting what output exists so far"
            out, err = p.communicate()
            print "got it: ", out
            sys.exit(0)

else: #child
    os.close(pr)
    child_script = """
    #!/bin/bash
    while [ 1 ]; do
        ((++i))
        echo "output line $i"
        sleep 1
    done
    """
    os.write(pw, child_script)

Answer 3

stackoverflow : " " . python (2.5)

pipe.readline() pipe.communicate(), .

Answer 4

I had the same problem. I decided to solve the problem (after cleaning Google and searching for many related problems) by simply setting the following parameters when calling subprocess.Popen(or .call):

stdout=None

and

stderr=None

There are many problems with these functions, but in my particular case, I believe that it was stdoutfilled with a process that I called, and then led to a blocking condition. By setting them to None(against something like subprocess.PIPE), I believe that this should be avoided.

Hope this helps someone.