Exception handling when using the Python.Popen subprocess

Question

When working with open files, Python has a with syntax that ensures that the file closes when the block exits - regardless of exceptions, etc.

 with open('foo.txt') as f: foo = f.read() 

Since processes are also resources, I was wondering: is something like this possible or recommended when using Popen ? For example, should Popen.kill(); Popen.communicate() Popen.kill(); Popen.communicate() work in a finally clause - if I don't mind blocking until the process completes?

Answer 1

Starting with Python 3.2, Popen is a context manager.

from docs :

Popen objects are supported as context managers through the with statement: upon exit, the standard file descriptors are closed and the process waits.

This should do pretty much what you want.

This is the corresponding part from subprocess.py from the standard lib library in Python 3.4:

 def __enter__(self): return self def __exit__(self, type, value, traceback): if self.stdout: self.stdout.close() if self.stderr: self.stderr.close() if self.stdin: self.stdin.close() # Wait for the process to terminate, to avoid zombies. self.wait() 

Now you can do it in Python 2.7

 from subprocess import Popen class MyPopen(Popen): def __enter__(self): return self def __exit__(self, type, value, traceback): if self.stdout: self.stdout.close() if self.stderr: self.stderr.close() if self.stdin: self.stdin.close() # Wait for the process to terminate, to avoid zombies. self.wait() if __name__ == '__main__': with MyPopen(['ls']) as p: print(p) 

Answer 2

For 2.7, you can also use @contextlib.contextmanager :

 import contextlib @contextlib.contextmanager def manage_process(process): try: yield process finally: for stream in [process.stdout, process.stdin, process.stderr]: if stream: stream.close() process.wait() 

eg:

 with manage_process(Popen(['ls'])) as p: print(p) 

Answer 3

You can add only two custom methods for any class to implement compatibility using the with statement.

 class CustomObject(object): def __enter__(self): """ This method execudes when entering block. """ return thing_you_want_to_use def __exit__(self, type, value, traceback): """ This method execudes on block exit. """ # Tear things down.