How to register atexit function in python multiprocessing subprocess?

Question

I have some subprocesses (using multiprocessing), and when they stop, each of them needs some final work. Something like the following that doesn't work though ...

import multiprocessing import atexit def final(): print "final work" def worker(): print 'Doing some work' atexit.register(final) if __name__ == '__main__': p = multiprocessing.Process(target=worker) p.start() p.join() 

So how can I do this?

Answer 1

Processes started using multiprocessing do not fail, and atexit functions are never called. You must start the cleaning manually through the try / finally block, for example:

 def worker(): try: print('Doing some work') finally: final() 

Or, if you really want to use atexit (see disadvantages below):

 def worker(): try: print('Doing some work') finally: atexit._run_exitfuncs() 

---

Why are subprocesses usually not executed?

Technically, since they exit using os._exit() , they skip any cleanup task (including atexit functions, __del__() and weak finalizers).

The reason for this decision is that the risks associated with cleaning are performed at the wrong time, in the wrong process. Most code that uses atexit (or other mechanisms) expects the code to be executed only once and only after the main interpreter exits (for example, you can use atexit to delete a temporary directory). By performing atexit functions every time a subprocess ends, you break that expectation and bad things can happen.

So, although you can run atexit functions through atexit._run_exitfuncs() , do not do this because the function is undocumented and because you cannot be the only atexit user (other third-party libraries can use this, and you can introduce errors in your code )