Multiprocessor Timeout Decorator

Question

I have this decorator taken directly from an example I found on the net:

class TimedOutExc(Exception): pass def timeout(timeout): def decorate(f): def handler(signum, frame): raise TimedOutExc() def new_f(*args, **kwargs): old = signal.signal(signal.SIGALRM, handler) signal.alarm(timeout) try: result = f(*args, **kwargs) except TimedOutExc: return None finally: signal.signal(signal.SIGALRM, old) signal.alarm(0) return result new_f.func_name = f.func_name return new_f return decorate 

It throws an exception if the function f expires.

Well, it works, but when I use this decorator for the multiprocessing function and stops due to a timeout, it does not interrupt the processes involved in the calculation. How can i do this?

I do not want to throw an exception and stop the program. Basically what I want, when f times, return None, and then terminate the processes involved.

Answer 1

Although I agree with the main point of Aaron's answer, I would like to tell a little.

Processes started using multiprocessing must be stopped in a function that will be decorated; I do not think that this can be done at all and simply from the decorator itself (the decorated function is the only entity that knows what calculations it launched).

Instead of decorating the catch SIGALARM function, you can also catch your own TimedOutExc exception - it can be more flexible. Then your example will be as follows:

 class TimedOutExc(Exception): """ Raised when a timeout happens """ def timeout(timeout): """ Return a decorator that raises a TimedOutExc exception after timeout seconds, if the decorated function did not return. """ def decorate(f): def handler(signum, frame): raise TimedOutExc() def new_f(*args, **kwargs): old_handler = signal.signal(signal.SIGALRM, handler) signal.alarm(timeout) result = f(*args, **kwargs) # f() always returns, in this scheme signal.signal(signal.SIGALRM, old_handler) # Old signal handler is restored signal.alarm(0) # Alarm removed return result new_f.func_name = f.func_name return new_f return decorate @timeout(10) def function_that_takes_a_long_time(): try: # ... long, parallel calculation ... except TimedOutExc: # ... Code that shuts down the processes ... # ... return None # Or exception raised, which means that the calculation is not complete 

Answer 2

I doubt that this can be done using a decorator: a decorator is a wrapper for a function; function - black box. There is no connection between the decorator and the function that it wraps.

What you need to do is rewrite the function code to use the SIGALRM handler to terminate all running processes.