Ctx parameter in multiprocessing.

Question

I am trying to use a queue from the multiprocessing.Queue module. The implementation ( https://docs.python.org/3.4/library/multiprocessing.html#exchanging-objects-between-processes ) uses

q = Queue() 

as an example to create an instance. If I try this, I get the following error:

 TypeError: __init__() missing 1 required keyword-only argument: 'ctx' 

As a result of the problem, the problem arose:

http://bugs.python.org/issue21367

How to find out if this is fixed? Is it impossible to use multiprocessing? If not, how do I get the required ctx object (and what is it?)

Answer 1

It looks like you are not importing Queue directly from multiprocessing . When the contexts were introduced, most of the objects you imported from the top-level multiprocessing package became functions that internally receive the context and then pass this to the base class initializer, not the classes themselves. For example, here is what multiprocessing.Queue :

 def Queue(self, maxsize=0): '''Returns a queue object''' from .queues import Queue return Queue(maxsize, ctx=self.get_context()) 

If you want to directly import multiprocessing.queues.Queue and try to instantiate it, you will get the error message that you see. But it should work fine if you import it directly from multiprocessing .

The context object tells multiprocessing which of the available methods to start subprocesses is used. multiprocessing.Queue uses multiprocessing.Lock internally, which must correctly know the correct context.

Answer 2

Thus, the Queue multiprocessor class should be inherited from Python 3.4 and on:

 from multiprocessing.queues import Queue class BlockedQueue(Queue): def __init__(self, maxsize=-1, block=True, timeout=None): self.block = block self.timeout = timeout super().__init__(maxsize, ctx=multiprocessing.get_context())