Sympy lambda pass for multiprocessing .Pool.map

Question

I want to execute a symmetric lambda function in parallel. I dont know:

• why does it work in parallel, although this is a lambda function
• why it stops working when I try to execute without a pool.
• why does it work if i uncomment the first return in lambdify

And, obviously, the markdown preprocessor requires a line of text above the code, so this is code:

 from multiprocessing import Pool import sympy from sympy.abc import x def f(m): return m.lambdify()(1) class Mult(): def lambdify(self): # return sympy.lambdify(x, 2*x, 'numpy') self._lambdify = sympy.lambdify(x, 2 * x, 'numpy') return self._lambdify if __name__ == '__main__': with Pool() as pool: m = Mult() print(pool.map(f, [m])) print(pool.map(f, [m])) print(f(m)) print(pool.map(f, [m])) 

He prints:

 [2] [2] 2 PicklingError: Can't pickle <function <lambda> at 0x000000000DF0D048>: attribute lookup <lambda> on numpy failed 

(I cut the trace)

If I uncomment, it works fine:

 [2] [2] 2 [2] 

I tested only on Windows, and it works exactly the same as "numexpr" instead of "numpy".

Answer 1

A Mult object has no fields when it is created. Thus, it can be pickled using the pickle stock library. Then, when you call lambdify , you add the _lambdify attribute to the object containing the lambda expression that cannot be etched. This causes a crash in the map function

This explains why you can select an object and use Pool.map before calling lambdify and why it does not work after the call. When you uncomment a line in lambdify , you are not adding the attribute to the class, and the Mult object can still be pickled after calling lambdify .