Python, parallelization with joblib: Delay with multiple arguments

I use something similar to the following to parallelize a for loop over two matrices

from joblib import Parallel, delayed import numpy def processInput(i,j): for k in range(len(i)): i[k] = 1 for t in range(len(b)): j[t] = 0 return i,j a = numpy.eye(3) b = numpy.eye(3) num_cores = 2 (a,b) = Parallel(n_jobs=num_cores)(delayed(processInput)(i,j) for i,j in zip(a,b)) 

but I get the following error: too many values ​​to unpack (2 expected)

Is there a way to return 2 values ​​with a delay? Or what solution do you propose?

In addition, the OP bit, is there a more compact way, for example, the following (which does not actually modify anything) for processing matrices?

 from joblib import Parallel, delayed def processInput(i,j): for k in i: k = 1 for t in b: t = 0 return i,j 

I would like to avoid using has_shareable_memory anyway to avoid possible bad interactions in the actual script and lower specs (?)

+5
source share
1 answer

Perhaps too late, but as an answer to the first part of your question: Just return the tuple in the delayed function.

 return (i,j) 

And for a variable holding the output of all your deferred functions

 results = Parallel(n_jobs=num_cores)(delayed(processInput)(i,j) for i,j in zip(a,b)) 

Now the results are a list of tuples, each of which contains some (i, j), and you can simply iterate over the results.

+4
source

Source: https://habr.com/ru/post/1258620/


All Articles