Unpacking the result of a deferred function

Question

During the conversion of my program with a delay, I came across a frequently used programming pattern that does not work with a delay. Example:

from dask import delayed @delayed def myFunction(): return 1,2 a, b = myFunction() a.compute() 

Raises: TypeError: Delayed objects of unspecified length are not iterable While the following work around does not. But it looks a lot more awkward

 from dask import delayed @delayed def myFunction(): return 1,2 dummy = myFunction() a, b = dummy[0], dummy[1] a.compute() 

Is this the intended behavior?

Answer 1

Use the nout= keyword as described in the deferred line of documentation

 @delayed(nout=2) def f(...): return a, b x, y = f(1) 

documentation line

 nout : int, optional The number of outputs returned from calling the resulting ''Delayed'' object. If provided, the ''Delayed'' output of the call can be iterated into ''nout'' objects, allowing for unpacking of results. By default iteration over ''Delayed'' objects will error. Note, that ''nout=1'' expects ''obj'', to return a tuple of length 1, and consequently for 'nout=0'', ''obj'' should return an empty tuple. 

Answer 2

So should the following code work? I try, but I have this error: ValueError: Expected iterable of tuples of (name, dtype)

 import dask.dataframe as dd from dask.delayed import delayed import pandas as pd def function(f,f1): data = pd.read_csv(f) data2 = pd.read_csv(f1) # Do some operations on data and data2... return df1,df2,df3,df4 dfs1,dfs2,dfs3,dfs4 = [delayed(function,nout=4)(f,f1) for f in list_path1 for f1 in list_path2] ddf1,ddf2,ddf3,ddf4 = dd.from_delayed(dfs1),dd.from_delayed(dfs2),dd.from_delayed(dfs3),dd.from_delayed(dfs4)