Luigi -% s failed at runtime

Question

I am trying to learn very simply how luigi works. Just like a newbie, I came up with this code

import luigi class class1(luigi.Task): def requires(self): return class2() def output(self): return luigi.LocalTarget('class1.txt') def run(self): print 'IN class A' class class2(luigi.Task): def requires(self): return [] def output(self): return luigi.LocalTarget('class2.txt') if __name__ == '__main__': luigi.run() 

Running this on the command line gives an error saying

 raise RuntimeError('Unfulfilled %s at run time: %s' % (deps, ', 

Answer 1

This is because you define the output for class2 , but you never create it.

Let me break it ...

At startup

 python file.py class2 --local-scheduler 

luigi will ask:

• Is class2 output already on disk? NO
• check class2 : NONE dependencies
• execute the run method (by default it is also an empty pass method)
• the start method did not return errors, so the operation succeeds.

However, at startup

 python file.py class1 --local-scheduler 

luigi will be:

• Is the output of class1 already on disk? NO
• check task dependencies: YES: class2
• pause class2 status check
  • is class2 output to disk? NO
  • execute class2 start -> no errors
  • is class2 output to disk? NO → raise error

luigi never starts a task if all its previous dependencies are not executed. (i.e. their output is in the file system)