Avoiding multiple tasks at the same time in Luigi

I use Luigi to create data analysis tasks, including plotting matplotlib.

It seems that parallel runs of the matplotlib graph are causing a problem that, for some reason, leads to a return from the task before doing nothing. (This seems to be a problem with matplotlib, although I may be wrong.)

To solve this problem, I want to avoid the simultaneous launch of several employees only for this task of construction, while simultaneously performing other tasks for several employees. How can i do this?

+4
source share
1 answer

resources. /etc/luigi/client.cfg :

[resources]
mathplotlib: 1

:

class MyTask(luigi.Task):
    resources = {"mathplotlib": 1}

, luigi, , , .

+6

Source: https://habr.com/ru/post/1613778/


All Articles