All geek questions in one place

How to reset luigi task status?

I currently have many luigi tasks queued up with a simple dependency chain ( a -> b -> c -> d). dperformed first, and a- at the end. a- This is a task that starts.

All targets except areturn an object luigi.LocalTarget()and have one in common luigi.Parameter(), which is a string (containing the date and time). Runs on a central luigi server (with history enabled).

The problem is that when you restart the specified task, aluigi checks the history and sees if this particular task was started earlier, if it had the DONE status, it does not start the task ( din this case), and I can’t do it, change the line does not help (a random microsecond is added to it). How to make a task run?

+8
source share
2 answers

: . , , , . . : , , , -, . , , , .

, , , . , , . , Target. , .

, , Target. , , .

Luigi . , .

+9

d6tflow , . https://d6tflow.readthedocs.io/en/latest/control.html#manually-forcing-task-reset-and-rerun.

: d6tflow, , luigi. . , .

# force execution including downstream tasks
d6tflow.run([TaskTrain()],force=[TaskGetData()])

# reset single task
TaskGetData().invalidate()

# reset all downstream task output
d6tflow.invalidate_downstream(TaskGetData(), TaskTrain())

# reset all upstream task input
d6tflow.invalidate_upstream(TaskTrain())
0

Source: https://habr.com/ru/post/1622917/

More articles:


All Articles