Restart part of the Airflow subband

I have a daily Dag that contains a subrange. The sub-task has five tasks: from T1 to T5, which must be performed in order (for example, T1 β†’ T2 β†’ T3 β†’ T4 β†’ T5)

Doug has been working successfully for several days, but then I find an error with T4. I am fixing the error and want to restart only T4 and T5 for all previous days. It is important NOT to restart T1-T3, because these steps take much longer than T4-T5.

What I tried failed:

  • Choose T4, Clear downstream + recursive - nothing will happen. The Dag tree shows that the subdag is "success", although T4 and T5 inside it are cleared.
  • Select T4, clear downstream + recursive, select subdag, clear only this task. This will restart the entire subband (T1-T5), although T1-T3 have been marked as successful.
  • Select T4, clear the downstream + recursive, select subdag, press run - Same as # 2. Re-runs the entire subset.
  • Select T4, clear the stream down + recursive, manually set the subject to "running". Nothing has happened. The tree view shows the subject in the β€œstart” state, but no tasks are performed.

This seems to be a problem only when trying to restart part of a subband. If I have a bunch of tasks in a regular dag, usually choosing a task in the middle and choosing clear downstream + recursive, it will start the dag again from this point.

Any suggestions would be appreciated.

+5
source share

Source: https://habr.com/ru/post/1274801/


All Articles