Can I set a different artist for each DAG Airflow?

I want to add another DAG to an existing Airflow server. The server currently uses the LocalExecutor, but I may need my DAG to use CeleryExecutor. It seems that the configuration file airflow.cfgallows only one executor:

# The executor class that airflow should use. Choices include
# SequentialExecutor, LocalExecutor, CeleryExecutor
executor = LocalExecutor

Is it possible to configure Airflow so that existing DAGs can continue to use LocalExecutor, and my new DAG can use CeleryExecutor or its own artist class? I did not find examples of people who did this, and did not come across anything in the Airflow documentation.

+4
source share
2 answers

It seems that the scheduler will only start one instance of the executor.

+1

SubDAG , SubDagOperator. , SequentialExecutor:

bar_subdag = SubDagOperator(
    task_id='bar',
    subdag=my_subdag('foo', 'bar', default_args),
    default_args=default_args,
    dag=foo_dag,
    executor=SequentialExecutor()
)

1.8, , 1.9 .

0

Source: https://habr.com/ru/post/1650064/


All Articles