All geek questions in one place

TemplateNotFound error when starting a simple Airflow BashOperator

I am trying to write our first Airflow DAG and I get the following error when I try to list tasks using the airflow list_tasks orderwarehouse :

 Traceback (most recent call last): File "/usr/local/lib/python2.7/site-packages/airflow/models.py", line 2038, in resolve_template_files setattr(self, attr, env.loader.get_source(env, content)[0]) File "/usr/local/lib/python2.7/site-packages/jinja2/loaders.py", line 187, in get_source raise TemplateNotFound(template) TemplateNotFound: ./home/deploy/airflow-server/task_scripts/orderwarehouse/load_warehouse_tables.sh

This DAG should not use a template. I am only trying to run a shell script at the specified location according to the instructions in the docs . A shell script exists in this place and is spelled correctly. My DAG looks like this:

 from airflow import DAG from airflow.operators.bash_operator import BashOperator default_args = { 'owner': 'airflow', 'depends_on_past': False, 'start_date': datetime(2015, 6, 1), 'email': [' airflow@airflow.com '], 'email_on_failure': False, 'email_on_retry': False, 'retries': 1, 'retry_delay': timedelta(minutes=5), # 'queue': 'bash_queue', # 'pool': 'backfill', # 'priority_weight': 10, # 'end_date': datetime(2016, 1, 1), } orderwarehouse = DAG('orderwarehouse', default_args=default_args) load_mysql = BashOperator( task_id='load_warehouse_mysql', bash_command='./home/deploy/airflow-server/task_scripts/orderwarehouse/load_warehouse_tables.sh', dag=orderwarehouse)

Not sure why he thinks he needs to look for a Jinja template. Launching ideas on this would be appreciated if anyone could tell me where I am going astray. Thanks.

+6
source share
1 answer

This is an airflow trap. Add a space at the end of your bash_command and it should work fine

Source: https://cwiki.apache.org/confluence/display/AIRFLOW/Common+Pitfalls

+10
source

Source: https://habr.com/ru/post/1014858/

More articles:


All Articles