Odd TypeError from the airflow scheduler - is using @once used for the scheduler interval changed in version 1.9?

Question

I have a super simple test DAG that looks like this:

from datetime import datetime

from airflow.models import DAG
from airflow.operators.python_operator import PythonOperator


DAG = DAG(
  dag_id='scheduler_test_dag',
  start_date=datetime(2017, 9, 9, 4, 0, 0, 0), #..EC2 time. Equal to 11pm hora México
  max_active_runs=1,
  schedule_interval='@once' #externally triggered
  )

def ticker_function():
    with open('/tmp/ticker', 'a') as outfile:
        outfile.write('{}\n'.format(datetime.now()))

time_ticker = PythonOperator(
    task_id='time_ticker',
    python_callable=ticker_function,
    dag=DAG
)

Since upgrading to apache-airflowv1.9, this DAG freezes and does not start. Digging into the scheduler logs, I found an error trace:

[2018-02-12 17:03:06,259] {jobs.py:1754} INFO - DAG(s) dict_keys(['scheduler_test_dag']) retrieved from /home/ubuntu/airflow/dags/scheduler_test_dag.py
[2018-02-12 17:03:06,315] {jobs.py:1386} INFO - Processing scheduler_test_dag
[2018-02-12 17:03:06,320] {jobs.py:379} ERROR - Got an exception! Propagating...
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/airflow/jobs.py", line 371, in helper
pickle_dags)
  File "/usr/local/lib/python3.5/dist-packages/airflow/utils/db.py", line 50, in wrapper
result = func(*args, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/airflow/jobs.py", line 1792, in process_file
self._process_dags(dagbag, dags, ti_keys_to_schedule)
  File "/usr/local/lib/python3.5/dist-packages/airflow/jobs.py", line 1388, in _process_dags
dag_run = self.create_dag_run(dag)
  File "/usr/local/lib/python3.5/dist-packages/airflow/utils/db.py", line 50, in wrapper
result = func(*args, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/airflow/jobs.py", line 807, in create_dag_run
if next_start <= now:
TypeError: unorderable types: NoneType() <= datetime.datetime()

Where does this error come from? The only thing I can think of is that the usage scheduler_interval='@once'has changed, which is the only thing that this DAG has with another broken DAG on my server since version v1.9 was updated. Otherwise, this is the most basic DAG, there should never seem to be a problem. I used to use a basic pip installation before switching to apache-airflowrepo.

-. , , DAGS , @once :

?

Answer 1

, True airflow.cfg, ? master. catchup dag, .

Answer 2

, "catchup = false" dag.

Answer 3

: https://issues.apache.org/jira/browse/AIRFLOW-1977

, , . ,

schedule_interval=None

And manually run it once, it will work as expected. After that, you can put @oncein quality schedule_intervaland the scheduler will no longer complain (of course, if you manually run the task, it defeats the launch target automatically once ...). I have not tested whether the same thing happens with other tags or not, but this can fix the same problem, for example with a tag @daily.