Orchestration & reliability · rule dag-catchup-unbounded
Airflow catchup: what it does and how to control it
Airflow catchup decides whether the scheduler creates runs for past intervals. With catchup=True it creates a run for each interval that hasn't run. With catchup=False it creates a run for the latest interval only.
A Metacenta review checks this under the rule A deploy does not trigger a backfill. Everything below applies whether or not you ever commission one.
How catchup works
Take a daily DAG with a start_date of 1 January, first switched on at the end of the month. With catchup on, the scheduler creates a run for each completed day. Today isn't included, because its interval hasn't finished.
Catchup also applies when you pause a DAG and unpause it later. Every interval missed during the pause gets a run.
The default in Airflow 2 and Airflow 3
A DAG that doesn't set catchup takes the catchup_by_default setting from the [scheduler] config. It's True in Airflow 2 and False in Airflow 3, so an unset DAG changes behaviour when you upgrade. Set catchup on each DAG to avoid depending on either default.
How to load history with catchup off
A backfill runs a date range you choose, so catchup=False doesn't stop you loading history. The command differs by version.
# Airflow 3
airflow backfill create --dag-id daily_orders \
--from-date 2026-01-01 \
--to-date 2026-01-31 \
--max-active-runs 3
# Airflow 2
airflow dags backfill --start-date 2026-01-01 --end-date 2026-01-31 daily_orders
See Airflow's catchup and backfill docs for the other options.
How max_active_runs limits catchup
max_active_runs caps how many runs of a DAG execute at once. It doesn't reduce how many get created, so a long catchup still works through every interval, just more slowly.
What this rule checks
This rule flags scheduled Airflow DAGs that set catchup=True, or leave it unset before Airflow 3. catchup=False passes, and so does a DAG with schedule=None or no schedule we can read. Dagster and Prefect have no such default, so they are not judged.
Why it matters
Catchup replays every interval between the start date and now. Pause a DAG for a fortnight, or deploy one with an old start date, and resuming it queues hundreds of runs at once. The warehouse bill and the load spike arrive together.
How to fix it
Stop a redeploy from replaying the schedule. Set catchup=False unless the backfill is wanted. Where it is wanted, bound it with max_active_runs.
Before:
with DAG("daily_orders", schedule="@daily", start_date=datetime(2024, 1, 1)):
...
After:
with DAG(
"daily_orders",
schedule="@daily",
start_date=datetime(2024, 1, 1),
catchup=False,
):
...
When it is fine to leave
A DAG meant to backfill, such as one loading partitions by date, is working as intended. An unset catchup is on in Airflow 2 and off in Airflow 3, so on Airflow 3 it passes. A deployment that changes the default in its own config is not visible to us; tell us, and the finding comes out.
What we need to check it
The Airflow DAG files you name to us one at a time. We read your Airflow version from requirements.txt or pyproject.toml if you send them, or from an airflow.sdk import. catchup=True written out is high confidence; unset on Airflow 2 is medium, and low when we cannot tell the version.