Airflow DAGs with no failure alert

This rule flags each job file where nothing would tell a person about a failure. A failure callback, email_on_failure=True, a Dagster run-failure sensor or a Slack, PagerDuty or Opsgenie operator all pass. A callback set to None does not.

This failure costs the most and announces itself the least. The job stops, the tables stop moving, and every dashboard downstream shows yesterday's numbers as today's. Somebody finds out days later, when a figure looks wrong in a meeting.

Make every job failure reach a person. Wire on_failure_callback to wherever the team is already paged. Set it in default_args, so every task in the DAG inherits it.

Before:

default_args = {
    "owner": "finance-data",
    "retries": 3,
}

After:

default_args = {
    "owner": "finance-data",
    "retries": 3,
    "on_failure_callback": notify_on_call_rota,
}

Alerting wired globally, through a cluster policy, airflow_local_settings.py or a monitor on scheduler metrics, covers every DAG without appearing in any of them. When no job we review alerts, each finding names that as the likely explanation.

The job files you name to us one at a time. When email_on_failure is set by an expression we do not evaluate, we report that we could not tell, never that nobody is told.