Airflow DAGs with no retries

This rule flags each job file that declares no retry count, or sets it to zero. A count above zero passes, as a default_args key, a retries= keyword, max_retries= or a Dagster RetryPolicy. We rate a job with ten or more tasks higher.

Warehouses drop connections, credentials refresh and APIs rate-limit. None of that means the pipeline is wrong. Without a retry, each one becomes a failed run that somebody reruns by hand the next morning, and the data is stale until then.

Retry transient failures in scheduled jobs. Set retries and retry_delay in default_args. Two or three retries with exponential backoff is the usual answer. The point is a decision rather than an omission.

Before:

default_args = {
    "owner": "finance-data",
}

After:

default_args = {
    "owner": "finance-data",
    "retries": 3,
    "retry_delay": timedelta(minutes=5),
    "retry_exponential_backoff": True,
}

A task that is not safe to run twice should not retry. An append without a unique key, or a call that sends email or charges a card, belongs here. Make those idempotent where you can, and set retries=0 on that task alone.

The job files you name to us one at a time. We read the count only when it is a number; a value imported from a shared module reads as undeclared. A file that only wires jobs declared elsewhere is reported as not examined, not as missing retries.