dbt source freshness: how to set it up

Add freshness and loaded_at_field under config: on the source. Every table inherits them, and a table can override them with its own values.

sources:
  - name: shop
    config:
      loaded_at_field: _loaded_at
      freshness:
        warn_after: {count: 6, period: hour}
        error_after: {count: 24, period: hour}
    tables:
      - name: orders
      - name: customers
        config:
          freshness:
            error_after: {count: 48, period: hour}
      - name: countries
        config:
          freshness: null

freshness: null excludes a table from a threshold it would otherwise inherit. freshness moved under config: in dbt 1.9, and loaded_at_field moved in 1.10. Older projects set both directly on the source.

Each threshold needs a count, which is a positive integer, and a period of minute, hour or day. You can set one threshold or both. Match them to how often the table loads: a table loaded every six hours needs a longer threshold than one loaded every 15 minutes.

On a large partitioned table, add filter to limit the freshness query. It's a SQL condition that applies only to this check.

loaded_at_field is the column or expression dbt uses to find the latest load time. Choose when the row reached the warehouse, not when the business event happened. Otherwise a quiet weekend can look like a broken feed.

Without loaded_at_field, dbt reads warehouse metadata where the adapter supports it. From dbt 1.10, loaded_at_query takes a custom SQL expression instead. You can't set both on one table. We cover missing timestamps in dbt freshness checks with no loaded_at_field.

Thresholds do nothing until a job runs the command. Schedule it at least as often as your shortest warn_after.

# every source with a freshness config
dbt source freshness

# one source, or one table
dbt source freshness --select "source:shop"
dbt source freshness --select "source:shop.orders"

A stale table reports a warning or an error, and results are written to target/sources.json. On dbt v2, dbt freshness replaces this command and also checks models. See dbt's freshness docs for every property.

This rule flags every dbt source table with no freshness threshold: no warn_after or error_after. A threshold set on the source group and inherited by the table counts.

If the upstream feed stops, every downstream number goes stale with no error. The dashboards keep rendering yesterday's data as today's, and nothing fails.

Add freshness monitoring to the ingested sources. Set freshness: with warn_after and error_after on the source, and run dbt source freshness on a schedule so the threshold is actually checked.

Before:

sources:
  - name: shop
    tables:
      - name: orders

After:

sources:
  - name: shop
    config:
      loaded_at_field: _loaded_at
      freshness:
        warn_after: {count: 6, period: hour}
        error_after: {count: 24, period: hour}
    tables:
      - name: orders

A static reference table that is loaded once, such as a list of countries, has nothing to be fresh about. The rule still flags it, so write the reason in the table's description and the gap reads as a decision.

manifest.json alone, which dbt parse writes.

dbt_project_evaluator, rule fct_sources_without_freshness. Ours checks the same condition and threshold.

This means our check corresponds to their rule. It does not mean the publisher reviewed or endorses it.