Metacenta

Execution health · rule run-slow-full-rebuild

dbt table models dominating build time

A Metacenta review checks this under the rule No full-rebuild model dominates the build. Everything below applies whether or not you ever commission one.

What this rule checks

From the timings in your last run, this rule flags models built as a table that took at least 10% of the measured build time. At 25% or more, the finding is high severity. It needs at least ten timed models, because below that every share is large by arithmetic.

Why it matters

Build time is the floor on how quickly a change can be made, tested and shipped. One model sets most of it, and the team waits for it on every run, whether or not its inputs moved.

How to fix it

Stop rebuilding the largest tables from scratch on every run. Confirm the timing over a few runs, then switch the model to an incremental materialisation with a unique_key, so a build recomputes only what changed.

Before:

{{ config(materialized='table') }}
select * from {{ ref('stg_shop__events') }}

After:

{{ config(materialized='incremental', unique_key='event_id') }}
select * from {{ ref('stg_shop__events') }}
{% if is_incremental() %}
where loaded_at > (select max(loaded_at) from {{ this }})
{% endif %}

When it is fine to leave

A small or fast-changing table, where most rows change every run, gains little from being incremental and adds complexity. So does a table rebuilt in full on purpose to correct late-arriving data.

What we need to check it

run_results.json alongside manifest.json. One run is one sample, so we report this at medium confidence. The rule declines when the run covered only part of the project.