Metacenta

Modelling correctness · rule incremental-without-unique-key

dbt incremental models with no unique_key

A Metacenta review checks this under the rule Incremental models declare a unique key. Everything below applies whether or not you ever commission one.

What this rule checks

This rule flags incremental models with no unique_key, or an empty one. Models using insert_overwrite, microbatch or append are not judged, because those strategies never take a key. Every other strategy, including the default, is.

Why it matters

A re-run that re-reads loaded rows appends them again, and totals drift upward until a full refresh. A >= watermark on {{ this }} re-reads the last period on every run. A strict > watermark is safe day to day, but a backfill still duplicates and late rows are skipped. We rate that shape medium rather than high.

How to fix it

Stop partial reloads from duplicating rows. Set unique_key in the model config, then full-refresh once to clear the duplicates already loaded. If the model is partitioned by date, set incremental_strategy: insert_overwrite or microbatch instead; they replace whole partitions and need no key.

Before:

{{ config(materialized='incremental') }}

select order_id, status, updated_at
from {{ ref('stg_shop__orders') }}
{% if is_incremental() %}
where updated_at >= (select max(updated_at) from {{ this }})
{% endif %}

After:

{{ config(materialized='incremental', unique_key='order_id') }}

select order_id, status, updated_at
from {{ ref('stg_shop__orders') }}
{% if is_incremental() %}
where updated_at >= (select max(updated_at) from {{ this }})
{% endif %}

When it is fine to leave

A true event log, where every row is new and a repeated row is a real repeat, has no key to merge on. Declare incremental_strategy: append so the choice is on record. This rule then stops flagging it.

What we need to check it

manifest.json, which dbt parse writes. We tell the watermark shapes apart by reading each model's raw_code. A manifest without it puts every flagged model in the high-severity tier.