Consumption & cost · rule warehouse-scan-hotspot
One dbt model dominating warehouse scans
A Metacenta review checks this under the rule No single model dominates warehouse scan volume. Everything below applies whether or not you ever commission one.
What this rule checks
This rule flags any dbt model whose queries scanned at least 10% of all the bytes your warehouse scanned in the collection window. At three times that share, the finding is high severity.
Why it matters
On a usage-priced warehouse, one relation at that share sets a visible fraction of the bill. It is usually the one nobody has looked at, because it has always worked.
How to fix it
Cut the scan volume of the model dominating the warehouse. Check whether the table is partitioned or clustered on the column its readers filter by. Then check whether the readers need the full history, or only a recent window.
Before:
{{ config(materialized='table') }}
select * from {{ ref('stg_shop__events') }}
After:
{{ config(
materialized='incremental',
partition_by={'field': 'event_date', 'data_type': 'date'},
cluster_by=['customer_id']
) }}
select * from {{ ref('stg_shop__events') }}
When it is fine to leave
The model at the centre of the business is often the most scanned, and rightly. The finding is a prompt to check it is stored for how it is read, not a sign that something is wrong.
What we need to check it
Warehouse query history with scanned bytes. We report the share, not the cost, because the price per byte depends on a contract we cannot see. Some warehouses do not report bytes, and then the rule declines.