AI lifecycle · rule ai-llm-without-observability
LLM calls with no tracing or cost tracking
A Metacenta review checks this under the rule Model calls are traced and costed. Everything below applies whether or not you ever commission one.
What this rule checks
Where a project declares a language-model SDK, this rule checks that a tracing or cost-observability library such as Langfuse or LangSmith is declared too. It reports once for the project.
Why it matters
Inference is the one dependency that bills per use and degrades without an error. Untraced, a prompt that grew twice as long shows up in next month's bill rather than on the day it shipped.
How to fix it
Trace and cost every model call. Log tokens, latency and model id for every call, and keep the totals. Those figures are what make the next conversation about cost short.
When it is fine to leave
Spend and latency tracked elsewhere, such as through a gateway or the provider's own usage dashboard broken down by feature, cover this in practice. We report it at medium confidence for that reason.
What we need to check it
The dependency manifests in your repository. We reason from what they declare, so the finding is medium confidence at most.