---
title: "LLM apps with no evaluation suite | Metacenta"
description: "A prompt change is a code change no ordinary test can see. How we spot projects that call a language model with no evaluation framework, and where to start."
url: "https://metacenta.com/rules/ai-llm-without-evaluation"
updated: "2026-10-14"
tokens: 491
---

AI lifecycle · rule ai-llm-without-evaluation

# LLM apps with no evaluation suite

A Metacenta review checks this under the rule **Model output is evaluated before it's trusted**. Everything below applies whether or not you ever commission one.

## What this rule checks

Where a project declares a language-model SDK, this rule checks that an evaluation framework such as promptfoo, Ragas or DeepEval is declared too. It reports once for the project.

## Why it matters

A prompt change is a code change whose effect is invisible to every test you have. Without an evaluation suite, the first sign that answers got worse is a customer noticing.

## How to fix it

**Evaluate model output before trusting it.** Start with a few dozen recorded inputs, each with the properties a good answer must have, and run them on every change. Having a baseline matters more than which framework runs it.

## When it is fine to leave

An evaluation suite in a separate repository, or a harness written in-house, does the job without appearing in this manifest. We report this at medium confidence for that reason.

## What we need to check it

The dependency manifests in your repository. We reason from what they declare, so the finding is medium confidence at most.

[Ask about a review](https://metacenta.com/enquire?src=rule-ai-llm-without-evaluation&via=md)

## Related pages

- [Unpinned AI and ML library versions](https://metacenta.com/rules/ai-dependency-unpinned)
- [Model training with no experiment tracking](https://metacenta.com/rules/ai-training-without-tracking)
- [LLM calls with no tracing or cost tracking](https://metacenta.com/rules/ai-llm-without-observability)
- [Jupyter notebooks committed with outputs](https://metacenta.com/rules/ai-notebook-outputs-committed)
- [A sample review, as a client reads it](https://metacenta.com/sample)
- [All published rules](https://metacenta.com/rules)
- [How a review is assessed](https://metacenta.com/standard)