Regression gate
A check in CI that blocks a merge when a change makes a scored dimension worse.
By Lattice editorial · Entry last reviewed · method
What it implies
The thing that makes evals an asset rather than a dashboard. Without it you can still debug with traces; you just cannot tell whether anything you changed helped. Put it in CI on every commit and let it fail — a failing eval is the system telling you something true.
Tools in this index
Related terms
- Golden setA small, curated set of real cases, with expected behaviour written down, used to tell whether a change helped.
- LLM-as-judgeUsing a model to score output against a written rubric, in place of a human.
- Shadow trafficReplaying a sample of production requests against a candidate model without serving the answers.