Cost per successful outcome
Total spend divided by the count of requests that passed whatever quality bar you can define.
By Lattice editorial · Entry last reviewed · method
What it implies
The metric that inverts priorities. Work that fails often and retries becomes visibly expensive; long contexts that do not improve accuracy become visibly wasteful; model calls made 'just in case' stop being invisible. Tokens are an input to this, not the number itself.
Related terms
- Golden setA small, curated set of real cases, with expected behaviour written down, used to tell whether a change helped.
- DistillationTraining a smaller model against the outputs of a larger one, to get most of the behaviour at a fraction of the cost.
- Semantic cacheReusing a previous response when a new request is sufficiently similar in meaning, skipping the model call entirely.