Across layers · read in stack order
Gateway vs guardrails vs evals: what to build first
These three never appear on the same vendor comparison page, because no vendor sells all three. They do appear on the same backlog. Each catches a different failure, sits in a different place in the request, and costs something different to adopt — so the real question is order, not choice.
Short answer
For most teams, evals first. A gateway makes calls cheaper and more reliable, and guardrails make individual responses safer, but neither tells you whether the system is getting better — and every later change to routing, prompts or retrieval needs that answer to be judged at all.
By Lattice editorial · Licence and cost facts verified · method
- LiteLLMLayer 02 · Band I ComputeDecides which model serves a request.
- Guardrails AILayer 07 · Band III ControlDecides whether a response may leave.
- promptfooLayer 09 · Band III ControlDecides whether a change made things better.
| Dimension | LiteLLM | Guardrails AI | promptfoo |
|---|---|---|---|
| Question it answers | Which model, provider and budget serves this request? | Is this response allowed to leave the system? | Did the last change make answers better or worse? |
| Failure it catches | Provider outages, rate limits, runaway spend | Schema violations and policy breaches, per request | Quality regressions, before they ship |
| Failure it misses | Whether any answer was correct | Answers that are well-formed and wrong | Anything happening in production right now |
| Where it runs | In the request path, on every call | In the request path, after generation | Offline, in CI or on a schedule |
| Cost to adopt | A proxy and a config file; one more network hop | A validator per rule; added latency on every response | A labelled dataset — the examples are the expensive part, not the tool |
| Adopt first when | You call more than one provider, or spend is already a line item | A malformed or unsafe output has a real cost on day one | You are about to change prompts, models or retrieval |
Scroll the table sideways to see every tool.
What this table is not. These are editorial judgements, not benchmarks: Lattice has not run these tools head to head, and no cell uses GitHub stars or vendor benchmark claims as evidence. Licence and cost facts are re-checked on a schedule the build enforces — the receipt is public.
The order
Which should you adopt first: LiteLLM, Guardrails AI or promptfoo?
For most teams, evals first. A gateway makes calls cheaper and more reliable, and guardrails make individual responses safer, but neither tells you whether the system is getting better — and every later change to routing, prompts or retrieval needs that answer to be judged at all. Put the gateway in early if you already run more than one provider, because retrofitting a proxy into every call site is tedious. Add guardrails for the specific failures your evals surface, not the ones you imagine.
Rules of thumb
- A guardrail without an eval is a guess about which failures matter. Write the eval that found the failure, then the guardrail that blocks it.
- Route on measured quality, not on price alone. A cheaper model that fails your eval set is not cheaper.
- Anything in the request path adds latency to every call; anything offline adds none. Stay offline until production forces otherwise.
Quick answers
- When should you use LiteLLM, and when should you skip it?
- Use LiteLLM when: the widest provider coverage, deployed the least committal way. Skip it when: you want routing policy to live in a SaaS console.
- When should you use Guardrails AI, and when should you skip it?
- Use Guardrails AI when: validators that check output against a schema you define. Skip it when: you need free-form moderation rather than structural checks.
- When should you use promptfoo, and when should you skip it?
- Use promptfoo when: declarative red-teaming and regression tests that run in CI. Skip it when: you need a managed UI rather than a test runner.