promptfoo alternatives
3 substitutes · 2 in evals · 1 adjacent
The short answer
3 tools on this index are recorded as a substitute for promptfoo. 3 of these run the same way you already deploy promptfoo (self-hosted), and 3 are permissively licensed, so they do not change what you can ship. 1 of them sit in a different layer, which means adjacent rather than interchangeable — check the section before treating it as a swap. There is no overall winner here: which one is right depends on traffic shape, operational budget and how much of promptfoo you have already built around.
By Lattice editorial · Comparison reviewed · method
Better than this page
Real substitutes
in Evaluation & Observability- DeepEvallibrary · Apache-2.0
Open-source pytest-style evaluation suite for LLM outputs.
Use when pytest-style evaluation you can run next to your unit tests.
Skip when You need a platform rather than a library.
- Evidently AIlibrary · Apache-2.0
Evaluation and monitoring for both classical and LLM systems.
Use when One evaluation model for both classical ML and LLM systems.
Skip when You only need LLM-specific tracing.
Adjacent, not substitutes
These appear in the same graph but sit in a different layer. They solve a neighbouring problem, not this one — including them in a shortlist is the most common mistake made when migrating off promptfoo.
This graph is bidirectional and unsponsored. A tool appears here if promptfoo or another entry names it as a substitute, and DeepEval, Evidently AI and garak point at promptfoo as well. How this index is compiled — including where it is wrong.