OpenTelemetry
opentelemetry.iolibraryself-hosted
OpenTelemetry is a self-hosted library in the Evaluation & Observability section of Lattice. Vendor-neutral standard for emitting traces and metrics.
By Lattice editorial · Facts verified · method
- licence
- Apache-2.0
- language
- multi
- cost
- free
- owned by
- Production & Governance
Verified 2026-09build refuses it from 2027-04
Use OpenTelemetry when
A vendor-neutral standard for emitting traces and metrics.
Skip OpenTelemetry when
You want a product with a user interface.
The only way to know whether any of the above works. — that is what this layer of the stack is answerable for. See the full Evaluation & Observability section for the rest of the options.
Visit opentelemetry.ioCompare with another toolBadge this entry
[](https://lattice.kunjshah.in/evaluation-observability/opentelemetry)Quick answers
- Is OpenTelemetry open source?
- Yes. OpenTelemetry is open source under the Apache-2.0 licence. It is free to use.
- What are the alternatives to OpenTelemetry?
- Other options in Evaluation & Observability include LangSmith, Braintrust, Arize Phoenix, Langfuse and promptfoo.
Reading on evals
- Fix the ranking, not the promptWhen a RAG system gives a bad answer, the instinct is to rewrite the prompt. Nine times out of ten the prompt is fine and the context was never there.
- Evals are the asset, not the loggingTraces tell you what happened. Evals tell you whether it got better. Most teams build the first and mistake it for the second.
- Build a cost model before you optimiseTeams optimise the thing they can see in the dashboard. The dashboard shows tokens. The number that matters is cost per successful outcome, and nobody is measuring it.
- Observability is not loggingA log line with a status code is not a trace. The difference is whether it can reconstruct a decision after the model that made it has been replaced twice.
Also in evals
- LangSmithLangSmithTracing, evaluation and dataset tooling across LangChain runs.Use when Teams already deep in LangChain needing tracing and datasets.Skip when Traces cannot leave your infrastructure.proprietary · saas · usage-based
- BraintrustBraintrustEvaluation platform with built-in scorers and a data flywheel.Use when Making evaluation, rather than tracing, the primary workflow.Skip when You need open formats and self-hosting.proprietary · saas · usage-based
- Arize PhoenixArize PhoenixOpen-source tracing and evaluation built on OpenTelemetry.Use when OpenTelemetry-native evaluation you can run yourself.Skip when You want a vendor support contract behind it.Apache-2.0 · self-hosted · free
- LangfuseLangfuseSelf-hostable LLM tracing, prompt management and cost analytics.Use when Self-hostable tracing, prompts and evals in one place.Skip when You want a fully managed product with a support contract.MIT · self-hosted · free-tier
- promptfoopromptfooDeclarative red-teaming and regression tests for prompts and agents.Use when Declarative red-teaming and regression tests that run in CI.Skip when You need a managed UI rather than a test runner.MIT · self-hosted · free
- DeepEvalDeepEvalOpen-source pytest-style evaluation suite for LLM outputs.Use when pytest-style evaluation you can run next to your unit tests.Skip when You need a platform rather than a library.Apache-2.0 · self-hosted · free
- HeliconeHeliconeGateway-level observability with per-request cost and latency.Use when Gateway-level observability with per-request cost and latency.Skip when You need eval primitives more than traffic logs.proprietary · saas · free-tier
- Weights & BiasesWeights & BiasesExperiment tracking and model registry with LLM eval surfaces.Use when Experiment tracking and a model registry, with eval surfaces on top.Skip when You need an open, exportable format.proprietary · saas · usage-based
- VellumVellumPrompt and eval platform with a visual debugger for chains.Use when Prompt and eval work with a visual debugger for chains.Skip when Self-hosting or reproducibility is a requirement.proprietary · saas · usage-based
- GentraceGentraceOpen-source tracing and dashboards for LLM applications.Use when Open-source tracing and dashboards for LLM applications.Skip when You want eval depth more than traffic visibility.Apache-2.0 · self-hosted · free-tier
- OpikOpikOpen-source LLM observability and evaluation from Comet.Use when Open-source observability and evaluation you run yourself.Skip when You need enterprise support behind it.Apache-2.0 · self-hosted · free
- Evidently AIEvidently AIEvaluation and monitoring for both classical and LLM systems.Use when One evaluation model for both classical ML and LLM systems.Skip when You only need LLM-specific tracing.Apache-2.0 · self-hosted · free