Evaluation & Observability

The only way to know whether any of the above works.

• LangSmithplatformTracing, evaluation and dataset tooling across LangChain runs.smith.langchain.comMore about LangSmith in this index
• BraintrustplatformEvaluation platform with built-in scorers and a data flywheel.braintrust.devMore about Braintrust in this index
• Arize PhoenixlibraryOpen-source tracing and evaluation built on OpenTelemetry.phoenix.arize.comMore about Arize Phoenix in this index
• LangfuseplatformSelf-hostable LLM tracing, prompt management and cost analytics.langfuse.comMore about Langfuse in this index
• promptfoolibraryDeclarative red-teaming and regression tests for prompts and agents.promptfoo.devMore about promptfoo in this index
• DeepEvallibraryOpen-source pytest-style evaluation suite for LLM outputs.deepeval.comMore about DeepEval in this index
• HeliconeplatformGateway-level observability with per-request cost and latency.helicone.aiMore about Helicone in this index
• Weights & BiasesplatformExperiment tracking and model registry with LLM eval surfaces.wandb.aiMore about Weights & Biases in this index
• OpenTelemetrylibraryVendor-neutral standard for emitting traces and metrics.opentelemetry.ioMore about OpenTelemetry in this index
• VellumplatformPrompt and eval platform with a visual debugger for chains.vellum.aiMore about Vellum in this index
• GentraceplatformOpen-source tracing and dashboards for LLM applications.gentrace.aiMore about Gentrace in this index
• OpikplatformOpen-source LLM observability and evaluation from Comet.comet.comMore about Opik in this index
• Evidently AIlibraryEvaluation and monitoring for both classical and LLM systems.evidentlyai.comMore about Evidently AI in this index