39 pages · 8 layers
What to use instead.
Leaving something is a decision, and it is the one a vendor cannot help with: nobody publishes an honest alternatives page for a competitor. These pages do, for every tool with enough of a substitutes graph to answer the question. Grouped by the layer you are leaving, because that is the constraint you already have.
Layer 01 · Inference & Serving · 5
vLLM alternatives
Paged-attention inference engine with an OpenAI-compatible server.
5 substitutes · 5 in inference · 0 adjacent
Swap for SGLang, TensorRT-LLM, Text Generation Inference, Triton Inference Server, PowerInfer
SGLang alternatives
Structured generation runtime built on RadixAttention prefix reuse.
3 substitutes · 3 in inference · 0 adjacent
Swap for vLLM, TensorRT-LLM, Text Generation Inference
llama.cpp alternatives
Portable CPU/GPU inference for quantized GGUF models.
5 substitutes · 5 in inference · 0 adjacent
Swap for Ollama, LM Studio, llamafile, KoboldCpp, PowerInfer
Ollama alternatives
Local model runner with a single-binary distribution and HTTP API.
4 substitutes · 4 in inference · 0 adjacent
Swap for llama.cpp, LM Studio, llamafile, KoboldCpp
Text Generation Inference alternatives
Hugging Face production server for LLMs with tensor parallelism.
3 substitutes · 3 in inference · 0 adjacent
Swap for vLLM, SGLang, Triton Inference Server
Layer 02 · Routing & Gateways · 4
LiteLLM alternatives
OpenAI-format proxy translating across 100+ model providers.
5 substitutes · 5 in routing · 0 adjacent
Swap for Portkey, OpenRouter, Envoy AI Gateway, Bifrost, Cloudflare AI Gateway
Portkey alternatives
AI gateway with routing, caching and guardrails in one layer.
3 substitutes · 3 in routing · 0 adjacent
Swap for LiteLLM, Cloudflare AI Gateway, TrueFoundry
Cloudflare AI Gateway alternatives
Edge gateway adding caching, retries and rate limiting.
3 substitutes · 3 in routing · 0 adjacent
Swap for LiteLLM, Portkey, Envoy AI Gateway
Martian alternatives
Model router that optimizes for quality, cost and latency.
3 substitutes · 3 in routing · 0 adjacent
Swap for OpenRouter, RouteLLM, Not Diamond
Layer 03 · Retrieval & Vector Stores · 8
pgvector alternatives
Vector similarity search as a Postgres extension.
4 substitutes · 4 in retrieval · 0 adjacent
Swap for Qdrant, Chroma, Milvus, Weaviate
Qdrant alternatives
Rust vector database with rich filtering payloads.
7 substitutes · 7 in retrieval · 0 adjacent
Swap for pgvector, Weaviate, Chroma, Pinecone, Milvus, Turbopuffer, Elasticsearch
Weaviate alternatives
Graph-aware vector database with hybrid and multi-vector search.
5 substitutes · 5 in retrieval · 0 adjacent
Swap for pgvector, Qdrant, Milvus, Elasticsearch, Vespa
Pinecone alternatives
Fully managed vector database with serverless scaling.
3 substitutes · 3 in retrieval · 0 adjacent
Swap for Turbopuffer, Milvus, Qdrant
Milvus alternatives
Cloud-native vector database supporting billion-scale indexes.
4 substitutes · 4 in retrieval · 0 adjacent
Swap for pgvector, Qdrant, Pinecone, Weaviate
Unstructured alternatives
Preprocessing library that partitions raw documents for indexing.
4 substitutes · 4 in retrieval · 0 adjacent
Swap for Rerankers, LangChain Text Splitters, Docling, LlamaParse
Elasticsearch alternatives
BM25 plus dense vectors in one index, with hybrid ranking.
3 substitutes · 3 in retrieval · 0 adjacent
Swap for Weaviate, Vespa, Qdrant
Docling alternatives
Layout-aware parsing that preserves table and heading structure.
3 substitutes · 3 in retrieval · 0 adjacent
Swap for Unstructured, LangChain Text Splitters, LlamaParse
Layer 04 · Fine-tuning & Training · 3
LLaMA-Factory alternatives
Unified interface for SFT, DPO and RLHF on open models.
3 substitutes · 3 in training · 0 adjacent
Swap for Axolotl, TRL, torchtune
TRL alternatives
Hugging Face library of post-training trainers for SFT, DPO and GRPO.
4 substitutes · 4 in training · 0 adjacent
Swap for LLaMA-Factory, PEFT, torchtune, Hugging Face TRL
Replicate alternatives
Hosted fine-tuning and deployment for open models.
3 substitutes · 2 in training · 1 adjacent
Swap for Colab, Weights & Biases Launch
Layer 05 · Agent Frameworks · 5
LangChain alternatives
Composable abstractions for model calls, tools and state.
5 substitutes · 5 in agents · 0 adjacent
Swap for LlamaIndex, Pydantic AI, AutoGen, CrewAI, Semantic Kernel
Pydantic AI alternatives
Type-safe agent framework that leans on Pydantic models.
5 substitutes · 5 in agents · 0 adjacent
Swap for LangChain, LlamaIndex, Semantic Kernel, Agno, smolagents
OpenAI Agents SDK alternatives
Lightweight primitives for handoffs, guardrails and tracing.
5 substitutes · 5 in agents · 0 adjacent
Swap for Mastra, Claude Agent SDK, Letta, Vercel AI SDK, smolagents
Letta alternatives
Agent runtime with persistent, editable memory as a first-class primitive.
3 substitutes · 3 in agents · 0 adjacent
Swap for Claude Agent SDK, smolagents, OpenAI Agents SDK
smolagents alternatives
Minimal code-first agent loop from the Hugging Face team.
4 substitutes · 4 in agents · 0 adjacent
Swap for Pydantic AI, OpenAI Agents SDK, Agno, Letta
Layer 06 · Workflow Orchestration · 2
Temporal alternatives
Durable execution engine that survives crashes and long waits.
4 substitutes · 4 in workflows · 0 adjacent
Swap for Inngest, Trigger.dev, Restate, DBOS
Trigger.dev alternatives
Background jobs and AI workflows with long-running task support.
3 substitutes · 3 in workflows · 0 adjacent
Swap for Inngest, Fly Machines, Temporal
Layer 08 · Prompt Engineering · 4
Humanloop alternatives
Prompt versioning and evaluation for production teams.
4 substitutes · 3 in prompts · 1 adjacent
Swap for PromptLayer, Agenta, Promptwatch
PromptLayer alternatives
Prompt registry with request logging and regression testing.
4 substitutes · 4 in prompts · 0 adjacent
Swap for DSPy, Humanloop, Agenta, Promptwatch
Agenta alternatives
Open-source prompt management with a playground and versioning.
3 substitutes · 3 in prompts · 0 adjacent
Swap for Humanloop, PromptLayer, Promptwatch
Promptwatch alternatives
Prompt regression tests and side-by-side comparison.
3 substitutes · 3 in prompts · 0 adjacent
Swap for PromptLayer, Agenta, Humanloop
Layer 09 · Evaluation & Observability · 8
LangSmith alternatives
Tracing, evaluation and dataset tooling across LangChain runs.
4 substitutes · 4 in evals · 0 adjacent
Swap for Braintrust, Langfuse, Weights & Biases, Arize Phoenix
Braintrust alternatives
Evaluation platform with built-in scorers and a data flywheel.
5 substitutes · 4 in evals · 1 adjacent
Swap for LangSmith, Langfuse, DeepEval, Vellum
Arize Phoenix alternatives
Open-source tracing and evaluation built on OpenTelemetry.
4 substitutes · 4 in evals · 0 adjacent
Swap for LangSmith, Langfuse, Gentrace, Opik
Langfuse alternatives
Self-hostable LLM tracing, prompt management and cost analytics.
6 substitutes · 6 in evals · 0 adjacent
Swap for LangSmith, Braintrust, Arize Phoenix, Helicone, Gentrace, Opik
promptfoo alternatives
Declarative red-teaming and regression tests for prompts and agents.
3 substitutes · 2 in evals · 1 adjacent
Swap for DeepEval, Evidently AI
DeepEval alternatives
Open-source pytest-style evaluation suite for LLM outputs.
3 substitutes · 3 in evals · 0 adjacent
Swap for Braintrust, promptfoo, Evidently AI
Gentrace alternatives
Open-source tracing and dashboards for LLM applications.
4 substitutes · 4 in evals · 0 adjacent
Swap for Arize Phoenix, Langfuse, Helicone, Opik
Opik alternatives
Open-source LLM observability and evaluation from Comet.
3 substitutes · 3 in evals · 0 adjacent
Swap for Arize Phoenix, Langfuse, Gentrace
Every graph here is bidirectional and unsponsored: a tool is listed if the tool being left names it, or if it names the tool being left. Where both sides deserve a paragraph rather than a row, the pair gets a comparison. How this index is compiled including where it is wrong.
Get a second pair of eyes on your AI stack.
Describe what you are building. You get back a written review of every layer: what to keep, what to swap, what it will cost and where it will break.