LiteLLM
litellm.ailibraryself-hosted
LiteLLM is a self-hosted library in the Routing & Gateways layer of the AI stack. OpenAI-format proxy translating across 100+ model providers.
By Lattice editorial · Facts verified · method
- licence
- MIT
- language
- Python
- cost
- free
- owned by
- ML Platform
Verified 2026-09build refuses it from 2027-04
Use LiteLLM when
The widest provider coverage, deployed the least committal way.
Skip LiteLLM when
You want routing policy to live in a SaaS console.
Decides which model answers, and what it costs. — that is what this layer of the stack is answerable for. See the full Routing & Gateways section for the rest of the options.
Visit litellm.aiCompare with another toolBadge this entry
[](https://lattice.kunjshah.in/routing-gateways/litellm)Quick answers
- Is LiteLLM open source?
- Yes. LiteLLM is open source under the MIT licence. It is free to use.
- What are the alternatives to LiteLLM?
- The closest alternatives to LiteLLM are Portkey, Cloudflare AI Gateway and Envoy AI Gateway.
Compared in
- LiteLLM vs Portkey vs Cloudflare AI GatewayLLM gateways compared on provider coverage, caching, where the data goes and whether the routing logic is yours or theirs.
- Gateway vs guardrails vs evals: what to build firstA gateway, a guardrail layer and an eval suite compete for the same first week of work. What each catches, what each misses, and the order to adopt them in.
Reading on routing
- The gateway is the productThe interesting engineering is not calling a model. It is deciding which model, failing over safely, and staying inside a budget while you do it.
- Build a cost model before you optimiseTeams optimise the thing they can see in the dashboard. The dashboard shows tokens. The number that matters is cost per successful outcome, and nobody is measuring it.
- Choosing a model is a routing problemThere is no best model. There is a best model for a task, at a quality bar, within a latency and cost budget — and that decision is a routing policy, not a benchmark.
Alternatives to LiteLLM
compare 3 →Listed as an alternative to
- PortkeyPortkeyAI gateway with routing, caching and guardrails in one layer.Use when A managed gateway with guardrails and analytics already bundled.Skip when Prompts cannot leave your infrastructure.proprietary · saas · usage-based
- OpenRouterOpenRouterSingle API key and billing across many hosted models.Use when One key and one bill across many hosted models, while experimenting.Skip when You need self-hosting or per-provider SLAs.proprietary · saas · usage-based
- Envoy AI GatewayEnvoy AI GatewayCNCF-track gateway for LLM traffic on an Envoy data plane.Use when A Kubernetes shop that already runs Envoy and needs the gateway in-cluster.Skip when You want a gateway configurable without a data plane.Apache-2.0 · self-hosted · free
- BifrostBifrostHigh-throughput LLM gateway with drop-in OpenAI compatibility.Use when A high-throughput OpenAI-compatible proxy with a small footprint.Skip when You need the breadth of a full routing stack.proprietary · self-hosted · free
Also in routing
- PortkeyPortkeyAI gateway with routing, caching and guardrails in one layer.Use when A managed gateway with guardrails and analytics already bundled.Skip when Prompts cannot leave your infrastructure.proprietary · saas · usage-based
- Cloudflare AI GatewayCloudflare AI GatewayEdge gateway adding caching, retries and rate limiting.Use when Edge caching and latency dominate, on Cloudflare already.Skip when Requests must not transit a third party's network.proprietary · saas · free-tier
- OpenRouterOpenRouterSingle API key and billing across many hosted models.Use when One key and one bill across many hosted models, while experimenting.Skip when You need self-hosting or per-provider SLAs.proprietary · saas · usage-based
- MartianMartianModel router that optimizes for quality, cost and latency.Use when Routing tuned on your own traffic rather than a public benchmark.Skip when You want the routing rule to be plain config you can read.proprietary · saas · usage-based
- Envoy AI GatewayEnvoy AI GatewayCNCF-track gateway for LLM traffic on an Envoy data plane.Use when A Kubernetes shop that already runs Envoy and needs the gateway in-cluster.Skip when You want a gateway configurable without a data plane.Apache-2.0 · self-hosted · free
- BifrostBifrostHigh-throughput LLM gateway with drop-in OpenAI compatibility.Use when A high-throughput OpenAI-compatible proxy with a small footprint.Skip when You need the breadth of a full routing stack.proprietary · self-hosted · free
- RouteLLMRouteLLMLearned router that cuts cost by matching difficulty to model size.Use when Cutting cost by matching query difficulty to model size, with a learned router.Skip when You cannot evaluate the quality you would be trading away.Apache-2.0 · self-hosted · free
- Not DiamondNot DiamondRouting and prompt optimisation tuned on your own traffic.Use when Routing and prompt optimisation tuned on your own traffic.Skip when Self-hosting or auditability is a requirement.proprietary · saas · usage-based
- TrueFoundryTrueFoundryGateway plus observability and guardrails as one deployment.Use when Gateway, observability and guardrails as one managed deployment.Skip when You want to adopt the pieces independently.proprietary · managed · subscription