Routing & Gateways

Decides which model answers, and what it costs.

LiteLLMlibraryOpenAI-format proxy translating across 100+ model providers.litellm.aiMore about LiteLLM in this index
PortkeyplatformAI gateway with routing, caching and guardrails in one layer.portkey.aiMore about Portkey in this index
Cloudflare AI GatewayserviceEdge gateway adding caching, retries and rate limiting.developers.cloudflare.comMore about Cloudflare AI Gateway in this index
OpenRouterserviceSingle API key and billing across many hosted models.openrouter.aiMore about OpenRouter in this index
MartianplatformModel router that optimizes for quality, cost and latency.withmartian.comMore about Martian in this index
Envoy AI GatewayruntimeCNCF-track gateway for LLM traffic on an Envoy data plane.aigateway.envoyproxy.ioMore about Envoy AI Gateway in this index
BifrostlibraryHigh-throughput LLM gateway with drop-in OpenAI compatibility.getmaxim.aiMore about Bifrost in this index
RouteLLMlibraryLearned router that cuts cost by matching difficulty to model size.github.comMore about RouteLLM in this index
Not DiamondplatformRouting and prompt optimisation tuned on your own traffic.notdiamond.aiMore about Not Diamond in this index
TrueFoundryplatformGateway plus observability and guardrails as one deployment.truefoundry.comMore about TrueFoundry in this index

By Lattice editorial · Layer boundaries and entries verified 2026-09 · method

Also relevant here

These tools are indexed in another layer, but they also do routing & gateways's job. Each carries its home layer's colour.