{"name":"Lattice — AI infrastructure index","description":"A curated index of AI infrastructure tools, ordered by stack layer — every tool with when to use it and when to skip it.","homepage":"https://lattice.kunjshah.in","verified":"2026-09","staleAfterMonths":6,"verification":"https://lattice.kunjshah.in/verification.json","citation":"Cite as \"Lattice\" and link the tool's canonical url field.","roles":[{"id":"platform","title":"ML Platform","owns":"The paved road other engineers build on.","url":"https://lattice.kunjshah.in/roles/platform"},{"id":"serving","title":"AI Infrastructure","owns":"Tokens per second, per dollar, per GPU.","url":"https://lattice.kunjshah.in/roles/serving"},{"id":"data","title":"Data & Retrieval","owns":"What the model knows, and whether it is the right thing.","url":"https://lattice.kunjshah.in/roles/data"},{"id":"applied","title":"Applied Engineering","owns":"The feature, end to end, in the product.","url":"https://lattice.kunjshah.in/roles/applied"},{"id":"production","title":"Production & Governance","owns":"Whether it can be trusted, and whether it can be let go.","url":"https://lattice.kunjshah.in/roles/production"}],"sections":[{"slug":"inference-serving","title":"Inference & Serving","layer":1,"role":"layer","band":"compute","responsibility":"Turns weights into throughput.","url":"https://lattice.kunjshah.in/inference-serving"},{"slug":"routing-gateways","title":"Routing & Gateways","layer":2,"role":"layer","band":"compute","responsibility":"Decides which model answers, and what it costs.","url":"https://lattice.kunjshah.in/routing-gateways"},{"slug":"retrieval-vector-stores","title":"Retrieval & Vector Stores","layer":3,"role":"layer","band":"state","responsibility":"Holds the embeddings, returns the few that matter.","url":"https://lattice.kunjshah.in/retrieval-vector-stores"},{"slug":"fine-tuning","title":"Fine-tuning & Training","layer":4,"role":"layer","band":"state","responsibility":"Produces the weights that layer 1 serves.","url":"https://lattice.kunjshah.in/fine-tuning"},{"slug":"agent-frameworks","title":"Agent Frameworks","layer":5,"role":"layer","band":"control","responsibility":"Turns a model call into a multi-step program.","url":"https://lattice.kunjshah.in/agent-frameworks"},{"slug":"workflow-orchestration","title":"Workflow Orchestration","layer":6,"role":"layer","band":"control","responsibility":"Survives the retries, the waits and the crashes.","url":"https://lattice.kunjshah.in/workflow-orchestration"},{"slug":"guardrails-safety","title":"Guardrails & Safety","layer":7,"role":"layer","band":"control","responsibility":"Stops the bad input before it, and the bad output after.","url":"https://lattice.kunjshah.in/guardrails-safety"},{"slug":"prompt-engineering","title":"Prompt Engineering","layer":8,"role":"layer","band":"control","responsibility":"Makes the prompt an artifact you can diff and test.","url":"https://lattice.kunjshah.in/prompt-engineering"},{"slug":"evaluation-observability","title":"Evaluation & Observability","layer":9,"role":"crosscutting","band":"control","responsibility":"The only way to know whether any of the above works.","url":"https://lattice.kunjshah.in/evaluation-observability"},{"slug":"learning-reference","title":"Learning & Reference","layer":null,"role":"offstack","band":null,"responsibility":"Not a layer. Reading, once you know what to look for.","url":"https://lattice.kunjshah.in/learning-reference"}],"tools":[{"name":"vLLM","url":"https://lattice.kunjshah.in/inference-serving/vllm","homepage":"https://vllm.ai","section":"inference-serving","layer":1,"band":"compute","kind":"runtime","roles":["serving"],"deployment":"self-hosted","license":"Apache-2.0","language":"Python","cost":"free","useWhen":"General GPU serving under bursty, high-concurrency traffic.","skipWhen":"Traffic is steady and low-concurrency, so batches never fill.","alternatives":["SGLang","Text Generation Inference","Triton Inference Server"],"alsoIn":[],"verified":"2026-09"},{"name":"SGLang","url":"https://lattice.kunjshah.in/inference-serving/sglang","homepage":"https://github.com/sgl-project/sglang","section":"inference-serving","layer":1,"band":"compute","kind":"runtime","roles":["serving"],"deployment":"self-hosted","license":"Apache-2.0","language":"Python","cost":"free","useWhen":"A large share of requests share a long prefix, or you need constrained decoding.","skipWhen":"You want the fewest moving parts and vLLM already clears your bar.","alternatives":["vLLM","Text Generation Inference"],"alsoIn":[],"verified":"2026-10"},{"name":"llama.cpp","url":"https://lattice.kunjshah.in/inference-serving/llama-cpp","homepage":"https://github.com/ggml-org/llama.cpp","section":"inference-serving","layer":1,"band":"compute","kind":"runtime","roles":["serving"],"deployment":"self-hosted","license":"MIT","language":"C++","cost":"free","useWhen":"The constraint is hardware, not throughput — no GPU, or an edge box.","skipWhen":"You need maximum concurrent throughput on datacentre GPUs.","alternatives":["Ollama","llamafile","KoboldCpp"],"alsoIn":[],"verified":"2026-09"},{"name":"Ollama","url":"https://lattice.kunjshah.in/inference-serving/ollama","homepage":"https://ollama.com","section":"inference-serving","layer":1,"band":"compute","kind":"runtime","roles":["serving","applied"],"deployment":"self-hosted","license":"MIT","language":"Go","cost":"free","useWhen":"Local model access for a developer or small team, with no setup.","skipWhen":"You need production throughput or a stable server API.","alternatives":["llama.cpp","LM Studio","llamafile"],"alsoIn":[],"verified":"2026-09"},{"name":"TensorRT-LLM","url":"https://lattice.kunjshah.in/inference-serving/tensorrt-llm","homepage":"https://github.com/NVIDIA/TensorRT-LLM","section":"inference-serving","layer":1,"band":"compute","kind":"runtime","roles":["serving"],"deployment":"self-hosted","license":"Apache-2.0","language":"CUDA","cost":"free","useWhen":"You are committed to NVIDIA hardware and need the last of the throughput.","skipWhen":"Hardware portability matters, or you have no GPUs to tune against.","alternatives":["vLLM","SGLang"],"alsoIn":[],"verified":"2026-09"},{"name":"Text Generation Inference","url":"https://lattice.kunjshah.in/inference-serving/text-generation-inference","homepage":"https://github.com/huggingface/text-generation-inference","section":"inference-serving","layer":1,"band":"compute","kind":"runtime","roles":["serving"],"deployment":"self-hosted","license":"Apache-2.0","language":"Rust","cost":"free","useWhen":"Multi-GPU tensor parallelism inside an existing Hugging Face stack.","skipWhen":"The model fits on one GPU, where this is overkill.","alternatives":["vLLM","SGLang"],"alsoIn":[],"verified":"2026-09"},{"name":"LM Studio","url":"https://lattice.kunjshah.in/inference-serving/lm-studio","homepage":"https://lmstudio.ai","section":"inference-serving","layer":1,"band":"compute","kind":"service","roles":["serving","applied"],"deployment":"self-hosted","license":"proprietary","language":"TypeScript","cost":"free","useWhen":"A desktop GUI for running and serving local models, aimed at one operator.","skipWhen":"You need headless, multi-tenant deployment.","alternatives":["Ollama","llama.cpp"],"alsoIn":[],"verified":"2026-09"},{"name":"Triton Inference Server","url":"https://lattice.kunjshah.in/inference-serving/triton-inference-server","homepage":"https://github.com/triton-inference-server/server","section":"inference-serving","layer":1,"band":"compute","kind":"runtime","roles":["serving"],"deployment":"self-hosted","license":"BSD-3-Clause","language":"C++","cost":"free","useWhen":"One endpoint serving ONNX, TensorRT and Python models side by side.","skipWhen":"You serve one model family and want peak LLM throughput.","alternatives":["vLLM","Text Generation Inference"],"alsoIn":[],"verified":"2026-09"},{"name":"llamafile","url":"https://lattice.kunjshah.in/inference-serving/llamafile","homepage":"https://github.com/Mozilla-Ocho/llamafile","section":"inference-serving","layer":1,"band":"compute","kind":"runtime","roles":["serving"],"deployment":"self-hosted","license":"Apache-2.0","language":"C++","cost":"free","useWhen":"Shipping a model as a single executable with no install step.","skipWhen":"You need batching, multi-GPU serving or a production HTTP server.","alternatives":["llama.cpp","Ollama"],"alsoIn":[],"verified":"2026-09"},{"name":"KoboldCpp","url":"https://lattice.kunjshah.in/inference-serving/koboldcpp","homepage":"https://github.com/LostRuins/koboldcpp","section":"inference-serving","layer":1,"band":"compute","kind":"runtime","roles":["serving"],"deployment":"self-hosted","license":"MIT","language":"C++","cost":"free","useWhen":"Local generation with a UI tuned for interactive, long-form use.","skipWhen":"You need headless serving at scale.","alternatives":["llama.cpp","Ollama"],"alsoIn":[],"verified":"2026-09"},{"name":"PowerInfer","url":"https://lattice.kunjshah.in/inference-serving/powerinfer","homepage":"https://github.com/AmateurChina/PowerInfer","section":"inference-serving","layer":1,"band":"compute","kind":"runtime","roles":["serving"],"deployment":"self-hosted","license":"Apache-2.0","language":"C++","cost":"free","useWhen":"CPU-first serving where the GPU accelerates rather than carries the load.","skipWhen":"You have GPUs to spare — it trades peak throughput for reach.","alternatives":["llama.cpp","vLLM"],"alsoIn":[],"verified":"2026-09"},{"name":"Marlin","url":"https://lattice.kunjshah.in/inference-serving/marlin","homepage":"https://github.com/vllm-project/marlin","section":"inference-serving","layer":1,"band":"compute","kind":"library","roles":["serving"],"deployment":"self-hosted","license":"Apache-2.0","language":"CUDA","cost":"free","useWhen":"Near-GPU throughput from 4-bit weights, as a kernel inside another runtime.","skipWhen":"You need a scheduler and a server as well as a kernel.","alternatives":[],"alsoIn":[],"verified":"2026-09"},{"name":"LiteLLM","url":"https://lattice.kunjshah.in/routing-gateways/litellm","homepage":"https://litellm.ai","section":"routing-gateways","layer":2,"band":"compute","kind":"library","roles":["platform"],"deployment":"self-hosted","license":"MIT","language":"Python","cost":"free","useWhen":"The widest provider coverage, deployed the least committal way.","skipWhen":"You want routing policy to live in a SaaS console.","alternatives":["Portkey","Cloudflare AI Gateway","Envoy AI Gateway"],"alsoIn":[],"verified":"2026-09"},{"name":"Portkey","url":"https://lattice.kunjshah.in/routing-gateways/portkey","homepage":"https://portkey.ai","section":"routing-gateways","layer":2,"band":"compute","kind":"platform","roles":["platform"],"deployment":"saas","license":"proprietary","language":null,"cost":"usage-based","useWhen":"A managed gateway with guardrails and analytics already bundled.","skipWhen":"Prompts cannot leave your infrastructure.","alternatives":["LiteLLM","Cloudflare AI Gateway"],"alsoIn":[{"section":"guardrails-safety","layer":7,"because":"Its guardrails run inline in the request path, which is where safety checks belong rather than after the response."}],"verified":"2026-09"},{"name":"Cloudflare AI Gateway","url":"https://lattice.kunjshah.in/routing-gateways/cloudflare-ai-gateway","homepage":"https://developers.cloudflare.com","section":"routing-gateways","layer":2,"band":"compute","kind":"service","roles":["platform"],"deployment":"saas","license":"proprietary","language":null,"cost":"free-tier","useWhen":"Edge caching and latency dominate, on Cloudflare already.","skipWhen":"Requests must not transit a third party's network.","alternatives":["Portkey","Envoy AI Gateway"],"alsoIn":[],"verified":"2026-09"},{"name":"OpenRouter","url":"https://lattice.kunjshah.in/routing-gateways/openrouter","homepage":"https://openrouter.ai","section":"routing-gateways","layer":2,"band":"compute","kind":"service","roles":["platform"],"deployment":"saas","license":"proprietary","language":null,"cost":"usage-based","useWhen":"One key and one bill across many hosted models, while experimenting.","skipWhen":"You need self-hosting or per-provider SLAs.","alternatives":["LiteLLM","Martian"],"alsoIn":[],"verified":"2026-09"},{"name":"Martian","url":"https://lattice.kunjshah.in/routing-gateways/martian","homepage":"https://withmartian.com","section":"routing-gateways","layer":2,"band":"compute","kind":"platform","roles":["platform"],"deployment":"saas","license":"proprietary","language":null,"cost":"usage-based","useWhen":"Routing tuned on your own traffic rather than a public benchmark.","skipWhen":"You want the routing rule to be plain config you can read.","alternatives":["RouteLLM","Not Diamond","OpenRouter"],"alsoIn":[],"verified":"2026-09"},{"name":"Envoy AI Gateway","url":"https://lattice.kunjshah.in/routing-gateways/envoy-ai-gateway","homepage":"https://aigateway.envoyproxy.io","section":"routing-gateways","layer":2,"band":"compute","kind":"runtime","roles":["platform","serving"],"deployment":"self-hosted","license":"Apache-2.0","language":"Go","cost":"free","useWhen":"A Kubernetes shop that already runs Envoy and needs the gateway in-cluster.","skipWhen":"You want a gateway configurable without a data plane.","alternatives":["LiteLLM","Cloudflare AI Gateway"],"alsoIn":[],"verified":"2026-09"},{"name":"Bifrost","url":"https://lattice.kunjshah.in/routing-gateways/bifrost","homepage":"https://getmaxim.ai","section":"routing-gateways","layer":2,"band":"compute","kind":"library","roles":["platform"],"deployment":"self-hosted","license":"proprietary","language":"Go","cost":"free","useWhen":"A high-throughput OpenAI-compatible proxy with a small footprint.","skipWhen":"You need the breadth of a full routing stack.","alternatives":["LiteLLM"],"alsoIn":[],"verified":"2026-09"},{"name":"RouteLLM","url":"https://lattice.kunjshah.in/routing-gateways/routellm","homepage":"https://github.com/lm-sys/RouteLLM","section":"routing-gateways","layer":2,"band":"compute","kind":"library","roles":["platform","applied"],"deployment":"self-hosted","license":"Apache-2.0","language":"Python","cost":"free","useWhen":"Cutting cost by matching query difficulty to model size, with a learned router.","skipWhen":"You cannot evaluate the quality you would be trading away.","alternatives":["Martian","Not Diamond"],"alsoIn":[],"verified":"2026-09"},{"name":"Not Diamond","url":"https://lattice.kunjshah.in/routing-gateways/not-diamond","homepage":"https://notdiamond.ai","section":"routing-gateways","layer":2,"band":"compute","kind":"platform","roles":["platform","applied"],"deployment":"saas","license":"proprietary","language":null,"cost":"usage-based","useWhen":"Routing and prompt optimisation tuned on your own traffic.","skipWhen":"Self-hosting or auditability is a requirement.","alternatives":["Martian","RouteLLM"],"alsoIn":[],"verified":"2026-09"},{"name":"TrueFoundry","url":"https://lattice.kunjshah.in/routing-gateways/truefoundry","homepage":"https://truefoundry.com","section":"routing-gateways","layer":2,"band":"compute","kind":"platform","roles":["platform","production"],"deployment":"managed","license":"proprietary","language":null,"cost":"subscription","useWhen":"Gateway, observability and guardrails as one managed deployment.","skipWhen":"You want to adopt the pieces independently.","alternatives":["Portkey","Braintrust"],"alsoIn":[{"section":"evaluation-observability","layer":9,"because":"Its tracing and dashboards are the observability half of the same deployment, not an add-on."},{"section":"guardrails-safety","layer":7,"because":"Its guardrails enforce in the call path alongside the gateway rather than as a separate service."}],"verified":"2026-09"},{"name":"pgvector","url":"https://lattice.kunjshah.in/retrieval-vector-stores/pgvector","homepage":"https://github.com/pgvector/pgvector","section":"retrieval-vector-stores","layer":3,"band":"state","kind":"database","roles":["data"],"deployment":"self-hosted","license":"PostgreSQL","language":"SQL","cost":"free","useWhen":"Under a few million vectors, or when retrieval joins rows you already have.","skipWhen":"Vector search has become the workload rather than a side feature.","alternatives":["Qdrant","Milvus","Weaviate"],"alsoIn":[],"verified":"2026-09"},{"name":"Qdrant","url":"https://lattice.kunjshah.in/retrieval-vector-stores/qdrant","homepage":"https://qdrant.tech","section":"retrieval-vector-stores","layer":3,"band":"state","kind":"database","roles":["data"],"deployment":"self-hosted","license":"Apache-2.0","language":"Rust","cost":"free","useWhen":"Vector-first workloads with heavy metadata filtering at scale.","skipWhen":"You want no new datastore to operate.","alternatives":["Weaviate","Milvus","pgvector"],"alsoIn":[],"verified":"2026-09"},{"name":"Weaviate","url":"https://lattice.kunjshah.in/retrieval-vector-stores/weaviate","homepage":"https://weaviate.io","section":"retrieval-vector-stores","layer":3,"band":"state","kind":"database","roles":["data"],"deployment":"self-hosted","license":"BSD-3-Clause","language":"Go","cost":"free","useWhen":"Hybrid or multi-vector search with a graph-shaped data model.","skipWhen":"Your data is genuinely relational and joins matter more than vectors.","alternatives":["Qdrant","Elasticsearch","Vespa"],"alsoIn":[],"verified":"2026-09"},{"name":"Chroma","url":"https://lattice.kunjshah.in/retrieval-vector-stores/chroma","homepage":"https://trychroma.com","section":"retrieval-vector-stores","layer":3,"band":"state","kind":"database","roles":["data","applied"],"deployment":"self-hosted","license":"Apache-2.0","language":"Python","cost":"free","useWhen":"Getting a prototype working in an afternoon.","skipWhen":"You need durability, scaling or concurrent writes.","alternatives":["Qdrant","pgvector"],"alsoIn":[],"verified":"2026-09"},{"name":"Pinecone","url":"https://lattice.kunjshah.in/retrieval-vector-stores/pinecone","homepage":"https://pinecone.io","section":"retrieval-vector-stores","layer":3,"band":"state","kind":"database","roles":["data"],"deployment":"saas","license":"proprietary","language":null,"cost":"usage-based","useWhen":"Zero operational work, and vector search is not your core competence.","skipWhen":"Residency, cost predictability or a query language you control.","alternatives":["Turbopuffer","Milvus","Qdrant"],"alsoIn":[],"verified":"2026-09"},{"name":"Milvus","url":"https://lattice.kunjshah.in/retrieval-vector-stores/milvus","homepage":"https://milvus.io","section":"retrieval-vector-stores","layer":3,"band":"state","kind":"database","roles":["data"],"deployment":"self-hosted","license":"Apache-2.0","language":"Go","cost":"free","useWhen":"Billion-scale indexes with a cloud-native deployment.","skipWhen":"You need a small, comprehensible system.","alternatives":["Qdrant","Weaviate","pgvector"],"alsoIn":[],"verified":"2026-09"},{"name":"Turbopuffer","url":"https://lattice.kunjshah.in/retrieval-vector-stores/turbopuffer","homepage":"https://turbopuffer.com","section":"retrieval-vector-stores","layer":3,"band":"state","kind":"database","roles":["data"],"deployment":"saas","license":"proprietary","language":null,"cost":"usage-based","useWhen":"High-recall retrieval at volume, without operating the index yourself.","skipWhen":"You need to tune the index directly.","alternatives":["Pinecone","Qdrant"],"alsoIn":[],"verified":"2026-09"},{"name":"Unstructured","url":"https://lattice.kunjshah.in/retrieval-vector-stores/unstructured","homepage":"https://unstructured.io","section":"retrieval-vector-stores","layer":3,"band":"state","kind":"library","roles":["data"],"deployment":"self-hosted","license":"Apache-2.0","language":"Python","cost":"free","useWhen":"Partitioning messy documents into retrievable chunks before indexing.","skipWhen":"Your input is already structured.","alternatives":["Docling","LlamaParse"],"alsoIn":[],"verified":"2026-09"},{"name":"Elasticsearch","url":"https://lattice.kunjshah.in/retrieval-vector-stores/elasticsearch","homepage":"https://elastic.co","section":"retrieval-vector-stores","layer":3,"band":"state","kind":"database","roles":["data"],"deployment":"self-hosted","license":"Elastic-License-2.0","language":"Java","cost":"usage-based","useWhen":"One index carrying BM25 and dense vectors, with ranking you can tune.","skipWhen":"You need a permissive licence — the Elastic licence is not OSI-approved.","alternatives":["Vespa","Weaviate","Qdrant"],"alsoIn":[],"verified":"2026-09"},{"name":"Vespa","url":"https://lattice.kunjshah.in/retrieval-vector-stores/vespa","homepage":"https://vespa.ai","section":"retrieval-vector-stores","layer":3,"band":"state","kind":"platform","roles":["data"],"deployment":"self-hosted","license":"Apache-2.0","language":"Java","cost":"free","useWhen":"Ranking-heavy retrieval where the ranking function is the product.","skipWhen":"You want a small, low-operations component.","alternatives":["Elasticsearch","Weaviate"],"alsoIn":[],"verified":"2026-09"},{"name":"Rerankers","url":"https://lattice.kunjshah.in/retrieval-vector-stores/rerankers","homepage":"https://cohere.com","section":"retrieval-vector-stores","layer":3,"band":"state","kind":"service","roles":["data"],"deployment":"saas","license":"proprietary","language":null,"cost":"usage-based","useWhen":"A cross-encoder in the second retrieval stage, to lift precision cheaply.","skipWhen":"You have no way to measure whether recall actually improved.","alternatives":["Jina AI","Unstructured"],"alsoIn":[],"verified":"2026-09"},{"name":"Jina AI","url":"https://lattice.kunjshah.in/retrieval-vector-stores/jina-ai","homepage":"https://jina.ai","section":"retrieval-vector-stores","layer":3,"band":"state","kind":"service","roles":["data"],"deployment":"saas","license":"proprietary","language":null,"cost":"usage-based","useWhen":"Multimodal embeddings and a reranker behind one endpoint.","skipWhen":"Embeddings must stay in-house.","alternatives":["Rerankers"],"alsoIn":[],"verified":"2026-09"},{"name":"LangChain Text Splitters","url":"https://lattice.kunjshah.in/retrieval-vector-stores/langchain-text-splitters","homepage":"https://python.langchain.com","section":"retrieval-vector-stores","layer":3,"band":"state","kind":"library","roles":["data","applied"],"deployment":"self-hosted","license":"MIT","language":"Python","cost":"free","useWhen":"Document loaders and splitters, if you are already in that ecosystem.","skipWhen":"Structure matters — fixed-size splitting is rarely the right boundary.","alternatives":["Unstructured","Docling"],"alsoIn":[],"verified":"2026-09"},{"name":"Docling","url":"https://lattice.kunjshah.in/retrieval-vector-stores/docling","homepage":"https://github.com/DS4SD/docling","section":"retrieval-vector-stores","layer":3,"band":"state","kind":"library","roles":["data"],"deployment":"self-hosted","license":"MIT","language":"Python","cost":"free","useWhen":"Layout-aware parsing that preserves tables and heading structure.","skipWhen":"Your documents are plain text.","alternatives":["Unstructured","LlamaParse"],"alsoIn":[],"verified":"2026-09"},{"name":"LlamaParse","url":"https://lattice.kunjshah.in/retrieval-vector-stores/llamaparse","homepage":"https://llamaindex.ai","section":"retrieval-vector-stores","layer":3,"band":"state","kind":"service","roles":["data"],"deployment":"managed","license":"proprietary","language":null,"cost":"usage-based","useWhen":"Managed parsing for PDFs, especially the ugly ones.","skipWhen":"Document contents cannot leave your infrastructure.","alternatives":["Docling","Unstructured"],"alsoIn":[],"verified":"2026-09"},{"name":"Unsloth","url":"https://lattice.kunjshah.in/fine-tuning/unsloth","homepage":"https://unsloth.ai","section":"fine-tuning","layer":4,"band":"state","kind":"library","roles":["applied","serving"],"deployment":"self-hosted","license":"Apache-2.0","language":"Python","cost":"free","useWhen":"LoRA fine-tuning on a single GPU, with memory and time roughly halved.","skipWhen":"You need a training stack you can audit line by line.","alternatives":["Axolotl","PEFT"],"alsoIn":[],"verified":"2026-09"},{"name":"Axolotl","url":"https://lattice.kunjshah.in/fine-tuning/axolotl","homepage":"https://axolotl.ai","section":"fine-tuning","layer":4,"band":"state","kind":"library","roles":["applied"],"deployment":"self-hosted","license":"Apache-2.0","language":"Python","cost":"free","useWhen":"Configuration-driven fine-tuning across many architectures.","skipWhen":"You need to hand-tune the training loop itself.","alternatives":["LLaMA-Factory","Unsloth"],"alsoIn":[],"verified":"2026-09"},{"name":"LLaMA-Factory","url":"https://lattice.kunjshah.in/fine-tuning/llama-factory","homepage":"https://github.com/hiyouga/LLaMA-Factory","section":"fine-tuning","layer":4,"band":"state","kind":"platform","roles":["applied"],"deployment":"self-hosted","license":"Apache-2.0","language":"Python","cost":"free","useWhen":"One interface for SFT, DPO and RLHF across open models.","skipWhen":"You want a minimal, readable training script.","alternatives":["Axolotl","TRL"],"alsoIn":[],"verified":"2026-09"},{"name":"PEFT","url":"https://lattice.kunjshah.in/fine-tuning/peft","homepage":"https://github.com/huggingface/peft","section":"fine-tuning","layer":4,"band":"state","kind":"library","roles":["applied"],"deployment":"self-hosted","license":"BSD-3-Clause","language":"Python","cost":"free","useWhen":"Parameter-efficient fine-tuning — LoRA and variants — on limited hardware.","skipWhen":"You need to change the model's capabilities, not bolt on an adapter.","alternatives":["Unsloth","TRL"],"alsoIn":[],"verified":"2026-09"},{"name":"TRL","url":"https://lattice.kunjshah.in/fine-tuning/trl","homepage":"https://github.com/huggingface/trl","section":"fine-tuning","layer":4,"band":"state","kind":"library","roles":["applied"],"deployment":"self-hosted","license":"Apache-2.0","language":"Python","cost":"free","useWhen":"Post-training trainers for SFT, DPO and GRPO on top of any PEFT setup.","skipWhen":"You are training from scratch rather than aligning an existing model.","alternatives":["PEFT","LLaMA-Factory","torchtune"],"alsoIn":[],"verified":"2026-09"},{"name":"DeepSpeed","url":"https://lattice.kunjshah.in/fine-tuning/deepspeed","homepage":"https://deepspeed.ai","section":"fine-tuning","layer":4,"band":"state","kind":"library","roles":["serving","applied"],"deployment":"self-hosted","license":"MIT","language":"Python","cost":"free","useWhen":"ZeRO sharding and pipeline parallelism across many GPUs.","skipWhen":"The model fits on one device — the complexity buys nothing.","alternatives":["Megatron-LM"],"alsoIn":[],"verified":"2026-09"},{"name":"Megatron-LM","url":"https://lattice.kunjshah.in/fine-tuning/megatron-lm","homepage":"https://github.com/NVIDIA/Megatron-LM","section":"fine-tuning","layer":4,"band":"state","kind":"library","roles":["serving"],"deployment":"self-hosted","license":"Apache-2.0","language":"CUDA","cost":"free","useWhen":"Tensor and pipeline parallel pretraining at cluster scale.","skipWhen":"You are fine-tuning rather than pretraining.","alternatives":["DeepSpeed"],"alsoIn":[],"verified":"2026-09"},{"name":"torchtune","url":"https://lattice.kunjshah.in/fine-tuning/torchtune","homepage":"https://github.com/pytorch/torchtune","section":"fine-tuning","layer":4,"band":"state","kind":"library","roles":["applied"],"deployment":"self-hosted","license":"BSD-3-Clause","language":"Python","cost":"free","useWhen":"PyTorch-native recipes that you can read and modify.","skipWhen":"You want the breadth of a config-driven stack.","alternatives":["TRL","LLaMA-Factory"],"alsoIn":[],"verified":"2026-09"},{"name":"Hugging Face TRL","url":"https://lattice.kunjshah.in/fine-tuning/hugging-face-trl","homepage":"https://huggingface.co","section":"fine-tuning","layer":4,"band":"state","kind":"platform","roles":["applied"],"deployment":"saas","license":"proprietary","language":null,"cost":"usage-based","useWhen":"Preference and reward optimisation hosted, without a cluster.","skipWhen":"Training data cannot be uploaded to a third party.","alternatives":["TRL"],"alsoIn":[],"verified":"2026-09"},{"name":"Colab","url":"https://lattice.kunjshah.in/fine-tuning/colab","homepage":"https://colab.research.google.com","section":"fine-tuning","layer":4,"band":"state","kind":"platform","roles":["applied"],"deployment":"saas","license":"proprietary","language":null,"cost":"free-tier","useWhen":"A disposable GPU for a quick experiment, with no setup.","skipWhen":"Anything reproducible — a notebook is not a training pipeline.","alternatives":["Replicate"],"alsoIn":[],"verified":"2026-09"},{"name":"Replicate","url":"https://lattice.kunjshah.in/fine-tuning/replicate","homepage":"https://replicate.com","section":"fine-tuning","layer":4,"band":"state","kind":"platform","roles":["applied"],"deployment":"saas","license":"proprietary","language":null,"cost":"usage-based","useWhen":"Hosted fine-tuning and deployment of an open model, billed per run.","skipWhen":"You need custom training code or strict residency.","alternatives":["Colab","Weights & Biases Launch"],"alsoIn":[],"verified":"2026-09"},{"name":"Weights & Biases Launch","url":"https://lattice.kunjshah.in/fine-tuning/weights-biases-launch","homepage":"https://wandb.ai","section":"fine-tuning","layer":4,"band":"state","kind":"platform","roles":["production","applied"],"deployment":"saas","license":"proprietary","language":null,"cost":"subscription","useWhen":"Managed training runs with sweeps and artifact tracking wired up.","skipWhen":"You want training infrastructure you run yourself.","alternatives":["Replicate","Weights & Biases"],"alsoIn":[{"section":"evaluation-observability","layer":9,"because":"A training run is an experiment, scored against a baseline the same way an eval set scores a prompt."}],"verified":"2026-09"},{"name":"LangChain","url":"https://lattice.kunjshah.in/agent-frameworks/langchain","homepage":"https://langchain.com","section":"agent-frameworks","layer":5,"band":"control","kind":"framework","roles":["applied"],"deployment":"self-hosted","license":"MIT","language":"Python","cost":"free","useWhen":"Tool use and state, in an ecosystem where the answers already exist.","skipWhen":"You want a thin, legible core — this is a large surface.","alternatives":["LlamaIndex","Pydantic AI","CrewAI"],"alsoIn":[],"verified":"2026-09"},{"name":"LlamaIndex","url":"https://lattice.kunjshah.in/agent-frameworks/llamaindex","homepage":"https://llamaindex.ai","section":"agent-frameworks","layer":5,"band":"control","kind":"framework","roles":["applied","data"],"deployment":"self-hosted","license":"MIT","language":"Python","cost":"free","useWhen":"Data-centric work where retrieval is the centre of the problem.","skipWhen":"You need agent orchestration more than data access.","alternatives":["LangChain","Pydantic AI"],"alsoIn":[],"verified":"2026-09"},{"name":"Pydantic AI","url":"https://lattice.kunjshah.in/agent-frameworks/pydantic-ai","homepage":"https://ai.pydantic.dev","section":"agent-frameworks","layer":5,"band":"control","kind":"framework","roles":["applied"],"deployment":"self-hosted","license":"MIT","language":"Python","cost":"free","useWhen":"Type-safe agents where schema correctness is the priority.","skipWhen":"You want a batteries-included ecosystem.","alternatives":["LangChain","smolagents","LlamaIndex"],"alsoIn":[],"verified":"2026-09"},{"name":"AutoGen","url":"https://lattice.kunjshah.in/agent-frameworks/autogen","homepage":"https://microsoft.github.io/autogen","section":"agent-frameworks","layer":5,"band":"control","kind":"framework","roles":["applied"],"deployment":"self-hosted","license":"MIT","language":"Python","cost":"free","useWhen":"Research into conversable multi-agent systems.","skipWhen":"You need something stable and documented for production.","alternatives":["CrewAI","LangChain"],"alsoIn":[],"verified":"2026-09"},{"name":"CrewAI","url":"https://lattice.kunjshah.in/agent-frameworks/crewai","homepage":"https://crewai.com","section":"agent-frameworks","layer":5,"band":"control","kind":"framework","roles":["applied"],"deployment":"self-hosted","license":"MIT","language":"Python","cost":"free","useWhen":"Role-based orchestration where agents are framed as a team.","skipWhen":"You want direct control over the loop itself.","alternatives":["AutoGen","LangChain"],"alsoIn":[],"verified":"2026-09"},{"name":"Semantic Kernel","url":"https://lattice.kunjshah.in/agent-frameworks/semantic-kernel","homepage":"https://learn.microsoft.com","section":"agent-frameworks","layer":5,"band":"control","kind":"framework","roles":["applied"],"deployment":"self-hosted","license":"MIT","language":"C#","cost":"free","useWhen":"Embedding AI steps into an existing .NET estate.","skipWhen":"Your stack is not .NET.","alternatives":["LangChain","Pydantic AI"],"alsoIn":[],"verified":"2026-09"},{"name":"Mastra","url":"https://lattice.kunjshah.in/agent-frameworks/mastra","homepage":"https://mastra.ai","section":"agent-frameworks","layer":5,"band":"control","kind":"framework","roles":["applied"],"deployment":"self-hosted","license":"Apache-2.0","language":"TypeScript","cost":"free","useWhen":"TypeScript agents with typed workflows and evals built in.","skipWhen":"You are working in Python.","alternatives":["Vercel AI SDK","OpenAI Agents SDK"],"alsoIn":[{"section":"evaluation-observability","layer":9,"because":"Its eval harness ships inside the framework, so grading an agent is part of running it rather than a separate tool."}],"verified":"2026-09"},{"name":"OpenAI Agents SDK","url":"https://lattice.kunjshah.in/agent-frameworks/openai-agents-sdk","homepage":"https://openai.github.io/openai-agents-python","section":"agent-frameworks","layer":5,"band":"control","kind":"framework","roles":["applied"],"deployment":"self-hosted","license":"Apache-2.0","language":"Python","cost":"free","useWhen":"A small, legible set of primitives for handoffs and guardrails.","skipWhen":"You need multi-provider abstraction at every layer.","alternatives":["Claude Agent SDK","smolagents"],"alsoIn":[{"section":"guardrails-safety","layer":7,"because":"Guardrails are a first-party primitive on the runner rather than a proxy in front of it."}],"verified":"2026-09"},{"name":"Agno","url":"https://lattice.kunjshah.in/agent-frameworks/agno","homepage":"https://agno.com","section":"agent-frameworks","layer":5,"band":"control","kind":"framework","roles":["applied"],"deployment":"self-hosted","license":"MPL-2.0","language":"Python","cost":"free","useWhen":"A minimal agent runtime with model-agnostic tool interfaces.","skipWhen":"You need the wider ecosystem around it.","alternatives":["smolagents","Pydantic AI"],"alsoIn":[],"verified":"2026-09"},{"name":"Claude Agent SDK","url":"https://lattice.kunjshah.in/agent-frameworks/claude-agent-sdk","homepage":"https://docs.anthropic.com","section":"agent-frameworks","layer":5,"band":"control","kind":"framework","roles":["applied"],"deployment":"self-hosted","license":"proprietary","language":"TypeScript","cost":"free","useWhen":"Building agents against Anthropic models, with tool use and hooks.","skipWhen":"You need provider independence at the framework layer.","alternatives":["OpenAI Agents SDK","Letta"],"alsoIn":[],"verified":"2026-09"},{"name":"Letta","url":"https://lattice.kunjshah.in/agent-frameworks/letta","homepage":"https://letta.com","section":"agent-frameworks","layer":5,"band":"control","kind":"platform","roles":["applied","data"],"deployment":"self-hosted","license":"Apache-2.0","language":"Python","cost":"free","useWhen":"Agents whose memory is a first-class, inspectable and editable state.","skipWhen":"You want memory to be a detail the framework handles for you.","alternatives":["smolagents","OpenAI Agents SDK"],"alsoIn":[{"section":"retrieval-vector-stores","layer":3,"because":"Its memory is persisted and retrieved over an embedding store, so it is a layer-3 store with an agent API on top."}],"verified":"2026-09"},{"name":"smolagents","url":"https://lattice.kunjshah.in/agent-frameworks/smolagents","homepage":"https://huggingface.co","section":"agent-frameworks","layer":5,"band":"control","kind":"framework","roles":["applied"],"deployment":"self-hosted","license":"Apache-2.0","language":"Python","cost":"free","useWhen":"Reading the whole agent loop in a page — it is deliberately tiny.","skipWhen":"You need a rich tool ecosystem around it.","alternatives":["Agno","Letta"],"alsoIn":[],"verified":"2026-09"},{"name":"Vercel AI SDK","url":"https://lattice.kunjshah.in/agent-frameworks/vercel-ai-sdk","homepage":"https://ai-sdk.dev","section":"agent-frameworks","layer":5,"band":"control","kind":"library","roles":["applied"],"deployment":"self-hosted","license":"Apache-2.0","language":"TypeScript","cost":"free","useWhen":"Streaming model calls in a TypeScript app, provider-agnostically.","skipWhen":"You need agent orchestration rather than model calls.","alternatives":["Mastra","OpenAI Agents SDK"],"alsoIn":[],"verified":"2026-09"},{"name":"Temporal","url":"https://lattice.kunjshah.in/workflow-orchestration/temporal","homepage":"https://temporal.io","section":"workflow-orchestration","layer":6,"band":"control","kind":"platform","roles":["platform"],"deployment":"self-hosted","license":"MIT","language":"Go","cost":"free","useWhen":"Work spanning minutes or hours that must survive restarts and retries.","skipWhen":"You need it live in a day — the adoption cost is real.","alternatives":["Inngest","Restate","DBOS"],"alsoIn":[],"verified":"2026-09"},{"name":"Inngest","url":"https://lattice.kunjshah.in/workflow-orchestration/inngest","homepage":"https://inngest.com","section":"workflow-orchestration","layer":6,"band":"control","kind":"platform","roles":["platform"],"deployment":"saas","license":"Apache-2.0","language":"TypeScript","cost":"free-tier","useWhen":"Durable step functions in TypeScript, without running the engine.","skipWhen":"You are not working in TypeScript.","alternatives":["Trigger.dev","Temporal"],"alsoIn":[],"verified":"2026-09"},{"name":"Trigger.dev","url":"https://lattice.kunjshah.in/workflow-orchestration/trigger-dev","homepage":"https://trigger.dev","section":"workflow-orchestration","layer":6,"band":"control","kind":"platform","roles":["platform"],"deployment":"saas","license":"Apache-2.0","language":"TypeScript","cost":"free-tier","useWhen":"Long-running background jobs that must survive a deploy.","skipWhen":"You need more than TypeScript.","alternatives":["Inngest","Temporal"],"alsoIn":[],"verified":"2026-09"},{"name":"Dagster","url":"https://lattice.kunjshah.in/workflow-orchestration/dagster","homepage":"https://dagster.io","section":"workflow-orchestration","layer":6,"band":"control","kind":"platform","roles":["platform","data"],"deployment":"self-hosted","license":"Apache-2.0","language":"Python","cost":"free","useWhen":"Data and ML pipelines framed as assets with explicit dependencies.","skipWhen":"You want a task queue rather than a lineage-aware orchestrator.","alternatives":["Prefect","Apache Airflow"],"alsoIn":[],"verified":"2026-09"},{"name":"Prefect","url":"https://lattice.kunjshah.in/workflow-orchestration/prefect","homepage":"https://prefect.io","section":"workflow-orchestration","layer":6,"band":"control","kind":"platform","roles":["platform","data"],"deployment":"self-hosted","license":"Apache-2.0","language":"Python","cost":"free-tier","useWhen":"Python-native flows with a managed cloud option behind them.","skipWhen":"You need deep lineage modelling.","alternatives":["Dagster","Apache Airflow"],"alsoIn":[],"verified":"2026-09"},{"name":"Apache Airflow","url":"https://lattice.kunjshah.in/workflow-orchestration/apache-airflow","homepage":"https://apache.org","section":"workflow-orchestration","layer":6,"band":"control","kind":"platform","roles":["platform","data"],"deployment":"self-hosted","license":"Apache-2.0","language":"Python","cost":"free","useWhen":"Scheduled batch data engineering, where DAGs are the shared language.","skipWhen":"You are building a latency-sensitive product — it was not designed for one.","alternatives":["Dagster","Prefect"],"alsoIn":[],"verified":"2026-09"},{"name":"Restate","url":"https://lattice.kunjshah.in/workflow-orchestration/restate","homepage":"https://restate.dev","section":"workflow-orchestration","layer":6,"band":"control","kind":"platform","roles":["platform"],"deployment":"self-hosted","license":"BSL-1.1","language":"Rust","cost":"free","useWhen":"Durable execution with a genuinely low-latency stateful API surface.","skipWhen":"You need the largest ecosystem around the engine.","alternatives":["Temporal","DBOS"],"alsoIn":[],"verified":"2026-09"},{"name":"DBOS","url":"https://lattice.kunjshah.in/workflow-orchestration/dbos","homepage":"https://dbos.dev","section":"workflow-orchestration","layer":6,"band":"control","kind":"library","roles":["platform"],"deployment":"self-hosted","license":"Apache-2.0","language":"Python","cost":"free","useWhen":"Durable workflows as ordinary Python functions and decorators.","skipWhen":"You want a separate service to operate.","alternatives":["Temporal","Restate"],"alsoIn":[],"verified":"2026-09"},{"name":"Fly Machines","url":"https://lattice.kunjshah.in/workflow-orchestration/fly-machines","homepage":"https://fly.io","section":"workflow-orchestration","layer":6,"band":"control","kind":"platform","roles":["platform","serving"],"deployment":"managed","license":"proprietary","language":null,"cost":"usage-based","useWhen":"Hosting stateful containers that fits durable agent workers.","skipWhen":"You want to own the infrastructure layer.","alternatives":["Modal","Trigger.dev"],"alsoIn":[],"verified":"2026-09"},{"name":"Modal","url":"https://lattice.kunjshah.in/workflow-orchestration/modal","homepage":"https://modal.com","section":"workflow-orchestration","layer":6,"band":"control","kind":"platform","roles":["platform","serving"],"deployment":"saas","license":"proprietary","language":null,"cost":"usage-based","useWhen":"Serverless GPUs and containers for batch inference and scheduled jobs.","skipWhen":"Data locality rules forbid ephemeral compute.","alternatives":["Fly Machines","Replicate"],"alsoIn":[],"verified":"2026-09"},{"name":"NeMo Guardrails","url":"https://lattice.kunjshah.in/guardrails-safety/nemo-guardrails","homepage":"https://github.com/NVIDIA/NeMo-Guardrails","section":"guardrails-safety","layer":7,"band":"control","kind":"library","roles":["production"],"deployment":"self-hosted","license":"Apache-2.0","language":"Python","cost":"free","useWhen":"Constraining conversational flow with programmable rails.","skipWhen":"You need deep semantic moderation — it is not a classifier.","alternatives":["Guardrails AI","Llama Guard"],"alsoIn":[],"verified":"2026-09"},{"name":"Guardrails AI","url":"https://lattice.kunjshah.in/guardrails-safety/guardrails-ai","homepage":"https://guardrailsai.com","section":"guardrails-safety","layer":7,"band":"control","kind":"library","roles":["production"],"deployment":"self-hosted","license":"Apache-2.0","language":"Python","cost":"free","useWhen":"Validators that check output against a schema you define.","skipWhen":"You need free-form moderation rather than structural checks.","alternatives":["NeMo Guardrails","Invariant Guardrails"],"alsoIn":[],"verified":"2026-09"},{"name":"Llama Guard","url":"https://lattice.kunjshah.in/guardrails-safety/llama-guard","homepage":"https://ai.meta.com/llama","section":"guardrails-safety","layer":7,"band":"control","kind":"library","roles":["production"],"deployment":"self-hosted","license":"Llama-3.1-Community","language":"Python","cost":"free","useWhen":"One classifier covering both prompt and response moderation.","skipWhen":"You need domain-specific policy enforcement.","alternatives":["Lakera Guard","NeMo Guardrails"],"alsoIn":[],"verified":"2026-09"},{"name":"Microsoft Presidio","url":"https://lattice.kunjshah.in/guardrails-safety/microsoft-presidio","homepage":"https://microsoft.github.io/presidio","section":"guardrails-safety","layer":7,"band":"control","kind":"library","roles":["production"],"deployment":"self-hosted","license":"MIT","language":"Python","cost":"free","useWhen":"Detecting and anonymising PII in text and images.","skipWhen":"Your data is already pseudonymised at source.","alternatives":[],"alsoIn":[],"verified":"2026-09"},{"name":"Lakera Guard","url":"https://lattice.kunjshah.in/guardrails-safety/lakera-guard","homepage":"https://lakera.ai","section":"guardrails-safety","layer":7,"band":"control","kind":"service","roles":["production"],"deployment":"saas","license":"proprietary","language":null,"cost":"usage-based","useWhen":"Prompt-injection and jailbreak detection at the gateway.","skipWhen":"Requests cannot leave your network.","alternatives":["Llama Guard","Invariant Guardrails"],"alsoIn":[],"verified":"2026-09"},{"name":"garak","url":"https://lattice.kunjshah.in/guardrails-safety/garak","homepage":"https://github.com/NVIDIA/garak","section":"guardrails-safety","layer":7,"band":"control","kind":"library","roles":["production"],"deployment":"self-hosted","license":"Apache-2.0","language":"Python","cost":"free","useWhen":"Probing your own system for known vulnerability classes.","skipWhen":"You have no remediation path for whatever it finds.","alternatives":["PyRIT"],"alsoIn":[],"verified":"2026-09"},{"name":"PyRIT","url":"https://lattice.kunjshah.in/guardrails-safety/pyrit","homepage":"https://github.com/Azure/PyRIT","section":"guardrails-safety","layer":7,"band":"control","kind":"library","roles":["production"],"deployment":"self-hosted","license":"MIT","language":"Python","cost":"free","useWhen":"Generating and scoring attack prompts systematically.","skipWhen":"You are not authorised to test the target.","alternatives":["garak"],"alsoIn":[],"verified":"2026-09"},{"name":"Invariant Guardrails","url":"https://lattice.kunjshah.in/guardrails-safety/invariant-guardrails","homepage":"https://invariantlabs.ai","section":"guardrails-safety","layer":7,"band":"control","kind":"platform","roles":["production"],"deployment":"managed","license":"proprietary","language":"Python","cost":"usage-based","useWhen":"Guardrails as code, enforced inline in the call path.","skipWhen":"Self-hosting is a hard requirement.","alternatives":["Guardrails AI","Lakera Guard"],"alsoIn":[],"verified":"2026-09"},{"name":"DSPy","url":"https://lattice.kunjshah.in/prompt-engineering/dspy","homepage":"https://dspy.ai","section":"prompt-engineering","layer":8,"band":"control","kind":"library","roles":["applied"],"deployment":"self-hosted","license":"MIT","language":"Python","cost":"free","useWhen":"Compiling prompts into optimised programs from labelled examples.","skipWhen":"You have no examples and no way to score them.","alternatives":["TextGrad","PromptLayer"],"alsoIn":[],"verified":"2026-09"},{"name":"Instructor","url":"https://lattice.kunjshah.in/prompt-engineering/instructor","homepage":"https://python.useinstructor.com","section":"prompt-engineering","layer":8,"band":"control","kind":"library","roles":["applied"],"deployment":"self-hosted","license":"MIT","language":"Python","cost":"free","useWhen":"Schema-validated extraction with typed retry and partial streaming.","skipWhen":"You need a hard token-level guarantee rather than validation.","alternatives":["Guidance","Outlines"],"alsoIn":[],"verified":"2026-09"},{"name":"Humanloop","url":"https://lattice.kunjshah.in/prompt-engineering/humanloop","homepage":"https://humanloop.com","section":"prompt-engineering","layer":8,"band":"control","kind":"platform","roles":["applied","platform"],"deployment":"saas","license":"proprietary","language":null,"cost":"usage-based","useWhen":"Prompt versioning and evaluation run by a product team.","skipWhen":"Prompts or data cannot be sent to a vendor.","alternatives":["PromptLayer","Agenta"],"alsoIn":[],"verified":"2026-09"},{"name":"Guidance","url":"https://lattice.kunjshah.in/prompt-engineering/guidance","homepage":"https://guidance.mit.edu","section":"prompt-engineering","layer":8,"band":"control","kind":"library","roles":["applied"],"deployment":"self-hosted","license":"MIT","language":"Python","cost":"free","useWhen":"Constrained decoding with control flow interleaved into generation.","skipWhen":"You need better ergonomics for plain extraction.","alternatives":["Outlines","Instructor"],"alsoIn":[],"verified":"2026-09"},{"name":"PromptLayer","url":"https://lattice.kunjshah.in/prompt-engineering/promptlayer","homepage":"https://promptlayer.com","section":"prompt-engineering","layer":8,"band":"control","kind":"platform","roles":["applied","platform"],"deployment":"saas","license":"proprietary","language":null,"cost":"usage-based","useWhen":"A prompt registry with request logging and regression tests.","skipWhen":"You need self-hosting.","alternatives":["Humanloop","Agenta","Promptwatch"],"alsoIn":[],"verified":"2026-09"},{"name":"Agenta","url":"https://lattice.kunjshah.in/prompt-engineering/agenta","homepage":"https://agenta.ai","section":"prompt-engineering","layer":8,"band":"control","kind":"platform","roles":["applied","platform"],"deployment":"self-hosted","license":"Apache-2.0","language":"Python","cost":"free-tier","useWhen":"Open-source prompt management with a playground and versioning.","skipWhen":"You need eval depth rather than prompt ergonomics.","alternatives":["Humanloop","PromptLayer"],"alsoIn":[],"verified":"2026-09"},{"name":"Promptwatch","url":"https://lattice.kunjshah.in/prompt-engineering/promptwatch","homepage":"https://promptwatch.com","section":"prompt-engineering","layer":8,"band":"control","kind":"platform","roles":["production"],"deployment":"saas","license":"proprietary","language":null,"cost":"usage-based","useWhen":"Prompt regression tests and side-by-side comparison.","skipWhen":"You need self-hosting.","alternatives":["PromptLayer","Agenta","Humanloop"],"alsoIn":[],"verified":"2026-09"},{"name":"TextGrad","url":"https://lattice.kunjshah.in/prompt-engineering/textgrad","homepage":"https://github.com/zou-group/textgrad","section":"prompt-engineering","layer":8,"band":"control","kind":"library","roles":["applied"],"deployment":"self-hosted","license":"MIT","language":"Python","cost":"free","useWhen":"Automatic prompt optimisation via textual feedback loops.","skipWhen":"Optimising without an eval set is guessing.","alternatives":["DSPy"],"alsoIn":[],"verified":"2026-09"},{"name":"Outlines","url":"https://lattice.kunjshah.in/prompt-engineering/outlines","homepage":"https://dottxt-ai.github.io/outlines","section":"prompt-engineering","layer":8,"band":"control","kind":"library","roles":["applied"],"deployment":"self-hosted","license":"Apache-2.0","language":"Python","cost":"free","useWhen":"Token-level constrained generation from a JSON schema or grammar.","skipWhen":"Post-hoc validation is enough for your use case.","alternatives":["Guidance","Instructor"],"alsoIn":[],"verified":"2026-09"},{"name":"LangSmith","url":"https://lattice.kunjshah.in/evaluation-observability/langsmith","homepage":"https://smith.langchain.com","section":"evaluation-observability","layer":9,"band":"control","kind":"platform","roles":["production"],"deployment":"saas","license":"proprietary","language":null,"cost":"usage-based","useWhen":"Teams already deep in LangChain needing tracing and datasets.","skipWhen":"Traces cannot leave your infrastructure.","alternatives":["Langfuse","Braintrust","Arize Phoenix"],"alsoIn":[],"verified":"2026-09"},{"name":"Braintrust","url":"https://lattice.kunjshah.in/evaluation-observability/braintrust","homepage":"https://braintrust.dev","section":"evaluation-observability","layer":9,"band":"control","kind":"platform","roles":["production"],"deployment":"saas","license":"proprietary","language":null,"cost":"usage-based","useWhen":"Making evaluation, rather than tracing, the primary workflow.","skipWhen":"You need open formats and self-hosting.","alternatives":["LangSmith","Langfuse","DeepEval"],"alsoIn":[],"verified":"2026-09"},{"name":"Arize Phoenix","url":"https://lattice.kunjshah.in/evaluation-observability/arize-phoenix","homepage":"https://phoenix.arize.com","section":"evaluation-observability","layer":9,"band":"control","kind":"library","roles":["production"],"deployment":"self-hosted","license":"Apache-2.0","language":"Python","cost":"free","useWhen":"OpenTelemetry-native evaluation you can run yourself.","skipWhen":"You want a vendor support contract behind it.","alternatives":["Langfuse","Opik","Gentrace"],"alsoIn":[],"verified":"2026-09"},{"name":"Langfuse","url":"https://lattice.kunjshah.in/evaluation-observability/langfuse","homepage":"https://langfuse.com","section":"evaluation-observability","layer":9,"band":"control","kind":"platform","roles":["production"],"deployment":"self-hosted","license":"MIT","language":"TypeScript","cost":"free-tier","useWhen":"Self-hostable tracing, prompts and evals in one place.","skipWhen":"You want a fully managed product with a support contract.","alternatives":["LangSmith","Braintrust","Arize Phoenix","Gentrace","Opik"],"alsoIn":[{"section":"prompt-engineering","layer":8,"because":"Its prompt registry versions prompts and ties each version to the traces and scores it produced."}],"verified":"2026-09"},{"name":"promptfoo","url":"https://lattice.kunjshah.in/evaluation-observability/promptfoo","homepage":"https://promptfoo.dev","section":"evaluation-observability","layer":9,"band":"control","kind":"library","roles":["production"],"deployment":"self-hosted","license":"MIT","language":"TypeScript","cost":"free","useWhen":"Declarative red-teaming and regression tests that run in CI.","skipWhen":"You need a managed UI rather than a test runner.","alternatives":["DeepEval","garak"],"alsoIn":[],"verified":"2026-09"},{"name":"DeepEval","url":"https://lattice.kunjshah.in/evaluation-observability/deepeval","homepage":"https://deepeval.com","section":"evaluation-observability","layer":9,"band":"control","kind":"library","roles":["production"],"deployment":"self-hosted","license":"Apache-2.0","language":"Python","cost":"free","useWhen":"pytest-style evaluation you can run next to your unit tests.","skipWhen":"You need a platform rather than a library.","alternatives":["promptfoo","Braintrust"],"alsoIn":[],"verified":"2026-09"},{"name":"Helicone","url":"https://lattice.kunjshah.in/evaluation-observability/helicone","homepage":"https://helicone.ai","section":"evaluation-observability","layer":9,"band":"control","kind":"platform","roles":["production"],"deployment":"saas","license":"proprietary","language":null,"cost":"free-tier","useWhen":"Gateway-level observability with per-request cost and latency.","skipWhen":"You need eval primitives more than traffic logs.","alternatives":["Langfuse","Gentrace"],"alsoIn":[{"section":"routing-gateways","layer":2,"because":"It observes at the gateway, so the requests it reports on have already passed through layer 2."}],"verified":"2026-09"},{"name":"Weights & Biases","url":"https://lattice.kunjshah.in/evaluation-observability/weights-biases","homepage":"https://wandb.ai","section":"evaluation-observability","layer":9,"band":"control","kind":"platform","roles":["production"],"deployment":"saas","license":"proprietary","language":null,"cost":"usage-based","useWhen":"Experiment tracking and a model registry, with eval surfaces on top.","skipWhen":"You need an open, exportable format.","alternatives":["Weights & Biases Launch","LangSmith"],"alsoIn":[{"section":"fine-tuning","layer":4,"because":"The model registry and experiment ledger it is known for are training-time surfaces; this entry carries its eval half."},{"section":"workflow-orchestration","layer":6,"because":"Sweeps and artifact tracking are the orchestration of many training runs, which is the layer 6 job."}],"verified":"2026-09"},{"name":"OpenTelemetry","url":"https://lattice.kunjshah.in/evaluation-observability/opentelemetry","homepage":"https://opentelemetry.io","section":"evaluation-observability","layer":9,"band":"control","kind":"library","roles":["production"],"deployment":"self-hosted","license":"Apache-2.0","language":"multi","cost":"free","useWhen":"A vendor-neutral standard for emitting traces and metrics.","skipWhen":"You want a product with a user interface.","alternatives":[],"alsoIn":[],"verified":"2026-09"},{"name":"Vellum","url":"https://lattice.kunjshah.in/evaluation-observability/vellum","homepage":"https://vellum.ai","section":"evaluation-observability","layer":9,"band":"control","kind":"platform","roles":["production"],"deployment":"saas","license":"proprietary","language":null,"cost":"usage-based","useWhen":"Prompt and eval work with a visual debugger for chains.","skipWhen":"Self-hosting or reproducibility is a requirement.","alternatives":["Humanloop","Braintrust"],"alsoIn":[{"section":"prompt-engineering","layer":8,"because":"Prompts are versioned artifacts here, diffed and promoted rather than edited in place."}],"verified":"2026-09"},{"name":"Gentrace","url":"https://lattice.kunjshah.in/evaluation-observability/gentrace","homepage":"https://gentrace.ai","section":"evaluation-observability","layer":9,"band":"control","kind":"platform","roles":["production"],"deployment":"self-hosted","license":"Apache-2.0","language":"TypeScript","cost":"free-tier","useWhen":"Open-source tracing and dashboards for LLM applications.","skipWhen":"You want eval depth more than traffic visibility.","alternatives":["Arize Phoenix","Helicone","Opik"],"alsoIn":[],"verified":"2026-09"},{"name":"Opik","url":"https://lattice.kunjshah.in/evaluation-observability/opik","homepage":"https://comet.com","section":"evaluation-observability","layer":9,"band":"control","kind":"platform","roles":["production"],"deployment":"self-hosted","license":"Apache-2.0","language":"Python","cost":"free","useWhen":"Open-source observability and evaluation you run yourself.","skipWhen":"You need enterprise support behind it.","alternatives":["Arize Phoenix","Gentrace"],"alsoIn":[],"verified":"2026-09"},{"name":"Evidently AI","url":"https://lattice.kunjshah.in/evaluation-observability/evidently-ai","homepage":"https://evidentlyai.com","section":"evaluation-observability","layer":9,"band":"control","kind":"library","roles":["production","data"],"deployment":"self-hosted","license":"Apache-2.0","language":"Python","cost":"free","useWhen":"One evaluation model for both classical ML and LLM systems.","skipWhen":"You only need LLM-specific tracing.","alternatives":["DeepEval","promptfoo"],"alsoIn":[],"verified":"2026-09"},{"name":"Hugging Face","url":"https://lattice.kunjshah.in/learning-reference/hugging-face","homepage":"https://huggingface.co","section":"learning-reference","layer":null,"band":null,"kind":"platform","roles":["data","applied"],"deployment":"saas","license":"proprietary","language":null,"cost":"free-tier","useWhen":"The model hub, the datasets and the open-source library ecosystem.","skipWhen":"You need a self-hosted artefact registry.","alternatives":[],"alsoIn":[],"verified":"2026-09"},{"name":"Full Stack Deep Learning","url":"https://lattice.kunjshah.in/learning-reference/full-stack-deep-learning","homepage":"https://fullstackdeeplearning.com","section":"learning-reference","layer":null,"band":null,"kind":"reading","roles":["applied"],"deployment":null,"license":null,"language":null,"cost":"free","useWhen":"The practical side of shipping LLM systems, taught end to end.","skipWhen":"You want an API reference.","alternatives":[],"alsoIn":[],"verified":"2026-09"},{"name":"Sebastian Raschka","url":"https://lattice.kunjshah.in/learning-reference/sebastian-raschka","homepage":"https://magazine.sebastianraschka.com","section":"learning-reference","layer":null,"band":null,"kind":"reading","roles":["applied"],"deployment":null,"license":null,"language":null,"cost":"free","useWhen":"Essays and books that distil research into working knowledge.","skipWhen":"You need primary sources.","alternatives":[],"alsoIn":[],"verified":"2026-09"},{"name":"arXiv","url":"https://lattice.kunjshah.in/learning-reference/arxiv","homepage":"https://arxiv.org","section":"learning-reference","layer":null,"band":null,"kind":"reading","roles":["applied"],"deployment":null,"license":null,"language":null,"cost":"free","useWhen":"The primary preprint record for machine learning research.","skipWhen":"You need a curated and explained version.","alternatives":[],"alsoIn":[],"verified":"2026-09"},{"name":"Papers with Code","url":"https://lattice.kunjshah.in/learning-reference/papers-with-code","homepage":"https://paperswithcode.com","section":"learning-reference","layer":null,"band":null,"kind":"reading","roles":["applied"],"deployment":null,"license":null,"language":null,"cost":"free","useWhen":"Linking papers to their reference implementations.","skipWhen":"You are offline — it is a live index.","alternatives":[],"alsoIn":[],"verified":"2026-09"},{"name":"Jay Alammar","url":"https://lattice.kunjshah.in/learning-reference/jay-alammar","homepage":"https://jalammar.github.io","section":"learning-reference","layer":null,"band":null,"kind":"reading","roles":["applied","data"],"deployment":null,"license":null,"language":null,"cost":"free","useWhen":"Visual explanations of transformers, RAG and model internals.","skipWhen":"You want depth beyond the intuition.","alternatives":[],"alsoIn":[],"verified":"2026-09"},{"name":"AI Engineer","url":"https://lattice.kunjshah.in/learning-reference/ai-engineer","homepage":"https://ai.engineer","section":"learning-reference","layer":null,"band":null,"kind":"reading","roles":["applied"],"deployment":null,"license":null,"language":null,"cost":"free","useWhen":"A community and conference focused on shipping LLM products.","skipWhen":"You need technical reference rather than practice.","alternatives":[],"alsoIn":[],"verified":"2026-09"},{"name":"Hugging Face Cookbook","url":"https://lattice.kunjshah.in/learning-reference/hugging-face-cookbook","homepage":"https://huggingface.co","section":"learning-reference","layer":null,"band":null,"kind":"reading","roles":["applied"],"deployment":null,"license":null,"language":null,"cost":"free","useWhen":"Task-oriented notebooks for fine-tuning, serving and RAG.","skipWhen":"You need maintained library documentation.","alternatives":[],"alsoIn":[],"verified":"2026-09"},{"name":"Made With ML","url":"https://lattice.kunjshah.in/learning-reference/made-with-ml","homepage":"https://madewithml.com","section":"learning-reference","layer":null,"band":null,"kind":"reading","roles":["platform"],"deployment":null,"license":null,"language":null,"cost":"free","useWhen":"Designing, building and deploying ML systems as a discipline.","skipWhen":"You only need LLM-specific material.","alternatives":[],"alsoIn":[],"verified":"2026-09"},{"name":"Distill","url":"https://lattice.kunjshah.in/learning-reference/distill","homepage":"https://distill.pub","section":"learning-reference","layer":null,"band":null,"kind":"reading","roles":["production"],"deployment":null,"license":null,"language":null,"cost":"free","useWhen":"Carefully explained model and method write-ups — though the publication is now archived.","skipWhen":"You need recent work; the archive stopped years ago.","alternatives":[],"alsoIn":[],"verified":"2026-09"}]}