Lattice
Skip to content

11 essays

Notes on building AI systems that hold up.

The index tells you what exists. These essays cover the decisions behind it — what to reach for first, what looks like an optimisation and is actually a rewrite, and where the failure modes live.

  1. Choosing an inference runtime

    7 min    The wrong runtime costs you throughput you never knew you were leaving on the table. The right one is usually decided by traffic shape, not model size. Inference & Serving

  2. The gateway is the product

    8 min    The interesting engineering is not calling a model. It is deciding which model, failing over safely, and staying inside a budget while you do it. Routing & Gateways

  3. Fix the ranking, not the prompt

    9 min    When a RAG system gives a bad answer, the instinct is to rewrite the prompt. Nine times out of ten the prompt is fine and the context was never there. Retrieval & Vector Stores Evaluation & Observability

  4. Evals are the asset, not the logging

    8 min    Traces tell you what happened. Evals tell you whether it got better. Most teams build the first and mistake it for the second. Evaluation & Observability

  5. Agents need a durable host

    8 min    The loop is a weekend project. The part where it keeps working at three in the morning is the actual product, and almost nobody budgets for it. Workflow Orchestration Agent Frameworks

  6. Prompt, or fine-tune?

    7 min    Almost every 'the model does not know this' problem is a retrieval problem wearing a hat. Here is how to tell the difference before you spend a week on a training run. Fine-tuning & Training Prompt Engineering

  7. Where guardrails belong

    8 min    Safety is not a layer you add. It is a set of controls at specific points in a request's path, and each one catches a different class of failure. Guardrails & Safety

  8. Context is a budget, not a container

    7 min    The instinct is to keep everything in case it is needed. The models are not filing systems, and past a certain length the middle of your context is doing nothing but costing money. Inference & Serving Prompt Engineering

  9. Build a cost model before you optimise

    7 min    Teams optimise the thing they can see in the dashboard. The dashboard shows tokens. The number that matters is cost per successful outcome, and nobody is measuring it. Routing & Gateways Evaluation & Observability

  10. Choosing a model is a routing problem

    8 min    There is no best model. There is a best model for a task, at a quality bar, within a latency and cost budget — and that decision is a routing policy, not a benchmark. Routing & Gateways Inference & Serving

  11. Observability is not logging

    7 min    A log line with a status code is not a trace. The difference is whether it can reconstruct a decision after the model that made it has been replaced twice. Evaluation & Observability