Start here
Your first working RAG app, in five steps.
For people who have never shipped one. Pick the tool at each layer, in order, and stop when you have something that answers a question.
The path
- 01
Pick a model you can call
Use a hosted API to start (OpenAI, Anthropic, or a free tier). Self-hosting is a later decision; it is not step one.
Tool: OpenRouter
- 02
Store your documents somewhere you can query
Start with pgvector if you already run Postgres; it is one less system to operate. Move to a dedicated store only when retrieval becomes the workload.
Tool: pgvector
- 03
Chunk and embed, then check recall
Chunk by structure, not by token count, and measure recall against ten real questions before adding more documents.
Tool: Qdrant
- 04
Put a gateway in front
One proxy for auth, logging and cost attribution. It is the only layer that sees the whole request.
Tool: LiteLLM
- 05
Add an eval before you ship
A small set of labelled examples and a score. Without it you cannot tell whether a change helped.
Tool: promptfoo
After step five
You have a working app and a measurement. From here the index earns its keep: compare alternatives, read the symptom checklists when something breaks, and use the Stack Builder when the shape of the workload changes.