Chunking
Splitting documents into retrievable pieces, which is where RAG quality is mostly won and lost.
By Lattice editorial · Entry last reviewed · method
What it implies
Fixed token count with overlap is close to the worst reasonable option: it cuts through sentences, tables and arguments. Chunk on structure — headings, paragraphs, table rows — and carry the heading path in the chunk metadata so retrieved text has its own context. Then check the number that correlates most with answer quality: how many chunks you actually feed the model. Fifty is almost always worse than five.
Tools in this index
Related terms
- EmbeddingA vector representation of text such that semantically similar inputs land near each other.
- RerankingRe-scoring retrieved candidates with a slower, more accurate model before the generator sees them.
- Context windowThe number of tokens a model can consider at once — a ceiling on what you can send, not a target for what you should.