Reranking
Re-scoring retrieved candidates with a slower, more accurate model before the generator sees them.
By Lattice editorial · Entry last reviewed · method
What it implies
Bi-encoders embed query and document independently — that is what makes them fast enough to search millions of vectors, and also why they judge relevance poorly. A cross-encoder sees both at once and is far better, but far too slow for the whole corpus. So: cheap retrieval for fifty candidates, cross-encoder for the best five. The highest-leverage addition to an existing RAG system, and the cheapest to try.