Semantic cache
Reusing a previous response when a new request is sufficiently similar in meaning, skipping the model call entirely.
By Lattice editorial · Entry last reviewed · method
What it implies
The fastest cost reduction available, and the easiest to get wrong: a cached answer goes stale in a way nobody notices. Scope it by embedding similarity *and* a version of everything that affected the answer, and never cache calls with side effects.