Distillation
Training a smaller model against the outputs of a larger one, to get most of the behaviour at a fraction of the cost.
By Lattice editorial · Entry last reviewed · method
What it implies
The best-understood reason to fine-tune, because you have a reference and the training signal is mechanical. The result is a model slightly worse at the edges and dramatically cheaper, which for high-volume well-defined tasks is usually the right trade. Note this is different from quantisation, which changes no behaviour at all.
Tools in this index
Related terms
- QuantisationStoring and computing model weights in fewer bits than they were trained in, to trade accuracy for memory and speed.
- Supervised fine-tuningTraining on curated input/output pairs so the model adopts a specific task or response style.
- Cost per successful outcomeTotal spend divided by the count of requests that passed whatever quality bar you can define.