LoRA
Training small low-rank matrices alongside frozen weights, so you can adapt a large model without touching it.
By Lattice editorial · Entry last reviewed · method
What it implies
Adapter weights are a few percent of the base model, so training needs a fraction of the memory. QLoRA additionally quantises the frozen base to 4-bit, which brings single-GPU adaptation of models that previously needed several. The practical cost is inference: adapters are only useful if someone is serving them.
Tools in this index
Related terms
- QuantisationStoring and computing model weights in fewer bits than they were trained in, to trade accuracy for memory and speed.
- DistillationTraining a smaller model against the outputs of a larger one, to get most of the behaviour at a fraction of the cost.
- Supervised fine-tuningTraining on curated input/output pairs so the model adopts a specific task or response style.