Record 082 · Tooling · 2021
67%21.82.F
LoRA
Microsoft / collaborators
“Low-rank adapters that made fine-tuning a local 4090 job instead of a cluster job.”
LoRA (and QLoRA after it) is why people own specialized local models: a 20MB adapter on a frozen 7B. The customization layer.
Opened local specialization of large models.
- Low-rank adapters
- Small trainable footprint
- Composable adapters
A small adapter, a new behavior.
