File 002 · Local Intelligence
Record 082 · Tooling · 2021
21.82.FLoRA
Microsoft / collaborators
“Low-rank adapters that made fine-tuning a local 4090 job instead of a cluster job.”
A small adapter, a new behavior.
LoRA (and QLoRA after it) is why people own specialized local models: a 20MB adapter on a frozen 7B. The customization layer.
Opened local specialization of large models.
Filed notes
- Low-rank adapters
- Small trainable footprint
- Composable adapters
SIC
7372
Prepackaged Software
NAICS
511210
Software Publishers
