QLoRA
Post-training & alignment · 2023
The idea
Section titled “The idea”LoRA still needs the full frozen model resident in memory at full precision. QLoRA quantizes it to 4 bits first - a new NF4 data type built for normally-distributed weights, plus double quantization and paged optimizers to absorb memory spikes - and trains the same thin LoRA adapters on top.
Verdict
Section titled “Verdict”What it fixedBackpropagate through a frozen 4-bit quantized model into LoRA adapters - a 65B model fine-tunes on one consumer GPU.
What it still costsThe extra quantize/dequantize step costs real training-time compute; it trades GPU memory for GPU time.
Where it sits
Section titled “Where it sits”Worth having first: LoRA · Quantization