Chapter 2: Introduction to PEFT

What is PEFT?

Parameter-Efficient Fine-Tuning: Update only a small subset of parameters.

LoRA (Low-Rank Adaptation)

Add low-rank matrices to existing weights:

$$W' = W + BA$$

Where $B \in \mathbb{R}^{d \times r}$, $A \in \mathbb{R}^{r \times k}$, and $r \ll \min(d, k)$

from peft import LoraConfig, get_peft_model

lora_config = LoraConfig(
    r=16,                      # Rank
    lora_alpha=32,             # Scaling
    target_modules=["q_proj", "v_proj"],
    lora_dropout=0.05,
)

model = get_peft_model(base_model, lora_config)
model.print_trainable_parameters()
# trainable params: 4,194,304 || all params: 6,742,609,920 || 0.06%

QLoRA

LoRA + Quantized base model:

from transformers import BitsAndBytesConfig

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
)

model = AutoModelForCausalLM.from_pretrained(
    "meta-llama/Llama-2-7b-hf",
    quantization_config=bnb_config,
)
model = get_peft_model(model, lora_config)

Comparison

Method	Trainable Params	Memory	Quality
Full	100%	High	Best
LoRA	0.1-1%	Low	Near-full
QLoRA	0.1-1%	Very Low	Good

---

← Chapter 1Chapter 3 →