Chapter 1: Introduction to PEFT

Making Fine-Tuning Accessible

Parameter-Efficient Fine-Tuning (PEFT) is a collection of techniques that enable fine-tuning of large language models with a fraction of the resources required for full fine-tuning.

---

The Problem PEFT Solves

Full Fine-Tuning Challenges

• 7B model: Needs ~60-80GB VRAM for full fine-tuning
• 13B model: Needs ~110-160GB VRAM
• 70B model: Needs ~600GB+ VRAM

This locks out most practitioners from fine-tuning larger models.

---

The PEFT Solution

Key Insight

Most of the model's knowledge is in the pre-trained weights. We only need to make small adjustments for new tasks.

How It Works

Instead of updating all parameters:

# Full Fine-Tuning (all parameters)
for param in model.parameters():
    param.requires_grad = True  # Update everything

# PEFT (only adapters)
for name, param in model.named_parameters():
    param.requires_grad = False  # Freeze base model
    
# Add small trainable adapter
adapter = LoRAAdapter(rank=16)  # ~0.1% of parameters

---

PEFT Benefits

1. Dramatic Memory Reduction

Model	Full FT	PEFT	Savings
7B	~80GB	~6-14GB	82-92%
13B	~160GB	~10-26GB	84-94%
70B	~600GB	~48-140GB	77-92%

2. Faster Training

Full FT: 100% of parameters updated per step
PEFT: Less than 1% of parameters updated per step
→ Faster gradient computation
→ Faster weight updates
→ Lower memory bandwidth

3. No Catastrophic Forgetting

# Base model frozen = original knowledge preserved
base_weights = model.get_base_weights()  # Never modified
adapter_weights = model.get_adapter_weights()  # Only these change

4. Multiple Adapters

# One base model, multiple adapters for different tasks
model.load_adapter("customer-support")  # Task 1
model.load_adapter("code-generation")   # Task 2
model.load_adapter("summarization")     # Task 3
# Just swap adapters at runtime!

---

Core Concepts

Frozen Parameters

Parameters that don't receive gradient updates during training.

# Freeze a parameter
param.requires_grad = False

# Check trainable parameters
for name, param in model.named_parameters():
    if param.requires_grad:
        print(f"Trainable: {name}")

Trainable Parameters

Only the small set of parameters that are updated during PEFT training.

Adapters

Additional modules added to the frozen model that contain the trainable parameters.

---

The PEFT Library

Hugging Face's PEFT library provides unified access to all major PEFT methods:

from peft import (
    LoraConfig,
    PrefixTuningConfig,
    PromptTuningConfig,
    IA3Config,
    AdaLoraConfig,
    get_peft_model,
    PeftModel,
)

# Configure PEFT
peft_config = LoraConfig(
    r=16,
    lora_alpha=32,
    target_modules=["q_proj", "v_proj"],
)

# Apply to model
model = get_peft_model(base_model, peft_config)

# Check trainable parameters
model.print_trainable_parameters()
# Output: trainable params: 4,194,304 || all params: 6,742,609,920 || trainable%: 0.0622

Key Functions

Function	Purpose
get_peft_model()	Wrap base model with PEFT
print_trainable_parameters()	Show trainable count
save_pretrained()	Save adapter only
merge_and_unload()	Merge adapter into base
load_adapter()	Load a saved adapter

---

Parameter Comparison

from transformers import AutoModelForCausalLM
from peft import LoraConfig, get_peft_model

# Load base model
model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.1-8B")

# Count base parameters
total_params = sum(p.numel() for p in model.parameters())
print(f"Total parameters: {total_params:,}")  # ~8,000,000,000

# Apply LoRA
lora_config = LoraConfig(r=16, lora_alpha=32, target_modules=["q_proj", "v_proj"])
model = get_peft_model(model, lora_config)

# Count trainable
trainable = sum(p.numel() for p in model.parameters() if p.requires_grad)
print(f"Trainable: {trainable:,}")  # ~4,000,000
print(f"Percentage: {100 * trainable / total_params:.4f}%")  # ~0.05%

---

Storage Benefits

Adapter-Only Saving

# Full model: ~15GB (7B model in fp16)
model.save_pretrained("./full-model")

# LoRA adapter only: ~10-50MB
model.save_pretrained("./lora-adapter", save_embedding_layers=False)

Multiple Adapters = Minimal Storage

Base Model:     15GB (shared)
Adapter A:      20MB (task 1)
Adapter B:      20MB (task 2)
Adapter C:      20MB (task 3)
Total:          15.06GB for 3 specialized models

vs Full FT:     45GB for 3 separate models

---

Quick Example

Complete PEFT training in under 50 lines:

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import LoraConfig, get_peft_model
from trl import SFTTrainer
from datasets import load_dataset

# Load model and tokenizer
model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.1-8B")
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-3.1-8B")
tokenizer.pad_token = tokenizer.eos_token

# Configure LoRA
lora_config = LoraConfig(
    r=16,
    lora_alpha=32,
    target_modules=["q_proj", "v_proj", "k_proj", "o_proj"],
    lora_dropout=0.05,
    task_type="CAUSAL_LM",
)

# Apply PEFT
model = get_peft_model(model, lora_config)
model.print_trainable_parameters()

# Load dataset
dataset = load_dataset("databricks/dolly-15k", split="train[:1000]")

# Train
trainer = SFTTrainer(
    model=model,
    train_dataset=dataset,
    dataset_text_field="text",
    max_seq_length=512,
)
trainer.train()

# Save adapter
model.save_pretrained("./my-lora-adapter")

---

Key Takeaways

✅ PEFT trains less than 1% of parameters while maintaining quality

✅ Memory savings of 80-95% compared to full fine-tuning

✅ Base model weights stay frozen, preventing forgetting

✅ Multiple adapters can share one base model

✅ The PEFT library provides unified access to all methods

✅ Adapter files are small (~10-50MB) vs full models (~15GB+)

---

Next Steps

Continue to Chapter 2: PEFT vs Full Fine-Tuning to understand the trade-offs in detail.

---

Quick Check

Before moving on, make sure you can answer:

1. What percentage of parameters does PEFT typically train?
2. Why doesn't PEFT cause catastrophic forgetting?
3. What is the main function to create a PEFT model?