Chapter 2: PEFT vs Full Fine-Tuning

Understanding the Trade-offs

Choosing between PEFT and full fine-tuning requires understanding their respective strengths and limitations.

---

Comprehensive Comparison

Aspect	Full Fine-Tuning	PEFT
Memory	4-6x model size	0.1-0.5x model size
Speed	Slower	2-3x faster
Quality Ceiling	Highest	Slightly lower
Forgetting Risk	High	None
Hardware	Expensive GPUs	Consumer GPUs
Adapter Swapping	No	Yes
Inference Overhead	None	Minimal (or merge)

---

Memory Comparison

Detailed Breakdown

def calculate_memory(params_billions, method):
    """Calculate approximate training memory."""
    params = params_billions * 1e9
    
    if method == "full_ft":
        # Weights + Gradients + Optimizer (2 states) + Activations
        weights = params * 2  # bf16
        gradients = params * 2
        optimizer = params * 4 * 2  # fp32, 2 states
        activations = params * 0.5  # Rough estimate
        return (weights + gradients + optimizer + activations) / 1e9
    
    elif method == "lora":
        # Frozen weights + small LoRA gradients/optimizer
        weights = params * 2
        lora_params = params * 0.01  # ~1% trainable
        lora_training = lora_params * (2 + 2 + 8)  # grad + optimizer
        return (weights + lora_training) / 1e9
    
    elif method == "qlora":
        # 4-bit weights + LoRA training
        weights = params * 0.5  # 4-bit
        lora_params = params * 0.01
        lora_training = lora_params * (2 + 2 + 8)
        return (weights + lora_training) / 1e9

# Results for 7B model
print(f"Full FT: {calculate_memory(7, 'full_ft'):.1f} GB")   # ~77 GB
print(f"LoRA: {calculate_memory(7, 'lora'):.1f} GB")         # ~15 GB
print(f"QLoRA: {calculate_memory(7, 'qlora'):.1f} GB")       # ~4 GB

---

Quality Comparison

Empirical Results

Research consistently shows PEFT achieves comparable quality:

Task	Full FT	LoRA	Difference
Text Classification	94.2%	93.8%	-0.4%
Summarization	45.3 ROUGE	44.9 ROUGE	-0.9%
Instruction Following	8.2/10	8.0/10	-2.4%
Code Generation	67.1%	65.8%	-1.9%

When PEFT Falls Short

PEFT may underperform for:

• Significant domain shifts: Pre-training → very specialized domain
• New output formats: Very different from any training data
• Maximum possible quality: When every 0.1% matters
• Complex new behaviors: Deeply embedded patterns

# Example: PEFT might struggle with
# 1. Medical model from general LLM (major domain shift)
# 2. Entirely new language not in pretraining
# 3. Competition benchmarks where every point counts

---

Training Speed Comparison

Throughput Metrics

Method	Time per Epoch	Relative Speed
Full FT	60 min	1x
LoRA	25 min	2.4x
QLoRA	30 min	2x

Why PEFT is Faster

1. Fewer gradient computations: Only compute gradients for trainable params
2. Less memory bandwidth: Fewer bytes to move
3. Smaller optimizer updates: Fewer states to maintain

# Gradient computation scales with trainable parameters
# Full FT: 7 billion gradient computations
# LoRA: 70 million gradient computations (100x fewer)

---

Flexibility Comparison

PEFT: Multiple Adapters

# Load base model once
base_model = AutoModelForCausalLM.from_pretrained("llama-3.1-8b")

# Create different adapters
adapters = {
    "customer_support": "adapters/support/",
    "code_review": "adapters/code/",
    "translation": "adapters/translate/",
}

# Runtime adapter switching
for task, adapter_path in adapters.items():
    model = PeftModel.from_pretrained(base_model, adapter_path)
    # Use model for task
    model.unload()  # Remove adapter

Full FT: Separate Models

# Each task requires loading a different full model
models = {
    "customer_support": "models/support-full/",     # 15GB
    "code_review": "models/code-full/",             # 15GB
    "translation": "models/translate-full/",        # 15GB
}
# Total: 45GB storage, high switching cost

---

Inference Comparison

PEFT with Adapters

# Slight overhead from adapter computation
# Latency: ~5-10% slower per token

# Option 1: Keep adapter separate
model = PeftModel.from_pretrained(base, adapter)
output = model.generate(input_ids)

# Option 2: Merge for zero overhead
model = model.merge_and_unload()  # No adapter overhead
output = model.generate(input_ids)

Merged PEFT = Full FT Speed

After merging:

W_merged = W_base + B × A  # One-time computation
# Inference is identical to full fine-tuned model

---

Cost Analysis

Training Costs

Method	GPU Needed	Time	Cost (On-Demand)
Full FT (7B)	A100-80GB	4 hr	$24
LoRA (7B)	A100-40GB	1.5 hr	$6
QLoRA (7B)	RTX 4090	2 hr	$2

Long-Term Considerations

Scenario: Fine-tune for 10 different tasks

Full FT:
- 10 training runs on expensive GPUs
- 10 full models to store (150GB)
- Loading different model per task

PEFT:
- 10 training runs on cheaper GPUs
- 1 base model + 10 adapters (15GB + 200MB)
- Quick adapter switching

---

Decision Framework

Choose PEFT When:

✅ GPU memory is limited (< 40GB)
✅ Need to experiment quickly
✅ Multiple task variations needed
✅ Want to preserve base model capabilities
✅ Cost is a significant factor
✅ Production requires adapter switching

Choose Full FT When:

✅ Maximum quality is critical
✅ Significant domain shift required
✅ Resources are not a constraint
✅ Single-purpose model needed
✅ Deploying as merged model anyway
✅ Research requires full control

---

Hybrid Strategies

Start with PEFT, Upgrade if Needed

# Phase 1: Quick experimentation with LoRA
lora_model = train_with_lora(dataset_small)
evaluate(lora_model)  # Check if quality is sufficient

# Phase 2: If PEFT isn't enough, try full FT
if quality_gap > threshold:
    full_ft_model = train_full(dataset_full)

PEFT for Iteration, Full FT for Final

# Iterate on data/hyperparameters with PEFT (fast, cheap)
for experiment in experiments:
    model = train_lora(config=experiment)
    score = evaluate(model)
    if score > best_score:
        best_config = experiment

# Final training with full FT using best settings
final_model = train_full(config=best_config)

---

Practical Recommendations

For Most Use Cases: Start with PEFT

# Default recommendation for most practitioners
from peft import LoraConfig, get_peft_model

config = LoraConfig(
    r=16,
    lora_alpha=32,
    target_modules=["q_proj", "v_proj", "k_proj", "o_proj"],
    lora_dropout=0.05,
)

model = get_peft_model(base_model, config)
# Train and evaluate

For Production: Consider Merging

# After training, merge for deployment
model = model.merge_and_unload()
model.save_pretrained("./production-model")
# Now behaves exactly like full fine-tuned model

---

Key Takeaways

✅ PEFT achieves 90-99% of full FT quality with 80-95% less memory

✅ Full FT has higher quality ceiling but at significant cost

✅ PEFT enables adapter switching; full FT creates fixed models

✅ Merged PEFT has identical inference performance to full FT

✅ Start with PEFT, upgrade to full FT only if needed

✅ Consider total cost including storage, switching, and iteration

---

Next Steps

Continue to Chapter 3: PEFT Methods Overview to explore all available PEFT techniques.

---

Quick Check

Before moving on, make sure you can answer:

1. How much memory does PEFT save compared to full fine-tuning?
2. When might full fine-tuning outperform PEFT?
3. What is the advantage of merged PEFT adapters?