Chapter 1: Traditional Fine-Tuning Approach

Updating All Model Parameters

Traditional fine-tuning involves updating every trainable parameter in the model during training. This provides the maximum flexibility for adapting models to new tasks.

---

How Full Fine-Tuning Works

The Training Loop

1. Forward Pass: Data flows through the entire model
2. Loss Calculation: Compare prediction to target
3. Backward Pass: Compute gradients for all parameters
4. Weight Update: Apply gradients to update all weights
5. Repeat: Continue until convergence

---

Complete Training Example

import torch
from transformers import (
    AutoModelForCausalLM,
    AutoTokenizer,
    TrainingArguments,
    Trainer,
    DataCollatorForLanguageModeling,
)
from datasets import load_dataset

# Load model and tokenizer
model_name = "meta-llama/Llama-3.1-8B"
tokenizer = AutoTokenizer.from_pretrained(model_name)
tokenizer.pad_token = tokenizer.eos_token

# Load model - all parameters trainable by default
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.bfloat16,  # Use bf16 for memory efficiency
    device_map="auto",
)

# Verify all parameters are trainable
trainable_params = sum(p.numel() for p in model.parameters() if p.requires_grad)
total_params = sum(p.numel() for p in model.parameters())
print(f"Trainable: {trainable_params:,} / {total_params:,} = 100%")

# Prepare dataset
dataset = load_dataset("databricks/dolly-15k", split="train[:1000]")

def format_instruction(example):
    return {
        "text": f"### Instruction:\n{example['instruction']}\n\n### Response:\n{example['response']}"
    }

dataset = dataset.map(format_instruction)

def tokenize(example):
    return tokenizer(
        example["text"],
        truncation=True,
        max_length=512,
        padding="max_length",
    )

tokenized_dataset = dataset.map(tokenize, remove_columns=dataset.column_names)

# Training configuration
training_args = TrainingArguments(
    output_dir="./llama-full-ft",
    num_train_epochs=3,
    per_device_train_batch_size=1,  # Limited by memory
    gradient_accumulation_steps=16,  # Simulate batch size of 16
    learning_rate=2e-5,
    weight_decay=0.01,
    warmup_ratio=0.03,
    logging_steps=10,
    save_strategy="epoch",
    bf16=True,  # Use bfloat16
    gradient_checkpointing=True,  # Save memory
    optim="adamw_torch_fused",  # Efficient optimizer
)

# Create trainer
trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=tokenized_dataset,
    data_collator=DataCollatorForLanguageModeling(tokenizer, mlm=False),
)

# Train
trainer.train()

# Save
model.save_pretrained("./llama-full-ft-final")
tokenizer.save_pretrained("./llama-full-ft-final")

---

Key Components Explained

Learning Rate

learning_rate = 2e-5  # Common range: 1e-5 to 5e-5

For full fine-tuning, use lower learning rates than PEFT:

• Too high: Model diverges or forgets pre-training
• Too low: Training is slow, may not adapt sufficiently

Weight Decay

weight_decay = 0.01  # Regularization to prevent overfitting

Weight decay prevents the model from overfitting by penalizing large weights.

Warmup

warmup_ratio = 0.03  # 3% of training steps

Gradually increase learning rate at the start for stable training.

Gradient Checkpointing

gradient_checkpointing = True  # Trade compute for memory

Recompute activations during backward pass instead of storing them—saves ~30% memory.

---

Optimizer Choices

Optimizer	Memory	Speed	Quality
AdamW	High	Fast	Best
Adam 8-bit	Medium	Medium	Good
SGD	Low	Slow	Requires tuning
Adafactor	Low	Medium	Good

# Standard AdamW
optim = "adamw_torch"

# Fused AdamW (faster)
optim = "adamw_torch_fused"

# 8-bit Adam (less memory)
optim = "adamw_8bit"

# Adafactor (no momentum states)
optim = "adafactor"

---

Training Dynamics

Loss Curve

Epoch 1: Training loss decreasing rapidly
         (Model adapting to new data)

Epoch 2: Training loss stabilizing
         (Watch validation loss for overfitting)

Epoch 3: Possible overfitting
         (Training loss still drops, validation may increase)

Monitoring Training

# Using Weights & Biases
training_args = TrainingArguments(
    # ... other args ...
    report_to="wandb",
    run_name="llama-full-ft-experiment-1",
)

Key metrics to watch:

• Training Loss: Should decrease
• Validation Loss: Should decrease but watch for divergence from training loss
• Gradient Norm: Spikes may indicate instability
• Learning Rate: Verify warmup and decay working

---

Mixed Precision Training

Why Use Mixed Precision?

# Half the memory, often faster training
bf16 = True  # Preferred for newer GPUs (Ampere+)
# OR
fp16 = True  # For older GPUs

Precision	Memory	Range	Stability
float32	4 bytes	Full	Highest
bfloat16	2 bytes	Full	High
float16	2 bytes	Limited	Medium

Float16 Stability

Float16 has a narrower range and can cause overflow/underflow. BFloat16 maintains float32's range with reduced precision, making it more stable.

---

Saving and Loading

During Training (Checkpoints)

training_args = TrainingArguments(
    save_strategy="steps",
    save_steps=500,
    save_total_limit=3,  # Keep only last 3 checkpoints
)

After Training

# Save the full model
model.save_pretrained("./final-model")
tokenizer.save_pretrained("./final-model")

# Later, load it
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("./final-model")

---

Advantages of Full Fine-Tuning

Maximum Customization

All parameters can be adjusted to your task.

Potentially Highest Quality

No constraints from frozen weights.

Simplicity

No adapter configuration needed.

Merged Output

Single model file, no adapter overhead at inference.

---

Disadvantages

High Memory Requirements

Need to store model, gradients, and optimizer states.

Catastrophic Forgetting Risk

Can lose pre-trained knowledge.

Expensive

Requires significant GPU resources.

Slow Training

Every parameter must be updated.

---

When to Use Full Fine-Tuning

Use Case	Recommendation
Maximum quality needed	✅ Full FT
Resource-constrained	❌ Use PEFT
Quick experimentation	❌ Use PEFT
Production at scale	⚠️ Consider trade-offs
Research/benchmarking	✅ Full FT

---

Key Takeaways

✅ Full fine-tuning updates all model parameters

✅ Requires significant GPU memory (model + gradients + optimizer)

✅ Use mixed precision and gradient checkpointing to reduce memory

✅ Lower learning rates than PEFT (1e-5 to 5e-5)

✅ Monitor for overfitting with validation loss

✅ Best for maximum customization when resources allow

---

Next Steps

Continue to Chapter 2: Computational Requirements to understand the exact memory and compute needs.

---

Quick Check

Before moving on, make sure you can answer:

1. What are the four main components stored in GPU memory during training?
2. Why is bfloat16 often preferred over float16?
3. What learning rate range is typical for full fine-tuning?