Quiz: Module 7 - Fine-Tuning Fundamentals

Question 1

What does PEFT stand for?

1. Parameter-Efficient Fine-Tuning
2. Parallel Efficient Fine-Tuning
3. Pre-trained Efficient Fine-Tuning
4. Partial Embedding Fine-Tuning

Question 2

What percentage of parameters does LoRA typically train?

1. 0.1-1%
2. 10-20%
3. 50%
4. 100%

Question 3

What is catastrophic forgetting?

1. Model losing pre-trained knowledge during fine-tuning
2. Memory overflow during training
3. Forgetting to save checkpoints
4. Loss function going to zero

Question 4

What does the "r" parameter in LoRA represent?

1. The rank of the low-rank matrices
2. Learning rate
3. Regularization strength
4. Number of repetitions

Question 5

What is QLoRA?

1. LoRA with quantized base model
2. Quantized LoRA weights
3. Quick LoRA training
4. Quality-aware LoRA

Question 6

What is instruction tuning?

1. Training models to follow natural language instructions
2. Tuning hyperparameters
3. Writing better prompts
4. Compiling training instructions

Question 7

Which layers are typically targeted by LoRA?

1. Attention projection layers (q_proj, v_proj)
2. Only the embedding layer
3. Only the output layer
4. All layers equally

Question 8

What is a typical learning rate for fine-tuning LLMs?

1. 1e-5 to 5e-5
2. 0.1
3. 0.01
4. 1e-2

Question 9

What is the Alpaca dataset?

1. 52K instruction-following examples generated by GPT-4
2. A image dataset
3. A code dataset
4. A translation dataset

Question 10

Why use gradient accumulation?

1. To simulate larger batch sizes with limited memory
2. To speed up training
3. To improve accuracy
4. To reduce overfitting

Questions 11-30

(Covering: adapter methods, prompt tuning, prefix tuning, full vs PEFT tradeoffs, dataset quality, evaluation metrics, hyperparameter tuning, multi-task fine-tuning, domain adaptation, continual learning)

Question 11

What is lora_alpha used for?

1. Scaling factor for LoRA updates
2. Learning rate
3. Dropout rate
4. Weight decay

Question 12

What VRAM is typically needed for 7B model full fine-tuning (FP16)?

1. ~56GB
2. 16GB
3. 8GB
4. 128GB

Question 13

What is prefix tuning?

1. Adding trainable prefix tokens to inputs
2. Training on text prefixes
3. Modifying tokenizer
4. Pre-processing data

Question 14

What makes instruction data high quality?

1. Diverse tasks, accurate responses, consistent format
2. Large quantity only
3. Long responses
4. Simple instructions

Question 15

What is the purpose of warmup steps?

1. Gradually increase learning rate to prevent training instability
2. Load model into memory
3. Validate the dataset
4. Compile the model

Question 16

When is full fine-tuning preferred over LoRA?

1. When maximum performance is needed and compute is available
2. Always
3. For quick experiments
4. When memory is limited

Question 17

What is the benefit of LoRA over full fine-tuning?

1. Much lower memory requirements
2. Always better quality
3. Faster inference
4. Simpler code

Question 18

What is adapter tuning?

1. Adding small trainable modules between frozen layers
2. Adapting input format
3. Converting model format
4. Hardware adaptation

Question 19

How can you merge LoRA weights into base model?

1. model.merge_and_unload()
2. model.combine()
3. Not possible
4. Requires retraining

Question 20

What is the Dolly dataset known for?

1. Human-written instruction data (15K examples)
2. Largest dataset
3. Code only
4. Synthetic data

Question 21

What does target_modules specify in LoRA config?

1. Which model layers to apply LoRA to
2. Output format
3. Training objectives
4. Evaluation metrics

Question 22

What is multi-task fine-tuning?

1. Training on multiple tasks simultaneously
2. Running multiple training jobs
3. Using multiple GPUs
4. Sequential task training

Question 23

What is the effect of higher LoRA rank (r)?

1. More parameters, potentially better quality
2. Faster training
3. Less memory
4. Simpler model

Question 24

Why use evaluation dataset during fine-tuning?

1. Monitor overfitting and generalization
2. Increase training data
3. Speed up training
4. Reduce memory

Question 25

What is the FLAN collection?

1. Large collection of academic task datasets for instruction tuning
2. A model architecture
3. A tokenizer
4. A fine-tuning library

Question 26

What library provides PEFT methods?

1. peft (by Hugging Face)
2. pytorch only
3. tensorflow only
4. numpy

Question 27

What is lora_dropout?

1. Dropout applied to LoRA layers for regularization
2. Removing LoRA layers
3. Reducing rank
4. Early stopping

Question 28

When might you need domain-specific fine-tuning?

1. Legal, medical, or technical domains with specialized vocabulary
2. General chatbot
3. Simple Q&A
4. Translation

Question 29

What is continual learning in LLMs?

1. Updating model on new data without forgetting old knowledge
2. Long training runs
3. Continuous inference
4. Real-time learning

Question 30

What's a good strategy to prevent overfitting during fine-tuning?

1. Early stopping based on validation loss
2. Train longer
3. Increase learning rate
4. Remove validation set

---

Passing Score: 24/30 (80%)

← Chapter 3Module 8 →