Final Exam: LLM Fine-Tuning Complete
Comprehensive assessment covering all 10 modules. 50 questions to test your mastery of LLM fine-tuning concepts and practices.
Time Limit: 90 minutes
Passing Score: 80% (40/50 correct)
Section 1: Fundamentals (Questions 1-10)
Question 1
What is the primary difference between fine-tuning and prompt engineering?
A) Prompt engineering is faster to iterate
B) Fine-tuning modifies the model's internal weights
C) Prompt engineering produces better results
D) Fine-tuning doesn't require any data
Show Answer
Correct Answer: B
Fine-tuning modifies the model's internal weights through gradient-based training, while prompt engineering only changes the input text without altering the model itself.
Question 2
When should you prefer RAG over fine-tuning?
A) When you need to change the model's writing style
B) When knowledge needs to be updated frequently
C) When you want the model to follow specific instructions
D) When compute resources are unlimited
Show Answer
Correct Answer: B
RAG (Retrieval-Augmented Generation) is preferred when knowledge needs frequent updates because you can simply update the retrieval database without retraining the model.
Question 3
What is transfer learning in the context of LLMs?
A) Moving a model from one GPU to another
B) Using knowledge from pre-training to improve performance on new tasks
C) Transferring data between datasets
D) Converting models between frameworks
Show Answer
Correct Answer: B
Transfer learning leverages the knowledge a model gained during pre-training on large datasets to improve its performance on specific downstream tasks through fine-tuning.
Question 4
What is catastrophic forgetting?
A) When a model becomes too large to fit in memory
B) When training data is lost
C) When a model loses previously learned knowledge after training on new data
D) When gradients become zero
Show Answer
Correct Answer: C
Catastrophic forgetting occurs when a model's performance on previously learned tasks degrades after being trained on new data, essentially "forgetting" what it learned before.
Question 5
Which scenario is BEST suited for fine-tuning rather than prompting?
A) Asking a model to summarize a document
B) Teaching a model a specific company's writing style
C) Getting a model to answer questions about current events
D) Having a model translate a sentence
Show Answer
Correct Answer: B
Fine-tuning is best for internalizing specific styles, formats, or behaviors that are difficult to convey through prompts alone. A company's writing style is a learned behavior that benefits from fine-tuning.
Question 6
What is the main advantage of PEFT over full fine-tuning?
A) Always produces better quality results
B) Requires significantly less memory and compute
C) Works with any model architecture
D) Doesn't require any training data
Show Answer
Correct Answer: B
PEFT methods only train a small number of parameters, dramatically reducing memory requirements and training time while often maintaining comparable quality to full fine-tuning.
Question 7
What does "trainable parameters" refer to in the context of PEFT?
A) All parameters in the model
B) Only the parameters that are updated during training
C) Parameters that require no gradient computation
D) The input embeddings only
Show Answer
Correct Answer: B
Trainable parameters are specifically those weights that receive gradient updates during training. In PEFT, this is a small subset of the total model parameters.
Question 8
What is the typical percentage of trainable parameters in LoRA compared to full fine-tuning?
A) About 50%
B) About 25%
C) Less than 1%
D) About 10%
Show Answer
Correct Answer: C
LoRA typically trains less than 1% of the original model's parameters (often 0.1-0.5%) while maintaining comparable performance to full fine-tuning.
Question 9
What determines the "capacity" of a LoRA adapter?
A) The size of the base model
B) The rank (r) of the low-rank matrices
C) The learning rate used
D) The dataset size
Show Answer
Correct Answer: B
The rank (r) determines how many parameters the LoRA adapter has and thus its capacity to learn. Higher ranks allow more complex adaptations but require more parameters.
Question 10
What is the purpose of the LoRA alpha parameter?
A) To control learning rate
B) To scale the contribution of the LoRA weights
C) To set the number of training epochs
D) To determine batch size
Show Answer
Correct Answer: B
LoRA alpha is a scaling factor that controls how much the learned LoRA weights contribute to the output. The effective scaling is alpha/rank.
Section 2: Technical Implementation (Questions 11-25)
Question 11
What quantization format does QLoRA primarily use?
A) INT8
B) Float16
C) NF4 (4-bit NormalFloat)
D) BFloat16
Show Answer
Correct Answer: C
QLoRA uses NF4 (4-bit NormalFloat) quantization, which is specifically designed to have equal numbers of values in each quantization bin for normally distributed weights.
Question 12
What is "double quantization" in QLoRA?
A) Quantizing the model twice
B) Using two different quantization schemes
C) Quantizing the quantization constants themselves
D) Running quantization on two GPUs
Show Answer
Correct Answer: C
Double quantization reduces memory further by also quantizing the constants used in the first level of quantization, saving an additional ~0.4 bits per parameter.
Question 13
What library provides the BitsAndBytesConfig for 4-bit quantization?
A) transformers
B) peft
C) bitsandbytes
D) accelerate
Show Answer
Correct Answer: C
The bitsandbytes library provides the quantization functionality, though it's configured through transformers' BitsAndBytesConfig.
Question 14
What is gradient checkpointing used for?
A) Saving training progress
B) Trading compute for memory by recomputing activations
C) Checking for gradient explosion
D) Synchronizing gradients across GPUs
Show Answer
Correct Answer: B
Gradient checkpointing saves memory by not storing all activations during the forward pass, instead recomputing them during backpropagation. This trades extra computation for reduced memory.
Question 15
Which weight matrices are most commonly targeted for LoRA in transformer models?
A) Embedding layers only
B) Output projection only
C) Query and Value projections (q_proj, v_proj)
D) Layer normalization parameters
Show Answer
Correct Answer: C
The query and value projection matrices (q_proj, v_proj) are most commonly targeted as they show good results with minimal parameters. Key and output projections can also be included.
Question 16
What does prepare_model_for_kbit_training() do?
A) Converts model to k-bit format
B) Prepares quantized models for training by handling layer norms and casting
C) Loads the model into GPU memory
D) Splits the model across multiple devices
Show Answer
Correct Answer: B
This function from PEFT prepares a quantized model for training by casting layer norms to float32 and enabling gradient checkpointing, which are necessary for stable training.
Question 17
What is the purpose of the Alpaca format in instruction tuning?
A) A specific tokenizer
B) A structured format with instruction, input, and output fields
C) A compression algorithm
D) A model architecture
Show Answer
Correct Answer: B
The Alpaca format structures training data with three fields: instruction (what to do), input (optional context), and output (expected response), making it easy to format instruction-following data.
Question 18
What is the main difference between ShareGPT and Alpaca format?
A) ShareGPT uses XML
B) ShareGPT supports multi-turn conversations
C) Alpaca supports multiple languages
D) ShareGPT is smaller
Show Answer
Correct Answer: B
ShareGPT format supports multi-turn conversations with role-based messages (system, user, assistant), while Alpaca is designed for single-turn instruction-response pairs.
Question 19
What is SFTTrainer in the TRL library?
A) A model architecture
B) A trainer class for Supervised Fine-Tuning
C) A dataset loader
D) A tokenizer wrapper
Show Answer
Correct Answer: B
SFTTrainer (Supervised Fine-Tuning Trainer) is a specialized trainer class in TRL that simplifies the fine-tuning process with features like automatic chat template handling and packing.
Question 20
What does "gradient accumulation" allow you to do?
A) Store gradients permanently
B) Simulate larger batch sizes with limited memory
C) Speed up training
D) Reduce model size
Show Answer
Correct Answer: B
Gradient accumulation allows you to simulate larger effective batch sizes by accumulating gradients over multiple forward passes before performing a weight update.
Question 21
What learning rate is typically recommended for LoRA fine-tuning?
A) 1e-2 to 1e-3
B) 1e-4 to 1e-5
C) 1e-6 to 1e-7
D) 1.0
Show Answer
Correct Answer: B
LoRA fine-tuning typically uses learning rates in the 1e-4 to 1e-5 range, which is higher than full fine-tuning because only a small number of parameters are being updated.
Question 22
What is DPO (Direct Preference Optimization)?
A) A data preprocessing technique
B) An alternative to RLHF that directly optimizes preferences without a reward model
C) A quantization method
D) A model architecture
Show Answer
Correct Answer: B
DPO directly optimizes the language model using preference pairs (chosen/rejected) without needing to train a separate reward model, simplifying the RLHF pipeline.
Question 23
What type of data does DPO require?
A) Only positive examples
B) Preference pairs (chosen and rejected responses)
C) Labeled classification data
D) Unlabeled text
Show Answer
Correct Answer: B
DPO requires preference data consisting of pairs where one response is preferred (chosen) over another (rejected) for the same prompt.
Question 24
What is ORPO's main innovation compared to DPO?
A) Uses less data
B) Combines SFT and preference optimization in a single stage
C) Doesn't require preferences
D) Only works with small models
Show Answer
Correct Answer: B
ORPO (Odds Ratio Preference Optimization) combines supervised fine-tuning and preference alignment into a single training stage, unlike DPO which typically requires a separate SFT step first.
Question 25
What does the "beta" parameter control in DPO?
A) Batch size
B) The strength of preference optimization vs staying close to the reference model
C) Learning rate
D) Number of epochs
Show Answer
Correct Answer: B
Beta controls the KL divergence penalty, balancing how strongly the model optimizes for preferences versus staying close to the reference (base) model's behavior.
Section 3: Evaluation & Deployment (Questions 26-40)
Question 26
What is perplexity measuring?
A) Model size
B) How well a model predicts text (lower is better)
C) Training speed
D) Memory usage
Show Answer
Correct Answer: B
Perplexity measures how well a probability model predicts a sample. Lower perplexity indicates the model is better at predicting the text, meaning it's less "surprised" by the data.
Question 27
What is TIES-Merging?
A) A training technique
B) A method for combining multiple models that handles parameter conflicts
C) A tokenization scheme
D) An evaluation metric
Show Answer
Correct Answer: B
TIES (Trim, Elect Sign & Merge) is a model merging technique that handles conflicts between model parameters better than simple averaging by resolving sign conflicts and trimming small values.
Question 28
What tool is commonly used for merging LoRA adapters and models?
A) PyTorch
B) MergeKit
C) TensorFlow
D) NumPy
Show Answer
Correct Answer: B
MergeKit is a popular toolkit specifically designed for merging language models, supporting various methods like linear merging, TIES, DARE, and SLERP.
Question 29
What format is commonly used for CPU inference with llama.cpp?
A) ONNX
B) GGUF
C) PyTorch
D) TensorFlow SavedModel
Show Answer
Correct Answer: B
GGUF (GPT-Generated Unified Format) is the standard format for quantized models used with llama.cpp for efficient CPU inference.
Question 30
What is vLLM primarily optimized for?
A) Model training
B) High-throughput inference serving
C) Data preprocessing
D) Model quantization
Show Answer
Correct Answer: B
vLLM is an inference engine optimized for high-throughput serving of LLMs, using techniques like PagedAttention to efficiently manage memory and batch requests.
Question 31
What is PagedAttention in vLLM?
A) A training technique
B) An attention mechanism that manages KV cache memory through paging
C) A tokenization method
D) A model architecture
Show Answer
Correct Answer: B
PagedAttention manages the Key-Value cache memory like virtual memory pages, allowing more efficient memory utilization and higher throughput for inference.
Question 32
To merge LoRA weights into the base model, which method should you use?
A) model.save_pretrained()
B) model.merge_and_unload()
C) model.quantize()
D) model.train()
Show Answer
Correct Answer: B
The merge_and_unload() method merges the LoRA adapter weights into the base model weights and removes the adapter, resulting in a standard model without LoRA overhead.
Question 33
What is an ablation study in the context of fine-tuning?
A) Removing the model from memory
B) Systematically testing different components to understand their contribution
C) A type of quantization
D) A deployment strategy
Show Answer
Correct Answer: B
An ablation study systematically removes or changes components (like LoRA rank, learning rate, dataset size) to understand their individual contribution to model performance.
Question 34
What is the purpose of evaluation loss during training?
A) To update model weights
B) To monitor for overfitting on a held-out set
C) To compute gradients
D) To save checkpoints
Show Answer
Correct Answer: B
Evaluation loss on a held-out validation set helps detect overfitting—when training loss decreases but evaluation loss increases, the model is memorizing rather than generalizing.
Question 35
What does "early stopping" prevent?
A) Slow training
B) Overfitting by stopping when validation performance stops improving
C) Gradient explosion
D) Memory overflow
Show Answer
Correct Answer: B
Early stopping monitors validation performance and stops training when it stops improving, preventing the model from overfitting to the training data.
Question 36
Which metric is most commonly used to evaluate instruction-following models?
A) BLEU
B) Win rate against a baseline (human or GPT-4 evaluation)
C) Word error rate
D) F1 score
Show Answer
Correct Answer: B
Win rate comparisons (often using GPT-4 as a judge or human evaluation) are commonly used because they better capture the quality of free-form instruction-following than traditional metrics.
Question 37
What is Weights & Biases (W&B) used for in the fine-tuning workflow?
A) Model quantization
B) Experiment tracking and visualization
C) Inference serving
D) Data storage
Show Answer
Correct Answer: B
Weights & Biases is an experiment tracking platform that logs training metrics, hyperparameters, and artifacts, making it easy to compare runs and reproduce experiments.
Question 38
What is the recommended approach when deploying a fine-tuned model with LoRA for production?
A) Keep LoRA separate for flexibility
B) Merge into base model for simpler deployment, unless you need adapter swapping
C) Only use quantized versions
D) Always retrain from scratch
Show Answer
Correct Answer: B
For most production deployments, merging LoRA into the base model simplifies the stack. Keep them separate only if you need to swap between multiple adapters dynamically.
Question 39
What quantization level is typically best for production inference quality?
A) 2-bit
B) 4-bit
C) 8-bit
D) 1-bit
Show Answer
Correct Answer: C
8-bit quantization (INT8) typically offers the best balance between inference speed/memory and quality retention. 4-bit works well for many cases but may show more quality degradation.
Question 40
What is speculative decoding?
A) Using a small draft model to speed up generation by proposing tokens verified by the large model
B) Generating multiple outputs in parallel
C) Predicting training outcomes
D) A training technique
Show Answer
Correct Answer: A
Speculative decoding uses a smaller, faster draft model to propose multiple tokens, which the larger model then verifies in parallel, speeding up generation by 2-3x.
Section 4: Best Practices & Advanced Concepts (Questions 41-50)
Question 41
What is the recommended minimum dataset size for fine-tuning an instruction-following model?
A) 10 examples
B) 100-500 high-quality examples
C) 100,000+ examples always
D) Dataset size doesn't matter
Show Answer
Correct Answer: B
100-500 high-quality, diverse examples can be sufficient for many fine-tuning tasks. Quality matters more than quantity—a smaller high-quality dataset often outperforms a larger noisy one.
Question 42
What is synthetic data generation in fine-tuning?
A) Creating fake evaluation metrics
B) Using AI models to generate training data
C) Simulating GPU performance
D) Automatically creating model architectures
Show Answer
Correct Answer: B
Synthetic data generation uses AI models (often stronger ones like GPT-4) to create training examples, which can be filtered and curated to build high-quality training datasets.
Question 43
What is the benefit of using a chat template during training?
A) Faster tokenization
B) Ensures consistent formatting that matches how the model will be used at inference
C) Reduces memory usage
D) Improves quantization
Show Answer
Correct Answer: B
Chat templates ensure training data is formatted the same way as inference inputs, including special tokens for different roles. Mismatched formats between training and inference can hurt performance.
Question 44
What is data contamination in model evaluation?
A) Data corruption
B) When evaluation data was seen during training
C) Missing values in the dataset
D) Incorrect labels
Show Answer
Correct Answer: B
Data contamination occurs when evaluation benchmark data appears in training data, artificially inflating performance metrics and making fair comparison impossible.
Question 45
What LoRA rank should you start with for experimentation?
A) r=1
B) r=16 or r=32
C) r=256
D) r=1024
Show Answer
Correct Answer: B
Starting with r=16 or r=32 is a good balance between capacity and efficiency. You can then tune up or down based on results—many tasks work well with r=8, while complex tasks may need r=64+.
Question 46
What is the relationship between LoRA rank and alpha in practice?
A) Alpha should equal rank
B) Alpha is typically 1.5-2x the rank
C) Alpha should be much smaller than rank
D) They are independent
Show Answer
Correct Answer: B
Common practice is to set alpha to 1.5-2x the rank (e.g., r=16, alpha=32). This is because the effective scaling is alpha/rank, and values around 2 work well in practice.
Question 47
What causes "NaN loss" during training?
A) Missing data
B) Numerical instability often from high learning rate or improper precision handling
C) Too much data
D) Wrong optimizer
Show Answer
Correct Answer: B
NaN (Not a Number) loss typically results from numerical instability caused by learning rates that are too high, exploding gradients, or improper handling of mixed precision training.
Question 48
What is the purpose of the warmup period in training?
A) To heat up the GPU
B) To gradually increase learning rate for stable training
C) To preload data
D) To compile the model
Show Answer
Correct Answer: B
Warmup gradually increases the learning rate from a small value to the target rate, helping stabilize training in the early stages when gradients may be unstable.
Question 49
When fine-tuning for a specific domain (e.g., medical), what's most important?
A) Using the largest possible model
B) High-quality, domain-specific data with proper formatting
C) Training for many epochs
D) Maximum batch size
Show Answer
Correct Answer: B
Domain-specific fine-tuning depends critically on the quality and relevance of training data. A smaller dataset of high-quality, properly formatted domain examples typically outperforms larger general datasets.
Question 50
What is the most important factor in determining fine-tuning success?
A) Model size
B) Training time
C) Data quality and relevance
D) GPU type
Show Answer
Correct Answer: C
Data quality and relevance are the most critical factors. A well-curated, relevant dataset will produce better results than larger but lower-quality data, regardless of model size or compute.
Results Interpretation
| Score | Level | Recommendation |
|---|---|---|
| 45-50 | Expert | Ready for production fine-tuning projects |
| 40-44 | Proficient | Strong understanding, review specific weak areas |
| 35-39 | Intermediate | Review advanced sections before production work |
| 30-34 | Developing | Revisit modules 4-8 for core technical concepts |
| Below 30 | Beginner | Complete all modules with exercises before retaking |
Certification
Upon achieving a score of 80% or higher, you have demonstrated comprehensive knowledge of LLM fine-tuning concepts, techniques, and best practices.
Key competencies validated:
- ✅ Understanding of fine-tuning vs alternatives (RAG, prompting)
- ✅ PEFT methods (LoRA, QLoRA) implementation
- ✅ Data preparation and quality control
- ✅ Training configuration and optimization
- ✅ Advanced techniques (DPO, ORPO)
- ✅ Evaluation and model merging
- ✅ Production deployment considerations