Quiz: Introduction to Fine-Tuning
Test your knowledge about fine-tuning concepts, decision making, and the fine-tuning landscape.
30 questions - Click on each question to reveal the correct answer.
Question 1
What is fine-tuning in the context of LLMs?
A) Training a model from scratch on new data
B) Continuing training a pre-trained model on specific data
C) Reducing the model's size
D) Converting the model to a different format
Show Answer
Correct Answer: B
Fine-tuning continues the training of a pre-trained model on specific data to adapt it for particular tasks or domains, leveraging the knowledge already learned during pre-training.
Question 2
What is transfer learning?
A) Moving a model between different GPUs
B) Translating text between languages
C) Using knowledge from pre-training to improve performance on new tasks
D) Transferring data between datasets
Show Answer
Correct Answer: C
Transfer learning is the concept of using knowledge gained during pre-training on large datasets to improve performance on specific downstream tasks through fine-tuning.
Question 3
Which scenario is BEST suited for RAG instead of fine-tuning?
A) Teaching a model a specific writing style
B) Making a model output JSON consistently
C) Providing access to frequently updated product information
D) Giving a model a specific personality
Show Answer
Correct Answer: C
RAG is ideal when knowledge needs to be updated frequently because you can simply update the retrieval database without retraining the model. Fine-tuning is better for style, format, and personality.
Question 4
What does PEFT stand for?
A) Parameter-Efficient Fine-Tuning
B) Pre-trained Efficient Feature Training
C) Parallel Execution Fine-Tuning
D) Precision-Enhanced Full Training
Show Answer
Correct Answer: A
PEFT stands for Parameter-Efficient Fine-Tuning, a family of methods that train only a small subset of model parameters while keeping most frozen.
Question 5
What is catastrophic forgetting?
A) When training data is lost
B) When a model becomes too large to run
C) When a model loses previously learned knowledge after training on new data
D) When the learning rate is too high
Show Answer
Correct Answer: C
Catastrophic forgetting occurs when a model's performance on previously learned tasks degrades after being trained on new data.
Question 6
Approximately what percentage of parameters does LoRA typically train?
A) 50%
B) 25%
C) Less than 1%
D) 10%
Show Answer
Correct Answer: C
LoRA typically trains less than 1% of the original model's parameters (often 0.1-0.5%) while maintaining comparable performance to full fine-tuning.
Question 7
What is the main advantage of QLoRA over LoRA?
A) Better quality results
B) Significantly lower memory requirements through quantization
C) Faster training speed
D) Works with more model architectures
Show Answer
Correct Answer: B
QLoRA combines 4-bit quantization with LoRA, reducing memory requirements dramatically (e.g., from 14GB to 6GB for a 7B model).
Question 8
How much VRAM is approximately needed to QLoRA a 7B parameter model?
A) 2GB
B) 6GB
C) 16GB
D) 28GB
Show Answer
Correct Answer: B
QLoRA requires approximately 6GB VRAM for a 7B parameter model, making it feasible on consumer GPUs and free Colab instances.
Question 9
Which library provides the BitsAndBytesConfig for quantization?
A) transformers
B) peft
C) bitsandbytes
D) accelerate
Show Answer
Correct Answer: C
The bitsandbytes library provides the quantization functionality, though it's configured through transformers' BitsAndBytesConfig class.
Question 10
What is DPO?
A) Direct Parameter Optimization
B) Deep Pre-training Orchestration
C) Direct Preference Optimization
D) Distributed Processing Operation
Show Answer
Correct Answer: C
DPO (Direct Preference Optimization) is an alignment method that directly optimizes the model using preference pairs without needing a separate reward model.
Question 11
When should you prefer prompt engineering over fine-tuning?
A) When you need maximum quality
B) When you need quick iteration and simple customization
C) When you have unlimited compute resources
D) When the task is very complex
Show Answer
Correct Answer: B
Prompt engineering is best for quick iteration, simple customization, and when you have limited budget. It requires no training infrastructure.
Question 12
What is the typical learning rate range for LoRA fine-tuning?
A) 1e-2 to 1e-3
B) 1e-4 to 1e-5
C) 1e-6 to 1e-7
D) 1.0
Show Answer
Correct Answer: B
LoRA fine-tuning typically uses learning rates in the 1e-4 to 1e-5 range, which is higher than full fine-tuning because only a small number of parameters are being updated.
Question 13
Which of the following is NOT a good use case for fine-tuning?
A) Teaching a specific writing style
B) Providing real-time stock prices
C) Learning domain terminology
D) Consistent output formatting
Show Answer
Correct Answer: B
Real-time or frequently changing information is better handled by RAG or external APIs. Fine-tuning cannot provide real-time data.
Question 14
What is the Alpaca format used for?
A) Image generation
B) Structuring instruction-following training data
C) Model quantization
D) Tokenization
Show Answer
Correct Answer: B
The Alpaca format structures training data with instruction, input, and output fields for training instruction-following models.
Question 15
What is the difference between ShareGPT and Alpaca format?
A) ShareGPT is for images
B) ShareGPT supports multi-turn conversations
C) Alpaca is for code only
D) They are the same
Show Answer
Correct Answer: B
ShareGPT format supports multi-turn conversations with role-based messages, while Alpaca is designed for single-turn instruction-response pairs.
Question 16
What does "pre-training" involve for an LLM?
A) Fine-tuning on user data
B) Training on massive text data to learn language patterns
C) Quantizing the model
D) Deploying to production
Show Answer
Correct Answer: B
Pre-training involves training on trillions of tokens from diverse sources to learn general language understanding, facts, and capabilities.
Question 17
What is an "adapter" in the context of PEFT?
A) A hardware component
B) A small trainable module added to a frozen model
C) A type of optimizer
D) A data format
Show Answer
Correct Answer: B
An adapter is a small neural network module added to a pre-trained model that contains trainable parameters while the base model remains frozen.
Question 18
Which approach typically has the lowest per-query cost at scale?
A) Prompt engineering with long prompts
B) RAG with large knowledge base
C) Fine-tuning (shorter prompts, built-in behavior)
D) They all cost the same
Show Answer
Correct Answer: C
Fine-tuning results in shorter prompts and built-in behaviors, reducing per-query token costs at scale despite higher initial training costs.
Question 19
What is RLHF?
A) Rapid Learning for High Frequency
B) Reinforcement Learning from Human Feedback
C) Regular Language High Fidelity
D) Retrieval Learning for Hybrid Functions
Show Answer
Correct Answer: B
RLHF (Reinforcement Learning from Human Feedback) uses human preferences to train a reward model, then uses RL to optimize the language model.
Question 20
How does DPO differ from RLHF?
A) DPO doesn't require any data
B) DPO doesn't need a separate reward model
C) DPO is only for image models
D) RLHF is simpler
Show Answer
Correct Answer: B
DPO directly optimizes the model using preference pairs without training a separate reward model, simplifying the RLHF pipeline.
Question 21
What GPU memory is typically needed for full fine-tuning a 7B model?
A) 6GB
B) 14GB
C) 28GB+
D) 4GB
Show Answer
Correct Answer: C
Full fine-tuning of a 7B model typically requires 28GB+ of VRAM due to storing model weights, gradients, and optimizer states.
Question 22
What is the purpose of the LoRA rank (r) parameter?
A) To set the learning rate
B) To determine the capacity of the low-rank adaptation
C) To specify the batch size
D) To control dropout
Show Answer
Correct Answer: B
The rank (r) determines the dimensionality of the low-rank matrices and thus the adapter's capacity. Higher ranks allow more complex adaptations.
Question 23
Which library provides SFTTrainer for supervised fine-tuning?
A) transformers
B) peft
C) trl
D) datasets
Show Answer
Correct Answer: C
The TRL (Transformer Reinforcement Learning) library provides SFTTrainer for supervised fine-tuning along with trainers for DPO and RLHF.
Question 24
What is a hybrid approach in LLM customization?
A) Using both CPU and GPU
B) Combining fine-tuning with RAG
C) Training on multiple languages
D) Using multiple optimizers
Show Answer
Correct Answer: B
A hybrid approach combines methods like fine-tuning for style/behavior with RAG for dynamic knowledge to get the benefits of both.
Question 25
What is the main benefit of using Unsloth for fine-tuning?
A) Better model quality
B) 2x faster training with less memory
C) Larger model support
D) Better tokenization
Show Answer
Correct Answer: B
Unsloth provides 2x faster training speed and reduced memory usage through optimizations, making it great for quick iteration.
Question 26
When would you use full fine-tuning over PEFT methods?
A) When you have limited compute
B) When maximum customization and quality are critical
C) When you want faster training
D) When memory is constrained
Show Answer
Correct Answer: B
Full fine-tuning is chosen when you need maximum customization potential and have sufficient compute resources for critical applications.
Question 27
What is instruction tuning?
A) Training on code
B) Training on instruction-response pairs for following instructions
C) Optimizing model speed
D) Reducing model size
Show Answer
Correct Answer: B
Instruction tuning trains models on instruction-response pairs to teach them to follow user instructions, forming the basis for chat-style models.
Question 28
What quantization format does QLoRA primarily use?
A) INT8
B) Float16
C) NF4 (4-bit NormalFloat)
D) BFloat16
Show Answer
Correct Answer: C
QLoRA uses NF4 (4-bit NormalFloat) quantization, designed to have equal numbers of values in each quantization bin for normally distributed weights.
Question 29
What is the best first step when approaching a new fine-tuning project?
A) Immediately start full fine-tuning
B) Try prompt engineering first to establish a baseline
C) Deploy the base model to production
D) Collect millions of training examples
Show Answer
Correct Answer: B
Starting with prompt engineering establishes a baseline and helps determine if fine-tuning is even necessary. Many tasks can be solved with good prompts.
Question 30
What is the typical number of high-quality examples needed for effective fine-tuning?
A) 10 examples
B) 100-500 examples
C) 1 million+ examples always
D) Only 1 example
Show Answer
Correct Answer: B
100-500 high-quality, diverse examples are often sufficient for many fine-tuning tasks. Quality matters more than quantity.
Results
| Score | Level |
|---|---|
| 27-30 | Excellent - Strong foundation for fine-tuning! |
| 21-26 | Good - Review any weak areas |
| 15-20 | Average - Revisit the chapters |
| Below 15 | Needs comprehensive review |