Quiz: Evaluation and Merging
30 questions on model evaluation, merging, and ablation.
Question 1
What does perplexity measure?
A) Model speed
B) How well a model predicts text
C) Model size
D) Training time
Show Answer
Correct Answer: B
Perplexity measures how well a probability model predicts a sample—lower is better.
Question 2
What is TIES merging?
A) A training method
B) A conflict resolution merging technique
C) A tokenization scheme
D) An optimizer
Show Answer
Correct Answer: B
TIES (Trim, Elect Sign & Merge) resolves conflicts between model parameters during merging.
Question 3
What does model.merge_and_unload() do?
A) Saves the model
B) Merges LoRA into base model and removes adapter
C) Loads a new model
D) Unloads from GPU
Show Answer
Correct Answer: B
merge_and_unload() integrates LoRA weights into the base model, removing adapter overhead.
Question 4
What is an ablation study?
A) Removing the model
B) Systematically testing components to understand their contribution
C) A type of loss function
D) Model compression
Show Answer
Correct Answer: B
Ablation studies systematically vary components to understand what contributes to performance.
Question 5
What tool is used for comprehensive LLM benchmarking?
A) PyTorch
B) lm-evaluation-harness
C) TensorFlow
D) NumPy
Show Answer
Correct Answer: B
lm-evaluation-harness by EleutherAI provides comprehensive benchmark evaluation for LLMs.
(Questions 6-30 cover additional topics)