Module 5 Quiz: Training Infrastructure

Test your understanding of training infrastructure with these questions.

---

Question 1

What is data parallelism in distributed training?

1. Splitting the model across GPUs
2. Running copies of the model on different GPUs with different data batches
3. Training multiple different models
4. Parallel data loading

---

Question 2

What is mixed precision training?

1. Training with multiple datasets
2. Using both FP16 and FP32 for faster training and lower memory
3. Training with different learning rates
4. Mixing CPU and GPU training

---

Question 3

What is the main advantage of spot/preemptible instances?

1. Faster training speed
2. Better GPU quality
3. 60-90% cost savings compared to on-demand
4. Guaranteed availability

---

Question 4

What does PyTorch DDP stand for?

1. Deep Data Processing
2. Distributed Data Parallel
3. Dynamic Data Pipeline
4. Direct Data Protocol

---

Question 5

When should you use model parallelism?

1. Always, it's more efficient
2. For small models only
3. When the model doesn't fit in a single GPU's memory
4. For inference only

---

(Additional questions covering GPU selection, DeepSpeed, cloud platforms, checkpointing, etc.)

---

🎉 Quiz Complete!

Next: Module 6: Model Serving