Quiz: Module 1 - Introduction to LLMs

Test your understanding of Large Language Models fundamentals. This quiz contains 30 questions covering all three chapters.

---

Chapter 1: What are Large Language Models?

Question 1

What primarily distinguishes a "Large" Language Model from earlier language models?

1. The programming language used to implement it
2. The number of parameters (billions) and scale of training data
3. The type of tasks it can perform
4. The company that created it

Question 2

What are "emergent capabilities" in the context of LLMs?

1. Capabilities programmed by developers
2. Capabilities that exist in all neural networks
3. Abilities that appear only when models reach a certain scale
4. Capabilities that emerge after fine-tuning

Question 3

How does GPT-4's estimated parameter count compare to GPT-2?

1. About 10 times larger
2. About 100 times larger
3. About 1000 times larger
4. About the same size

Question 4

What is "in-context learning"?

1. Training the model on new data
2. Learning from examples provided in the prompt without weight updates
3. Learning from the model's previous responses
4. Learning from user feedback

Question 5

What is the primary objective during LLM text generation?

1. Minimizing the number of tokens generated
2. Maximizing semantic coherence
3. Predicting the next token given the context
4. Matching training data exactly

Question 6

Which of these is NOT typically a characteristic of LLMs?

1. Billions of parameters
2. Training on terabytes of text
3. Real-time learning from user interactions
4. Long context windows

Question 7

What does the "scaling hypothesis" suggest?

1. Smaller models are always more efficient
2. Larger models with more data lead to predictable capability improvements
3. Models should be scaled down for deployment
4. Scaling always leads to diminishing returns

Question 8

What is tokenization in the context of LLMs?

1. Encrypting the model weights
2. Converting text into numerical representations the model can process
3. Creating authentication tokens for API access
4. Dividing the model into smaller parts

Question 9

In autoregressive generation, how are tokens produced?

1. All tokens are generated simultaneously
2. One token at a time, each conditioned on previous tokens
3. Random tokens are generated and then sorted
4. Tokens are copied from training data

Question 10

What is an embedding in neural language models?

1. The model's configuration file
2. The input text before processing
3. A dense vector representation of a token
4. The final output of the model

---

Chapter 2: History - From RNNs to Transformers

Question 11

What was the main limitation of statistical n-gram language models?

1. They were too computationally expensive
2. Data sparsity for longer contexts and no semantic understanding
3. They required too much training data
4. They could only work with English text

Question 12

What problem did LSTM networks address that vanilla RNNs had?

1. Slow inference speed
2. Vanishing gradients over long sequences
3. Limited vocabulary size
4. Inability to process text

Question 13

What was the title of the paper that introduced the Transformer architecture?

1. "Deep Learning"
2. "BERT: Pre-training of Deep Bidirectional Transformers"
3. "Attention Is All You Need"
4. "Language Models are Few-Shot Learners"

Question 14

What year was the Transformer paper published?

1. 2015
2. 2016
3. 2017
4. 2018

Question 15

What major limitation of RNNs did Transformers solve?

1. Memory usage
2. Sequential processing that prevented parallelization
3. Limited vocabulary
4. Requirement for labeled data

Question 16

What was innovative about the attention mechanism introduced by Bahdanau et al.?

1. It removed the need for neural networks
2. It allowed the model to learn which parts of input are relevant for each output
3. It made models smaller
4. It enabled real-time learning

Question 17

Which model introduced the pre-training + fine-tuning paradigm to transformers?

1. GPT-1
2. BERT
3. GPT-3
4. T5

Question 18

How does BERT differ from GPT architecturally?

1. BERT is larger
2. BERT uses encoder-only (bidirectional) while GPT uses decoder-only (unidirectional)
3. BERT doesn't use attention
4. There is no difference

Question 19

Approximately how many parameters did GPT-3 have?

1. 1.5 billion
2. 17.5 billion
3. 175 billion
4. 1.75 trillion

Question 20

What was a key demonstration of GPT-3's capabilities?

1. Perfect grammar correction
2. Few-shot and zero-shot learning on diverse tasks
3. Real-time translation
4. Image generation

---

Chapter 3: The LLM Landscape Today

Question 21

Which company developed Claude?

1. OpenAI
2. Google
3. Anthropic
4. Meta

Question 22

What is Anthropic's distinctive approach to AI alignment?

1. Reinforcement Learning from Human Feedback
2. Constitutional AI
3. Red teaming
4. Chain-of-thought prompting

Question 23

Which company released the Llama series of open-weight models?

1. Google
2. Anthropic
3. OpenAI
4. Meta

Question 24

What is the primary advantage of open-weight models like Llama?

1. They are always more capable
2. They are easier to use
3. Full control, local deployment, and custom fine-tuning
4. They have better documentation

Question 25

Which platform serves as the central repository for open-source models?

1. GitHub
2. Hugging Face Hub
3. Kaggle
4. Model Zoo

Question 26

What is distinctive about Mistral's Mixtral model?

1. It's the largest model ever
2. It uses Mixture of Experts (MoE) architecture
3. It's multimodal
4. It's trained only on code

Question 27

What context window length does Claude 3 support?

1. 4,000 tokens
2. 32,000 tokens
3. 100,000 tokens
4. 200,000 tokens

Question 28

Which of these is NOT a major consideration when choosing between open and closed models?

1. Data privacy requirements
2. Need for custom fine-tuning
3. The color of the company logo
4. Latency requirements

Question 29

What trend has been observed in LLM API pricing over time?

1. Prices have remained constant
2. Prices have increased significantly
3. Prices have decreased substantially
4. Pricing models have stayed the same

Question 30

Which model family is specifically designed for code generation from Meta?

1. Llama
2. Code Llama
3. StarCoder
4. Copilot

---

Quiz Complete!

Congratulations on completing the Module 1 Quiz! Review any questions you found challenging and revisit the relevant chapter sections.

Passing Score: 24/30 (80%)

---

← Chapter 3: LLM Landscape

Module 2: Transformer Architecture →