Final Exam: Large Language Models Complete

This comprehensive exam covers all 12 modules. You need 40/50 (80%) to pass.

Time Estimate: 60-90 minutes

---

Section A: Foundations (Questions 1-15)

Question 1

What distinguishes a Large Language Model from earlier models?

1. Billions of parameters and emergent capabilities from scale
2. Different programming language
3. Real-time learning
4. Smaller size

Question 2

What architecture do modern LLMs use?

1. RNN
2. LSTM
3. Transformer
4. CNN

Question 3

What is the attention formula?

1. softmax(QK^T / √d_k) × V
2. Q + K + V
3. Q × K × V
4. softmax(Q) × softmax(K)

Question 4

Why scale by √d_k in attention?

1. Prevent large dot products from pushing softmax to extremes
2. Speed up computation
3. Reduce memory
4. Improve accuracy

Question 5

What does multi-head attention enable?

1. Attending to different representation subspaces in parallel
2. Faster training
3. Smaller model
4. Less memory

Question 6

Why do Transformers need positional encoding?

1. Self-attention is permutation invariant without position info
2. Reduce computation
3. Improve accuracy
4. Enable training

Question 7

What is causal attention used for?

1. Preventing attention to future positions in autoregressive models
2. Finding causes
3. Bidirectional understanding
4. Speeding inference

Question 8

What is the GPT pre-training objective?

1. Next token prediction
2. Masked language modeling
3. Sentence classification
4. Translation

Question 9

What is the Chinchilla-optimal tokens-per-parameter ratio?

1. ~20 tokens per parameter
2. 1 token per parameter
3. 100 tokens per parameter
4. 0.1 tokens per parameter

Question 10

What is ZeRO-3?

1. Partitioning optimizer states, gradients, AND parameters
2. Zero learning rate
3. Zero gradient
4. Three GPUs

Question 11

What company developed Claude?

1. OpenAI
2. Anthropic
3. Google
4. Meta

Question 12

What is Constitutional AI?

1. Training with principles instead of case-by-case labeling
2. Government regulations
3. Legal AI
4. Constitutional law dataset

Question 13

What is BPE tokenization?

1. Iteratively merging frequent character pairs
2. Byte processing encoding
3. Breaking into paragraphs
4. Binary processing

Question 14

What is RoPE?

1. Rotary Position Embedding - encoding position via rotation
2. Random position encoding
3. Recursive operation
4. Rate of processing

Question 15

What does embedding dimension determine?

1. Size of vector representing each token
2. Number of tokens
3. Vocabulary size
4. Sequence length

---

Section B: Training & Optimization (Questions 16-25)

Question 16

What is KV cache?

1. Storing computed key-value pairs to avoid recomputation
2. Key-value database
3. Cache for keys only
4. Training cache

Question 17

What memory reduction does INT4 provide?

1. ~87.5% (to 12.5% of FP32)
2. 50%
3. 25%
4. 75%

Question 18

What is vLLM's key innovation?

1. PagedAttention for efficient memory management
2. Better tokenization
3. Improved training
4. New architecture

Question 19

What does LoRA add to existing weights?

1. Low-rank matrices (W + BA)
2. Full new weights
3. Bias terms only
4. Normalization layers

Question 20

What is QLoRA?

1. LoRA with quantized base model
2. Quantized LoRA weights
3. Quick LoRA
4. Quality LoRA

Question 21

What is RLHF?

1. Reinforcement Learning from Human Feedback
2. Random Learning Human Feedback
3. Recursive Learning
4. Real-time Learning

Question 22

What does the reward model predict?

1. Which response humans would prefer
2. Token probabilities
3. Training loss
4. Model accuracy

Question 23

What is DPO?

1. Direct Preference Optimization - alternative to RLHF
2. Data Processing Optimization
3. Distribution Policy Optimization
4. Dynamic Parameter Optimization

Question 24

What is catastrophic forgetting?

1. Model losing pre-trained knowledge during fine-tuning
2. Memory overflow
3. Forgetting checkpoints
4. Data loss

Question 25

Why use gradient accumulation?

1. Simulate larger batch sizes with limited memory
2. Speed up training
3. Improve accuracy
4. Reduce overfitting

---

Section C: APIs & Applications (Questions 26-35)

Question 26

What does temperature=0 produce?

1. Most deterministic/predictable output
2. Most random output
3. No output
4. Error

Question 27

What is function calling?

1. LLM requesting external tool execution
2. Calling Python functions
3. API debugging
4. Code execution

Question 28

What is RAG?

1. Retrieval-Augmented Generation
2. Random Answer Generation
3. Recursive API Grouping
4. Response Accuracy Grading

Question 29

What does MMLU measure?

1. Knowledge across 57 academic subjects
2. Code generation
3. Translation quality
4. Image understanding

Question 30

What is red teaming?

1. Adversarial testing to find vulnerabilities
2. Team building
3. Performance testing
4. Color-coded testing

Question 31

What is HumanEval?

1. Code generation benchmark
2. Human preference collection
3. Writing quality test
4. Chatbot testing

Question 32

What is data contamination?

1. Training on benchmark test data
2. Corrupted data
3. Privacy breach
4. Noise in data

Question 33

What is jailbreaking?

1. Bypassing safety measures through clever prompts
2. Escaping prisons
3. Unlocking model weights
4. Removing limits

Question 34

How to handle rate limits?

1. Exponential backoff with retries
2. Ignore and continue
3. Change API key
4. Restart application

Question 35

What is streaming useful for?

1. Showing partial results to users immediately
2. Reducing costs
3. Improving accuracy
4. Saving memory

---

Section D: Advanced Topics (Questions 36-50)

Question 36

What is MoE?

1. Mixture of Experts - sparse architecture with router
2. Multiple Output Encoding
3. Model of Everything
4. Method of Extraction

Question 37

In Mixtral, how many experts are active per token?

1. 2 out of 8
2. All 8
3. 1
4. 4

Question 38

What is GPT-4V?

1. GPT-4 with vision capabilities
2. GPT-4 version
3. Video model
4. Voice model

Question 39

What is ReAct?

1. Reasoning + Acting pattern for agents
2. React.js framework
3. Reaction testing
4. Real-time actions

Question 40

What is an LLM agent?

1. LLM that can plan, use tools, and take actions
2. Customer service bot
3. API wrapper
4. Training assistant

Question 41

What is Mamba?

1. Selective State Space Model with linear complexity
2. New Transformer variant
3. Training framework
4. Dataset

Question 42

What complexity does standard attention have?

1. O(n²) in sequence length
2. O(n)
3. O(log n)
4. O(1)

Question 43

What is AGI?

1. Artificial General Intelligence
2. Advanced Generated Intelligence
3. Automated Global Intelligence
4. Artificial Gradient Intelligence

Question 44

What do current LLMs struggle with?

1. Reliable factual accuracy and complex reasoning
2. Text generation
3. Following instructions
4. Basic tasks

Question 45

What is speculative decoding?

1. Small model drafts, large model verifies
2. Random decoding
3. Parallel decoding
4. Speculative execution

Question 46

What is continuous batching?

1. Dynamically adding/removing requests from batch
2. Fixed batch size
3. Continuous training
4. Batch normalization

Question 47

What is the alignment tax?

1. Potential helpfulness reduction due to safety measures
2. Government tax
3. Compute cost
4. API pricing

Question 48

What is perplexity?

1. Exponential of average cross-entropy loss (lower is better)
2. Model confusion
3. Complexity measure
4. Training metric

Question 49

What is cosine similarity used for?

1. Measuring semantic similarity between embeddings
2. Angle calculation
3. Training loss
4. Attention weights

Question 50

What is the likely future of LLM architectures?

1. Hybrid combining attention and linear layers
2. Pure RNN return
3. No change
4. Abandoning neural networks

---

Exam Complete! 🎓

Passing Score: 40/50 (80%)

Congratulations on completing the Large Language Models Complete course!

← GlossaryResources →