Quiz: How Generative AI Works
Test your knowledge about neural networks, transformers, and the training process.
30 questions - Click on each question to reveal the correct answer.
Question 1
What is a neural network inspired by?
A) Computer circuits
B) The human brain
C) The internet
D) Quantum mechanics
Show Answer
Correct Answer: B
Neural networks are computing systems loosely inspired by the human brain, with interconnected nodes (neurons) that process information through weighted connections.
Question 2
What are the three types of layers in a neural network?
A) First, middle, last
B) Input, hidden, output
C) Start, process, end
D) Data, compute, result
Show Answer
Correct Answer: B
Neural networks have input layers (receive data), hidden layers (process data), and output layers (produce results).
Question 3
What makes a neural network "deep"?
A) It's buried underground
B) It has many hidden layers
C) It uses deep data
D) It thinks deeply
Show Answer
Correct Answer: B
A neural network is called "deep" when it has many hidden layers. Training such networks is called "deep learning."
Question 4
What does the attention mechanism in transformers do?
A) Makes the AI pay attention to users
B) Allows each element to relate to all other elements
C) Filters out irrelevant data
D) Speeds up processing
Show Answer
Correct Answer: B
The attention mechanism allows every element in the input to "attend" to (relate to) every other element, enabling the model to understand relationships regardless of position.
Question 5
What was the key paper that introduced the Transformer architecture called?
A) "The Deep Learning Revolution"
B) "Attention Is All You Need"
C) "Transformers: More Than Meets the Eye"
D) "Neural Networks Unleashed"
Show Answer
Correct Answer: B
The 2017 paper "Attention Is All You Need" by Google researchers introduced the Transformer architecture that powers all modern large language models.
Question 6
What are parameters in a neural network?
A) User settings
B) Adjustable weights and biases that store learning
C) Speed controls
D) Data inputs
Show Answer
Correct Answer: B
Parameters are the adjustable numbers (weights and biases) in a neural network that are modified during training. They represent what the model has learned.
Question 7
Approximately how many parameters does GPT-3 have?
A) 1 million
B) 175 million
C) 175 billion
D) 1.75 trillion
Show Answer
Correct Answer: C
GPT-3 has approximately 175 billion parameters, which was a major scale-up from previous models.
Question 8
What is the main task used in pre-training language models?
A) Translation
B) Next word prediction
C) Question answering
D) Image recognition
Show Answer
Correct Answer: B
Pre-training uses next word prediction—the model learns to predict what word comes next in a sequence. This simple task teaches grammar, facts, and reasoning patterns.
Question 9
What is RLHF?
A) Real-time Learning and Handling Framework
B) Reinforcement Learning from Human Feedback
C) Recursive Language Helper Function
D) Random Learning and High Fidelity
Show Answer
Correct Answer: B
RLHF stands for Reinforcement Learning from Human Feedback—a technique where human preferences guide the model toward better, more aligned responses.
Question 10
What is a token in transformer models?
A) A currency for AI
B) A piece of text (word, word part, or character)
C) A reward signal
D) A type of layer
Show Answer
Correct Answer: B
A token is a piece of text that the model processes—typically a common word, a word part, or a punctuation mark. "ChatGPT" might be split into multiple tokens.
Question 11
What is a context window?
A) A visual display
B) The maximum text a model can process at once
C) A type of attention
D) Training data size
Show Answer
Correct Answer: B
The context window is the maximum amount of text (in tokens) that a transformer can process at once. Larger context windows allow for processing longer documents.
Question 12
Why is positional encoding needed in transformers?
A) To make processing faster
B) Because attention processes all positions simultaneously
C) To reduce memory usage
D) To connect to external databases
Show Answer
Correct Answer: B
Since attention processes all tokens simultaneously (unlike sequential processing), positional encoding adds information about word order so the model knows "Dog bites man" differs from "Man bites dog."
Question 13
What is fine-tuning?
A) Adjusting display settings
B) Adapting a pre-trained model for specific tasks
C) Making the model smaller
D) Fixing bugs in the code
Show Answer
Correct Answer: B
Fine-tuning is the process of adapting a pre-trained model for specific tasks or behaviors by training on a curated dataset relevant to the desired application.
Question 14
What problem did transformers solve compared to RNNs?
A) They use less memory
B) They can process all words simultaneously
C) They don't need training
D) They only work with images
Show Answer
Correct Answer: B
Transformers can process all input simultaneously (parallel processing), while RNNs had to process sequentially (one word at a time), making them slower and causing them to "forget" distant context.
Question 15
What does backpropagation do?
A) Backs up the data
B) Traces errors back through the network to adjust weights
C) Moves data backward through layers
D) Reverses the model's predictions
Show Answer
Correct Answer: B
Backpropagation calculates how much each weight contributed to the error and adjusts them accordingly, propagating the error signal backward through the network.
Question 16
What is overfitting?
A) When the model is too large
B) When the model memorizes training data instead of learning patterns
C) When training takes too long
D) When the model uses too much memory
Show Answer
Correct Answer: B
Overfitting occurs when a model memorizes specific training examples rather than learning general patterns, resulting in poor performance on new data.
Question 17
Which type of transformer is GPT (used in ChatGPT)?
A) Encoder-only
B) Decoder-only
C) Encoder-decoder
D) Multi-encoder
Show Answer
Correct Answer: B
GPT (Generative Pre-trained Transformer) is a decoder-only model, optimized for generating text word by word.
Question 18
What is an emergent ability in AI?
A) An ability that appears suddenly at certain scales
B) An emergency feature
C) A basic capability
D) An ability that emerges during inference
Show Answer
Correct Answer: A
Emergent abilities are capabilities that appear suddenly when models reach certain sizes—like arithmetic or reasoning—that weren't explicitly trained for.
Question 19
What is multi-head attention?
A) An AI with multiple screens
B) Multiple attention mechanisms running in parallel
C) Attention for multiple languages
D) Head-mounted display interface
Show Answer
Correct Answer: B
Multi-head attention runs multiple attention mechanisms in parallel, each focusing on different aspects of the relationships in the data.
Question 20
Why is RLHF important?
A) It makes training faster
B) It aligns model behavior with human preferences
C) It reduces model size
D) It improves image quality
Show Answer
Correct Answer: B
RLHF aligns model behavior with human preferences, making outputs more helpful, less harmful, and more appropriate for user needs.
Question 21
What is the learning rate in training?
A) How fast the AI learns facts
B) How big each weight adjustment step is
C) The speed of data processing
D) Time to train one epoch
Show Answer
Correct Answer: B
The learning rate determines how big each adjustment to the weights is during training. Too big = unstable; too small = too slow.
Question 22
What type of neural network excels at image processing?
A) Recurrent Neural Network (RNN)
B) Convolutional Neural Network (CNN)
C) Feedforward Network
D) Transformer
Show Answer
Correct Answer: B
Convolutional Neural Networks (CNNs) are specialized for image processing, using filters to detect features like edges, shapes, and patterns.
Question 23
What is the purpose of an activation function?
A) To activate the computer
B) To introduce non-linearity so networks can learn complex patterns
C) To speed up processing
D) To connect to the internet
Show Answer
Correct Answer: B
Activation functions introduce non-linearity, allowing neural networks to learn complex, non-linear patterns. Without them, networks could only learn simple linear relationships.
Question 24
Approximately how much did training GPT-4 cost?
A) $100,000
B) $1 million
C) $100+ million
D) $10 billion
Show Answer
Correct Answer: C
Training GPT-4 reportedly cost over $100 million, reflecting the massive computational resources required.
Question 25
What is instruction fine-tuning?
A) Fine-tuning based on user instructions
B) Training the model to follow instructions properly
C) Writing instructions for the model
D) Manual adjustment of weights
Show Answer
Correct Answer: B
Instruction fine-tuning trains the model to follow user instructions properly, converting a base model into a helpful assistant that responds appropriately to prompts.
Question 26
What does the loss function measure?
A) How much data is lost
B) How wrong the model's predictions are
C) How fast the model learns
D) Memory usage
Show Answer
Correct Answer: B
The loss function measures how wrong the model's predictions are compared to the correct answers. Training aims to minimize this loss.
Question 27
What is gradient descent?
A) A type of neural network
B) The method to adjust weights by moving in the direction that reduces error
C) A way to compress data
D) A training dataset
Show Answer
Correct Answer: B
Gradient descent is an optimization method that adjusts weights by moving in the direction that reduces error, like walking downhill to find the lowest point.
Question 28
What are Q, K, and V in attention?
A) Quality, Knowledge, Value
B) Query, Key, Value
C) Quantum, Kinetic, Vector
D) Question, Keyword, Variable
Show Answer
Correct Answer: B
In attention mechanisms, Q (Query) represents what we're looking for, K (Key) represents what each element contains, and V (Value) represents the information to use.
Question 29
What is Constitutional AI?
A) AI that follows the law
B) An approach where AI critiques itself using defined principles
C) Government-regulated AI
D) AI based on constitutional documents
Show Answer
Correct Answer: B
Constitutional AI (developed by Anthropic) is a training approach where the AI is trained to critique its own outputs according to a set of defined principles.
Question 30
Why is data quality important in AI training?
A) It affects storage costs
B) Biased or incorrect data leads to biased or incorrect models
C) It determines processing speed
D) It's not actually important
Show Answer
Correct Answer: B
Data quality is crucial because "garbage in, garbage out"—biased, incorrect, or limited training data results in a model that produces biased, incorrect, or limited outputs.
Results
| Score | Level |
|---|---|
| 27-30 | Excellent - Strong understanding of AI mechanics |
| 21-26 | Good - Some review may be helpful |
| 15-20 | Average - Review the chapters again |
| Below 15 | Needs comprehensive review |