Generative AI Glossary
A comprehensive glossary of 150+ terms used in Generative AI.
A
Activation Function - Mathematical function that determines the output of a neural network node. Common types include ReLU, sigmoid, and tanh.
Adversarial Attack - Input designed to deceive AI systems into making incorrect predictions or generating harmful outputs.
AGI (Artificial General Intelligence) - AI systems capable of understanding, learning, and applying knowledge across any domain at human level or beyond.
AI Agent - AI system capable of autonomous action, tool use, and multi-step task completion.
AI Alignment - The challenge of ensuring AI systems behave according to human values and intentions.
Algorithm - A step-by-step procedure for solving a problem or accomplishing a task.
Anthropic - AI safety company that developed Claude, focused on AI alignment research.
API (Application Programming Interface) - Interface allowing software applications to communicate and interact with AI models.
Attention Mechanism - Neural network component that weighs the importance of different parts of input data.
Autoencoder - Neural network that learns efficient data representations by encoding and decoding input.
Autoregressive Model - Model that generates outputs sequentially, with each token depending on previous tokens.
B
Backpropagation - Algorithm for training neural networks by propagating errors backward through the network.
Batch Size - Number of training examples processed together before updating model weights.
BERT (Bidirectional Encoder Representations from Transformers) - Google's language model that reads text bidirectionally for better understanding.
Bias (in AI) - Systematic errors in AI outputs due to skewed training data or flawed algorithms.
Bias (in Neural Networks) - Constant term added to weighted inputs in neural network calculations.
BPE (Byte Pair Encoding) - Tokenization algorithm that iteratively merges frequent character pairs.
C
Chain-of-Thought (CoT) - Prompting technique that encourages step-by-step reasoning.
ChatGPT - OpenAI's conversational AI assistant based on GPT models.
Claude - Anthropic's AI assistant, designed with focus on helpfulness and safety.
Classification - Task of assigning inputs to predefined categories.
CLIP (Contrastive Language-Image Pre-training) - OpenAI model connecting text and images for understanding both.
Codex - OpenAI model specialized for code generation, powers GitHub Copilot.
Completion - Generated text that continues or finishes a given prompt.
Constitutional AI - Training approach using AI to evaluate and improve AI behavior based on principles.
Context Length/Window - Maximum amount of text a model can process at once.
Contrastive Learning - Training approach that learns by comparing similar and dissimilar examples.
Convolutional Neural Network (CNN) - Neural network architecture particularly effective for image processing.
D
DALL-E - OpenAI's text-to-image generation model.
Data Augmentation - Techniques to increase training data variety through transformations.
Data Poisoning - Attack that corrupts AI training data to influence model behavior.
Decoder - Component that generates output from encoded representations.
Deep Learning - Machine learning using neural networks with multiple layers.
Diffusion Model - Generative model that creates data by reversing a noise addition process.
Discriminator - GAN component that distinguishes real from generated data.
Distillation - Technique for creating smaller models that mimic larger ones.
Dropout - Regularization technique that randomly disables neurons during training.
E
Embedding - Dense numerical representation of data (text, images) in vector space.
Emergent Capabilities - Abilities that appear in AI models at certain scales without explicit training.
Encoder - Component that converts input into internal representations.
Epoch - One complete pass through the entire training dataset.
Ethical AI - Development and use of AI systems that are fair, transparent, and beneficial.
F
Feedforward Network - Neural network where information flows in one direction.
Few-Shot Learning - Learning from very few examples.
Fine-Tuning - Adapting a pre-trained model for specific tasks or domains.
Foundation Model - Large pre-trained model that serves as base for various applications.
G
GAN (Generative Adversarial Network) - Two neural networks (generator and discriminator) competing to create realistic content.
Gemini - Google's multimodal AI model series.
Generalization - Model's ability to perform well on new, unseen data.
Generative AI - AI systems that create new content (text, images, audio, video, code).
Generator - GAN component that creates new data samples.
GitHub Copilot - AI coding assistant powered by OpenAI's Codex.
GPT (Generative Pre-trained Transformer) - OpenAI's series of language models.
Gradient - Mathematical value indicating how to adjust weights to reduce error.
Gradient Descent - Optimization algorithm that adjusts weights to minimize loss.
Grounding - Connecting AI outputs to factual information or real-world data.
H
Hallucination - AI generation of false or fabricated information presented as fact.
Hidden Layer - Neural network layers between input and output layers.
Human-in-the-Loop (HITL) - Systems requiring human oversight or intervention.
Hyperparameter - Model setting determined before training (learning rate, batch size, etc.).
I
Image-to-Image - AI models that transform input images into modified outputs.
Inference - Running a trained model to generate predictions or outputs.
In-Context Learning - Model learning from examples provided in the prompt without weight updates.
Inpainting - Filling in missing or masked parts of an image.
Instruction Tuning - Training models to follow natural language instructions.
Interpretability - Ability to understand how AI models make decisions.
J
Jailbreaking - Techniques to bypass AI safety restrictions and guidelines.
K
Knowledge Base - Structured repository of information for AI systems to access.
Knowledge Cutoff - Date after which a model has no training information.
L
Langchain - Framework for building applications with large language models.
Large Language Model (LLM) - AI model trained on vast text data capable of understanding and generating language.
Latent Space - Abstract multi-dimensional space where AI models represent data features.
Layer - Component of a neural network that processes inputs.
Learning Rate - Parameter controlling how much to adjust weights during training.
LLaMA - Meta's family of open-weight large language models.
LoRA (Low-Rank Adaptation) - Efficient fine-tuning technique using small trainable matrices.
Loss Function - Mathematical function measuring the difference between predictions and actual values.
M
Machine Learning (ML) - AI systems that learn from data without explicit programming.
Masked Language Model - Model trained to predict hidden tokens in text.
Midjourney - Popular AI image generation service.
Mistral - French AI company and their series of efficient language models.
Mixture of Experts (MoE) - Architecture using specialized sub-networks for efficiency.
Model Collapse - Performance degradation when models are trained on AI-generated data.
Multimodal AI - AI systems that process and generate multiple types of data (text, images, audio).
Multi-Task Learning - Training models on multiple related tasks simultaneously.
N
Natural Language Processing (NLP) - AI field focused on understanding and generating human language.
Neural Network - Computing system inspired by biological neural networks.
Neuron/Node - Basic computational unit in neural networks.
Next-Token Prediction - Training objective where models predict the next token in a sequence.
Noise - Random variations in data, or intentional randomness in generation.
O
One-Shot Learning - Learning from a single example.
OpenAI - AI research company that created GPT, DALL-E, and ChatGPT.
Optimization - Process of adjusting model parameters to improve performance.
Outpainting - Extending images beyond their original boundaries.
Overfitting - When a model memorizes training data and fails to generalize.
P
Parameter - Trainable value in a neural network (weights and biases).
Perplexity - Metric measuring how well a language model predicts text.
Pre-Training - Initial training phase on large datasets before fine-tuning.
Prompt - Input text given to an AI model to guide its output.
Prompt Engineering - Crafting effective prompts to achieve desired AI outputs.
Prompt Injection - Attack that manipulates AI behavior through malicious prompts.
Q
QLoRA - Quantized LoRA combining quantization with efficient fine-tuning.
Quantization - Reducing model precision to decrease size and improve speed.
R
RAG (Retrieval-Augmented Generation) - Combining language models with external knowledge retrieval.
Recurrent Neural Network (RNN) - Network architecture that processes sequential data with memory.
Reinforcement Learning (RL) - Learning through trial and error with rewards and penalties.
RLHF (Reinforcement Learning from Human Feedback) - Training using human preferences to improve AI behavior.
Regularization - Techniques to prevent overfitting during training.
S
Safety - Measures ensuring AI systems don't cause harm.
Sampling - Process of generating outputs from a model's probability distribution.
Scaling Laws - Observations about how AI performance improves with size and compute.
Self-Attention - Attention mechanism where each element attends to all other elements.
Self-Supervised Learning - Learning from unlabeled data by creating prediction tasks.
Semantic Search - Search based on meaning rather than keywords.
Sequence-to-Sequence (Seq2Seq) - Models that transform one sequence into another.
Softmax - Function converting values to probability distribution.
Sora - OpenAI's text-to-video generation model.
Stable Diffusion - Open-source text-to-image diffusion model.
Style Transfer - Applying artistic style from one image to another.
Supervised Learning - Learning from labeled examples.
Synthetic Data - Artificially generated data used for training.
System Prompt - Instructions defining AI behavior and context.
T
Temperature - Parameter controlling randomness in AI outputs.
Text-to-Image - AI models generating images from text descriptions.
Text-to-Speech (TTS) - Converting text into spoken audio.
Token - Basic unit of text processed by language models.
Tokenization - Process of breaking text into tokens.
Top-k Sampling - Selecting from the k most probable next tokens.
Top-p (Nucleus) Sampling - Selecting from tokens within a probability threshold.
Training - Process of teaching a model from data.
Transfer Learning - Applying knowledge from one task to another.
Transformer - Neural network architecture using attention mechanisms.
Turing Test - Test of machine intelligence through conversation.
U
Underfitting - When a model is too simple to capture data patterns.
Unsupervised Learning - Learning from data without labels.
Upscaling - Increasing image resolution using AI.
V
Variational Autoencoder (VAE) - Generative model learning latent representations.
Vector Database - Database optimized for storing and searching embeddings.
Vision-Language Model - Model that understands both images and text.
Voice Cloning - Creating synthetic speech mimicking a specific voice.
W
Weight - Learnable parameter in neural network connections.
Word Embedding - Vector representation of words capturing semantic meaning.
Wrapper - Software layer that simplifies interaction with AI APIs.
X
XAI (Explainable AI) - AI systems designed to explain their decisions.
Z
Zero-Shot Learning - Performing tasks without any training examples.
Zero-Shot Prompting - Getting AI to perform tasks without examples in the prompt.
Numbers
1B, 7B, 70B, etc. - Model sizes measured in billions of parameters.
Quick Reference by Category
Model Types
GAN, VAE, Diffusion Model, Transformer, LLM, CNN, RNN
Major Models
GPT, BERT, DALL-E, Stable Diffusion, Claude, Gemini, LLaMA, Midjourney
Training Concepts
Fine-tuning, RLHF, Pre-training, Transfer Learning, Distillation, LoRA
Generation Concepts
Token, Temperature, Top-k, Top-p, Sampling, Prompt, Completion
Safety & Ethics
Alignment, Hallucination, Bias, Jailbreaking, Constitutional AI
This glossary is regularly updated as the field evolves.