LLM Glossary
A comprehensive glossary of 40+ terms used in the world of Large Language Models.
A
AGI (Artificial General Intelligence) - AI systems capable of understanding, learning, and applying knowledge across any domain at human level or beyond. Currently theoretical.
AI Agent - An AI system that can take autonomous actions, use tools, and complete multi-step tasks with minimal human intervention.
Alignment - The challenge of ensuring AI systems behave according to human values and intentions.
Anthropic - AI safety company that created Claude, founded by former OpenAI researchers.
API (Application Programming Interface) - A way for applications to communicate with LLM services, sending prompts and receiving responses programmatically.
Attention Mechanism - The key innovation in Transformers that allows models to weigh the importance of different parts of the input when generating output.
Autoregressive - Generation method where each output token is fed back as input for predicting the next token.
C
ChatGPT - OpenAI's conversational AI product, based on GPT models fine-tuned for dialogue.
Claude - Anthropic's AI assistant, known for long context windows and safety focus.
Closed Source - AI models whose internals are not publicly available; access only via API.
Completion - The text generated by an LLM in response to a prompt.
Context Window - The maximum amount of text (in tokens) that an LLM can process at once. Like short-term memory.
E
Emergent Abilities - Capabilities that appear in LLMs at sufficient scale without being explicitly programmed (e.g., reasoning, translation).
Embedding - A numerical representation of text as a vector, capturing semantic meaning.
F
Few-Shot Learning - The ability to perform tasks with just a few examples provided in the prompt, without additional training.
Fine-Tuning - Training an existing model further on specific data to specialize it for particular tasks.
G
Gemini - Google's multimodal AI model family, successor to Bard.
GPT (Generative Pre-trained Transformer) - OpenAI's series of language models, the basis for ChatGPT.
Guardrails - Safety measures that restrict what an LLM will generate or discuss.
H
Hallucination - When an LLM generates plausible-sounding but factually incorrect or made-up information.
I
Inference - The process of running a trained model to generate outputs. (Training teaches; inference uses.)
Input Tokens - The tokens from your prompt that the model processes. Usually cheaper than output tokens.
K
Knowledge Cutoff - The date beyond which an LLM has no training data. Information after this date is unknown to the model.
L
Llama - Meta's family of open-source large language models.
LLM (Large Language Model) - AI models with billions of parameters, trained on vast text data, capable of understanding and generating human-like language.
Local LLM - Running an LLM on your own hardware rather than through cloud APIs.
M
Max Tokens - A parameter limiting how many tokens the model will generate in its response.
Mistral - French AI company known for efficient open-source models like Mistral 7B.
Multimodal - AI that can process and generate multiple types of data (text, images, audio, video).
O
Ollama - A tool for easily running open-source LLMs locally on your computer.
Open Source - LLMs whose model weights are publicly available for download and use.
OpenAI - AI research company that created GPT, DALL-E, and ChatGPT.
Output Tokens - The tokens generated by the model as its response. Usually more expensive than input tokens.
P
Parameters - The adjustable values (weights) in a neural network that are learned during training. More parameters generally means more capability.
Pre-training - The initial phase of training an LLM on massive general text data before task-specific fine-tuning.
Prompt - The text input given to an LLM to guide its response.
Prompt Engineering - The practice of crafting effective prompts to get desired outputs from LLMs.
Q
Quantization - Reducing model precision (e.g., 16-bit to 4-bit) to make models smaller and faster, with some quality trade-off.
R
RAG (Retrieval-Augmented Generation) - Combining LLMs with external knowledge retrieval to ground responses in verified information.
Rate Limit - Restrictions on how many API requests or tokens you can use in a given time period.
RLHF (Reinforcement Learning from Human Feedback) - Training technique where human raters score outputs, and the model learns to produce higher-rated responses.
S
Streaming - Receiving LLM responses token-by-token as they're generated, rather than waiting for the complete response.
System Prompt - Hidden instructions that define how an AI should behave throughout a conversation.
T
Temperature - A parameter controlling randomness in generation. Low = predictable; high = creative but risky.
Token - The basic unit LLMs process—roughly ¾ of a word on average. "Understanding" might be 3 tokens.
Tokenization - The process of breaking text into tokens for the model to process.
Training - The process of teaching an LLM by adjusting its parameters based on massive amounts of text data.
Transformer - The neural network architecture that powers modern LLMs, introduced in 2017.
Z
Zero-Shot - Performing a task with no examples—just instructions in the prompt.
Key Relationships
Quick Reference
Model Sizes
| Size | Parameters | Typical Use |
|---|---|---|
| Small | 7B | Local running, light tasks |
| Medium | 13B-30B | Good balance |
| Large | 70B+ | Complex tasks, high quality |
Token Approximations
- 1 token ≈ ¾ word
- 100 tokens ≈ 75 words
- 1 page ≈ 500-750 tokens
Temperature Settings
| Setting | Behavior |
|---|---|
| 0.0-0.3 | Deterministic, factual |
| 0.4-0.7 | Balanced |
| 0.8-1.0+ | Creative, variable |