Glossary
A comprehensive glossary of MLOps and LLMOps terminology.
A
A/B Testing - A method of comparing two versions (A and B) of a model or prompt to determine which performs better.
API (Application Programming Interface) - A set of protocols and tools for building software applications, commonly used to expose ML models as services.
Artifact - Any file or data produced during the ML lifecycle, including models, datasets, and configuration files.
Autolog - MLflow feature that automatically logs parameters, metrics, and models without explicit code.
AutoML - Automated Machine Learning; techniques that automate model selection and hyperparameter tuning.
B
Batch Processing - Processing data in groups rather than in real-time streams.
BentoML - An open-source framework for packaging and deploying ML models.
BLEU Score - Bilingual Evaluation Understudy; metric for evaluating machine translation quality.
C
Canary Deployment - Gradually rolling out changes to a small subset of users before full deployment.
CI/CD - Continuous Integration/Continuous Deployment; practices for automating software delivery.
Concept Drift - When the relationship between model inputs and outputs changes over time.
Containerization - Packaging software with its dependencies into standardized units (containers).
Covariate Drift - When the distribution of input features changes over time.
D
DAG (Directed Acyclic Graph) - A graph structure used by orchestration tools like Airflow to represent pipeline dependencies.
Data Drift - Changes in the statistical properties of input data over time.
Data Version Control (DVC) - Tool for versioning data and ML models alongside code.
DeepSpeed - Microsoft's library for efficient distributed training of large models.
Docker - Platform for containerizing applications.
E
Embedding - Dense vector representation of data (text, images) learned by neural networks.
Experiment Tracking - Recording parameters, metrics, and artifacts from ML experiments.
F
Faithfulness - In LLM evaluation, whether generated content is grounded in provided context.
Feature Engineering - Creating new input features from raw data to improve model performance.
Feature Store - Centralized repository for storing and serving ML features.
Feast - Open-source feature store for ML.
Fine-tuning - Adapting a pre-trained model to a specific task or domain.
G
GPU (Graphics Processing Unit) - Hardware accelerator commonly used for ML training and inference.
Gradient Checkpointing - Memory optimization technique that trades compute for memory.
Great Expectations - Python library for data validation and documentation.
H
Hallucination - When LLMs generate plausible but incorrect or fabricated information.
HPA (Horizontal Pod Autoscaler) - Kubernetes feature for automatically scaling pods.
Hyperparameter - Configuration values set before training (learning rate, batch size, etc.).
I
Inference - Using a trained model to make predictions on new data.
Infrastructure as Code (IaC) - Managing infrastructure through code (Terraform, Pulumi).
K
KFServing/KServe - Kubernetes-native platform for serving ML models.
Kubeflow - ML toolkit for Kubernetes with pipelines, notebooks, and serving components.
Kubernetes - Container orchestration platform for deploying and managing applications.
KV Cache - Key-Value cache storing attention computations in transformer models.
L
Label Drift - When the distribution of target labels changes over time.
LangChain - Framework for developing LLM-powered applications.
LangSmith - Platform for debugging, testing, and monitoring LLM applications.
Latency - Time taken to process a request and return a response.
LLM (Large Language Model) - Neural network trained on massive text data (GPT, Claude, Llama).
LLMOps - Specialized practices for deploying and maintaining LLM applications.
M
Mixed Precision Training - Using both FP16 and FP32 for faster training.
MLflow - Open-source platform for ML lifecycle management.
MLOps - Practices for deploying and maintaining ML models in production.
Model Card - Documentation describing a model's purpose, performance, and limitations.
Model Parallelism - Splitting a model across multiple GPUs.
Model Registry - Centralized repository for storing and managing model versions.
N
Neptune.ai - Experiment tracking and model registry platform.
O
ONNX - Open Neural Network Exchange; format for model interoperability.
Observability - Ability to understand system state through logs, metrics, and traces.
P
P99 Latency - 99th percentile response time (99% of requests are faster).
PagedAttention - vLLM's memory management technique for efficient KV cache.
Pipeline - Sequence of data processing or ML steps.
Pod - Smallest deployable unit in Kubernetes, containing one or more containers.
Prometheus - Open-source monitoring and alerting toolkit.
Prompt Engineering - Crafting effective inputs to get desired outputs from LLMs.
Prompt Injection - Attack where malicious prompts manipulate LLM behavior.
Q
Quantization - Reducing model precision (FP32 → INT8) to decrease memory and increase speed.
R
RAG (Retrieval-Augmented Generation) - Combining retrieval systems with LLMs for grounded responses.
RAGAS - Framework for evaluating RAG pipelines.
Rate Limiting - Restricting the number of requests to prevent abuse.
Red Teaming - Deliberately testing systems to find vulnerabilities.
Reproducibility - Ability to recreate the same results given the same inputs.
Runbook - Documentation of procedures for responding to incidents.
S
Shadow Mode - Running new model alongside production without affecting users.
Spot Instance - Cloud computing capacity available at reduced prices.
Streaming - Sending LLM output incrementally as it's generated.
Sweep - Automated hyperparameter search (W&B terminology).
T
TensorFlow Serving - Google's system for serving TensorFlow models.
TGI (Text Generation Inference) - HuggingFace's LLM serving solution.
Token - Basic unit of text that LLMs process.
TorchServe - PyTorch's model serving framework.
Triton Inference Server - NVIDIA's high-performance inference serving.
V
Vector Database - Database optimized for storing and searching embeddings.
vLLM - High-throughput LLM serving engine with PagedAttention.
W
Weights & Biases (W&B) - MLOps platform for experiment tracking and collaboration.
Add to Your Vocabulary
This glossary covers the essential terms. As you progress through the course, you'll develop a deeper understanding of each concept through hands-on practice.