Glossary of LLM Fine-Tuning Terms
A comprehensive glossary of 150+ terms used in LLM fine-tuning, organized alphabetically.
A
Adapter
A small neural network module added to a pre-trained model that contains trainable parameters. During fine-tuning, only adapter parameters are updated while the base model remains frozen.
Alpaca Format
A popular dataset format for instruction tuning consisting of instruction, input, and output fields. Named after Stanford's Alpaca model.
Attention Mechanism
The core mechanism in transformers that allows the model to weigh the importance of different parts of the input when processing each position.
AutoModelForCausalLM
Hugging Face class that automatically loads the appropriate model architecture for causal language modeling (text generation).
B
Backpropagation
The algorithm used to compute gradients during training by propagating errors backward through the network.
Base Model
The pre-trained foundation model before any fine-tuning is applied. Examples include Llama, Mistral, and Falcon.
Batch Size
The number of training examples processed together in one forward/backward pass. Larger batches require more memory but can be more stable.
BFloat16 (bf16)
A 16-bit floating-point format that maintains the same range as float32 but with reduced precision. Often preferred for training over float16.
bitsandbytes
A library that enables efficient 4-bit and 8-bit quantization for loading and training large language models.
C
Catastrophic Forgetting
The phenomenon where a model loses previously learned knowledge when trained on new data. A major concern in fine-tuning.
Causal Language Modeling (CLM)
A training objective where the model predicts the next token based only on previous tokens. Used for GPT-style models.
Chat Template
A format specification for converting conversations into tokens that the model expects, including special tokens for different roles.
Checkpoint
A saved snapshot of model weights during training, allowing recovery and continuation of training.
Constitutional AI (CAI)
An AI safety technique that trains models to follow a set of principles or "constitution" for ethical behavior.
D
Dataset Sharding
Splitting a large dataset into smaller chunks for distributed training or to reduce memory usage.
DeepSpeed
A deep learning optimization library by Microsoft that enables efficient training of large models through various techniques.
Direct Preference Optimization (DPO)
A simpler alternative to RLHF that directly optimizes the model using preference data without a separate reward model.
Double Quantization
A technique in QLoRA that quantizes the quantization constants themselves, further reducing memory usage.
Dropout
A regularization technique that randomly sets some activations to zero during training to prevent overfitting.
E
Embedding Layer
The layer that converts token IDs into dense vector representations that the model can process.
Epoch
One complete pass through the entire training dataset.
Evaluation Loss
The loss computed on a held-out validation set to monitor for overfitting during training.
F
Fine-Tuning
The process of further training a pre-trained model on a specific dataset to adapt it to a particular task or domain.
Flash Attention
An efficient attention implementation that reduces memory usage and improves speed by avoiding materialization of the full attention matrix.
Float16 (fp16)
A 16-bit floating-point format that uses half the memory of float32. Can cause numerical instability in some cases.
Float32 (fp32)
Standard 32-bit floating-point precision, offering high accuracy but requiring more memory.
Frozen Parameters
Model parameters that are not updated during training. In PEFT, most parameters are frozen.
Full Fine-Tuning
Traditional fine-tuning approach where all model parameters are updated during training.
G
GGUF
A file format for storing quantized models, commonly used with llama.cpp for efficient CPU inference.
Gradient Accumulation
Technique to simulate larger batch sizes by accumulating gradients over multiple forward passes before updating weights.
Gradient Checkpointing
A memory-saving technique that recomputes activations during the backward pass instead of storing them.
Gradient Clipping
Limiting the magnitude of gradients during training to prevent exploding gradients.
H
Hallucination
When a model generates plausible-sounding but factually incorrect or fabricated information.
Hugging Face Hub
A platform for sharing and discovering pre-trained models, datasets, and demos.
Hyperparameter
A configuration value set before training begins (e.g., learning rate, batch size) as opposed to learned parameters.
I
IA3 (Infused Adapter by Inhibiting and Amplifying Inner Activations)
A PEFT method that learns to scale activations rather than adding new weights, using even fewer parameters than LoRA.
Inference
Using a trained model to generate predictions on new inputs.
Instruction Tuning
Fine-tuning a model to follow instructions by training on instruction-response pairs.
INT4/INT8
4-bit and 8-bit integer quantization formats used to reduce model size and memory usage.
K
Knowledge Distillation
Training a smaller "student" model to mimic the behavior of a larger "teacher" model.
KV Cache
Key-Value cache that stores computed attention keys and values to speed up autoregressive generation.
L
Label Smoothing
A regularization technique that softens the target probability distribution to prevent overconfidence.
Learning Rate
The step size for gradient updates during optimization. Critical hyperparameter for successful training.
Learning Rate Schedule
A strategy for adjusting the learning rate during training (e.g., warmup, decay, cosine annealing).
Linear Probe
Fine-tuning only a linear classification head on top of frozen features.
LoRA (Low-Rank Adaptation)
A PEFT method that adds trainable low-rank decomposition matrices to frozen weights, dramatically reducing trainable parameters.
LoRA Alpha
A scaling hyperparameter in LoRA that controls the magnitude of the adaptation.
LoRA Rank
The rank of the low-rank matrices in LoRA. Higher ranks increase capacity but also parameters.
Loss Function
A function that measures the difference between model predictions and ground truth, guiding optimization.
M
Masked Language Modeling (MLM)
A training objective where random tokens are masked and the model predicts them. Used for BERT-style models.
MergeKit
A tool for merging multiple fine-tuned models or adapters into a single model.
Mixed Precision Training
Training with both float16 and float32 to reduce memory while maintaining numerical stability.
Model Merging
Combining multiple models or adapters into one, often averaging weights or using more sophisticated methods like TIES or DARE.
Multi-GPU Training
Distributing training across multiple GPUs using data parallelism or model parallelism.
N
NF4 (4-bit NormalFloat)
A quantization format designed to have equal number of values in each quantization bin for normally distributed weights.
Nucleus Sampling (Top-p)
A decoding strategy that samples from the smallest set of tokens whose cumulative probability exceeds p.
O
ONNX
Open Neural Network Exchange format for model interoperability and deployment.
Optimizer
Algorithm for updating model weights during training (e.g., Adam, AdamW, SGD).
ORPO (Odds Ratio Preference Optimization)
A preference optimization method that combines supervised fine-tuning and preference alignment in a single stage.
Overfitting
When a model memorizes training data instead of learning general patterns, leading to poor generalization.
P
PagedAttention
An attention mechanism used in vLLM that manages memory more efficiently through paging.
Parameter Count
The total number of trainable weights in a model, often measured in billions for LLMs.
PEFT (Parameter-Efficient Fine-Tuning)
A family of methods that fine-tune only a small subset of parameters while keeping most of the model frozen.
Perplexity
A metric measuring how well a model predicts text. Lower perplexity indicates better performance.
Preference Data
Training data consisting of pairs or rankings of responses, used for RLHF, DPO, and related methods.
Prefix Tuning
A PEFT method that prepends trainable "virtual tokens" to the input without modifying model weights.
Pre-training
Initial training of a model on large-scale data before task-specific fine-tuning.
Prompt Tuning
Similar to prefix tuning, learning soft prompts that are prepended to inputs.
Q
QLoRA (Quantized LoRA)
Combining 4-bit quantization with LoRA to enable fine-tuning of large models on consumer hardware.
Quantization
Reducing the precision of model weights (e.g., from float16 to int4) to reduce memory and accelerate inference.
R
Rank (in LoRA)
The dimensionality of the low-rank matrices. Controls the capacity vs. efficiency trade-off.
Regularization
Techniques to prevent overfitting, such as dropout, weight decay, or early stopping.
Reinforcement Learning from Human Feedback (RLHF)
A training approach using human preferences to train a reward model, then using RL to optimize the language model.
Reward Model
A model trained to predict human preferences, used in RLHF to provide training signal.
S
Sampling Temperature
A parameter controlling randomness in generation. Higher values produce more diverse but less coherent outputs.
Self-Attention
Attention mechanism where queries, keys, and values all come from the same sequence.
Sequence Length
The number of tokens in an input or output sequence. Limited by model architecture and memory.
ShareGPT Format
A dataset format derived from shared ChatGPT conversations, using role-based message structure.
SFTTrainer
Hugging Face TRL class for Supervised Fine-Tuning of language models.
SLERP (Spherical Linear Interpolation)
A method for smoothly interpolating between model weights during merging.
Soft Prompt
Learned embedding vectors prepended to inputs instead of discrete text prompts.
Speculative Decoding
A technique that uses a smaller draft model to propose tokens, verified by the larger model for faster generation.
Supervised Fine-Tuning (SFT)
Standard fine-tuning using labeled input-output pairs with cross-entropy loss.
System Prompt
Instructions provided to set the model's behavior and context for a conversation.
T
Target Modules
In LoRA, the specific weight matrices (e.g., q_proj, v_proj) where low-rank adaptations are applied.
Temperature
See Sampling Temperature.
TIES (TrIm, Elect Sign & Merge)
A model merging method that handles parameter conflicts better than simple averaging.
Tokenizer
Component that converts text to token IDs and vice versa for model input/output.
Top-k Sampling
A decoding strategy that only considers the k most likely next tokens.
Trainable Parameters
Parameters that are updated during training, as opposed to frozen parameters.
Transfer Learning
Using knowledge learned from one task to improve performance on a different but related task.
Transformer
The neural network architecture underlying modern LLMs, based on self-attention mechanisms.
TRL (Transformer Reinforcement Learning)
Hugging Face library providing trainers for SFT, RLHF, DPO, and other training methods.
V
Validation Set
A held-out portion of data used to evaluate model performance during training.
vLLM
A high-throughput inference engine for LLMs using PagedAttention.
VRAM (Video RAM)
GPU memory used to store model weights, activations, and gradients during training.
W
Warmup
Gradually increasing the learning rate at the start of training to improve stability.
Weight Decay
A regularization technique that penalizes large weights to prevent overfitting.
Weights & Biases (W&B)
A popular platform for experiment tracking, model logging, and visualization.
Z
Zero Redundancy Optimizer (ZeRO)
DeepSpeed's memory optimization that partitions optimizer states, gradients, and parameters across devices.
Zero-Shot
Using a model for a task without any task-specific training or examples.
Quick Reference Card
| Concept | Key Information |
|---|---|
| LoRA Rank | Typical values: 8, 16, 32, 64 |
| LoRA Alpha | Often set to 2x the rank |
| Learning Rate | 1e-5 to 1e-4 for fine-tuning |
| Batch Size | 4-32 depending on VRAM |
| Epochs | 1-5 for most fine-tuning tasks |
| QLoRA Memory | ~4-6GB for 7B models |
| Full FT Memory | ~28GB+ for 7B models |