Skip to main content

LLM Fine-Tuning Resources

A curated collection of notebooks, datasets, papers, tools, and communities for LLM fine-tuning.

📓 Official Course Notebooks​

Module-Specific Notebooks​

ModuleNotebookDescription
Module 4LoRA TrainingBasic LoRA fine-tuning with PEFT
Module 5QLoRA Training4-bit quantized training
Module 6Data PreparationDataset formatting pipelines
Module 7SFTTrainerComplete training workflow
Module 8DPO TrainingPreference optimization
Module 10DeploymentModel export and serving

🤗 Hugging Face Resources​

Essential Libraries​

LibraryPurposeInstall
transformersModel loading and inferencepip install transformers
peftParameter-efficient fine-tuningpip install peft
trlTraining with RL and SFTpip install trl
datasetsDataset loading and processingpip install datasets
accelerateDistributed trainingpip install accelerate
bitsandbytesQuantizationpip install bitsandbytes
ModelParametersUse CaseHF Link
Llama-3.18B/70BGeneral purposemeta-llama/Llama-3.1-8B
Mistral7BEfficient performancemistralai/Mistral-7B-v0.3
Qwen2.50.5B-72BMultilingualQwen/Qwen2.5-7B
Phi-33.8BSmall but capablemicrosoft/Phi-3-mini-4k-instruct
Gemma 22B/9B/27BGoogle's open modelgoogle/gemma-2-9b

📊 Training Datasets​

Instruction Tuning Datasets​

DatasetSizeFormatUse Case
databricks/dolly-15k15KAlpacaGeneral instructions
OpenAssistant/oasst1161KConversationsChat models
teknium/OpenHermes-2.51M+ShareGPTHigh quality
HuggingFaceH4/ultrachat_200k200KConversationsChat
meta-math/MetaMathQA395KMathMath reasoning

Preference/DPO Datasets​

DatasetPairsUse Case
Anthropic/hh-rlhf170KHelpfulness & Harmlessness
argilla/ultrafeedback-binarized60KGeneral preferences
Intel/orca_dpo_pairs12KDPO training

Code Datasets​

DatasetSizeLanguages
codeparrot/github-codeLargeMultiple
sahil2801/CodeAlpaca-20k20KMultiple
iamtarun/python_code_instructions_18k18KPython

📚 Research Papers​

Foundational Papers​

PaperYearKey Contribution
LoRA: Low-Rank Adaptation2021Introduced LoRA method
QLoRA: Efficient Finetuning20234-bit quantized LoRA
Prefix-Tuning2021Continuous prompts
P-Tuning v22021Deep prompt tuning
AdaLoRA2023Adaptive rank allocation

Alignment Papers​

PaperYearKey Contribution
DPO: Direct Preference Optimization2023Simplified RLHF
ORPO: Monolithic Preference Optimization2024Combined SFT+alignment
Constitutional AI2022Principle-based alignment
RLHF Training at Scale2022InstructGPT paper

Model Merging Papers​

PaperYearMethod
TIES-Merging2023Conflict resolution
DARE2023Drop and rescale
Model Soups2022Weight averaging

🛠️ Tools & Frameworks​

Training Tools​

ToolPurposeLink
AxolotlEasy fine-tuning CLIOpenAccess-AI-Collective/axolotl
LLaMA-FactoryUnified fine-tuning frameworkhiyouga/LLaMA-Factory
Unsloth2x faster fine-tuningunslothai/unsloth
MLXApple Silicon trainingml-explore/mlx

Inference & Deployment​

ToolPurposeLink
vLLMHigh-throughput servingvllm-project/vllm
Text Generation InferenceHuggingFace servinghuggingface/text-generation-inference
llama.cppCPU inferenceggerganov/llama.cpp
OllamaLocal model runningollama/ollama

Model Merging​

ToolPurpose
mergekitModel merging toolkit
LazyMergeKitColab notebook for merging

Experiment Tracking​

ToolPurpose
Weights & BiasesExperiment tracking
MLflowML lifecycle
TensorBoardTraining visualization

☁️ Compute Resources​

Cloud GPU Providers​

ProviderBest ForPricing Model
Google ColabLearning, small experimentsFree tier + Pro
Lambda LabsA100/H100 accessPer-hour
RunPodFlexible GPU rentalPer-hour
Vast.aiBudget GPU accessMarketplace
AWS SageMakerEnterprisePer-instance

GPU Memory Requirements​

Model SizeFull FTQLoRALoRA
7B~28GB~6GB~14GB
13B~52GB~10GB~26GB
70B~280GB~48GB~140GB

🌐 Communities​

Discord Servers​

Forums & Discussions​

Blogs & Newsletters​

📖 Books & Courses​

BookAuthorFocus
Natural Language Processing with TransformersTunstall, von Werra, WolfPractical NLP with HF
Build a Large Language Model (From Scratch)Sebastian RaschkaLLM fundamentals

Online Courses​

CoursePlatformFocus
Hugging Face NLP CourseHFTransformers & NLP
Full Stack LLM BootcampFSDLProduction LLMs
Fast.aiFast.aiDeep learning

🧪 Benchmarks & Evaluation​

LLM Leaderboards​

Evaluation Tools​

ToolPurpose
lm-evaluation-harnessComprehensive benchmarks
HELMHolistic evaluation
MT-BenchMulti-turn conversations

💡 Quick Start Templates​

Basic QLoRA Training​

from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import LoraConfig, get_peft_model, prepare_model_for_kbit_training
from trl import SFTTrainer

# 4-bit quantization config
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
)

# Load model
model = AutoModelForCausalLM.from_pretrained(
    "meta-llama/Llama-3.1-8B",
    quantization_config=bnb_config,
)

# LoRA config
lora_config = LoraConfig(
    r=16,
    lora_alpha=32,
    target_modules=["q_proj", "v_proj"],
    lora_dropout=0.05,
    task_type="CAUSAL_LM",
)

# Prepare and train
model = prepare_model_for_kbit_training(model)
model = get_peft_model(model, lora_config)

trainer = SFTTrainer(
    model=model,
    train_dataset=dataset,
    # ... training args
)
trainer.train()

Return to Course Home