Chapter 1: Instruction Tuning
Teaching Models to Follow Instructions
Instruction tuning trains models to follow user instructions effectively.
What is Instruction Tuning?
Training on instruction-response pairs to create helpful assistants:
{
"instruction": "Write a poem about the ocean",
"response": "Waves crash upon the shore..."
}
Dataset Requirements
| Quality | Requirement |
|---|---|
| Diverse | Cover many instruction types |
| Accurate | Correct responses |
| Helpful | Genuinely useful answers |
| Safe | No harmful content |
Popular Instruction Datasets
| Dataset | Size | Quality |
|---|---|---|
| Dolly 15k | 15K | High |
| OpenAssistant | 161K | High |
| Alpaca | 52K | Medium |
| OpenHermes | 1M+ | High |
Training Example
from trl import SFTTrainer
from datasets import load_dataset
dataset = load_dataset("databricks/dolly-15k")
def format_instruction(example):
return {
"text": f"""### Instruction:
{example['instruction']}
### Response:
{example['response']}"""
}
dataset = dataset.map(format_instruction)
trainer = SFTTrainer(
model=model,
train_dataset=dataset,
dataset_text_field="text",
...
)