Chapter 1: What is RAG?

Introduction

Retrieval-Augmented Generation (RAG) is a technique that enhances Large Language Models (LLMs) by providing them with relevant information retrieved from external knowledge sources. Instead of relying solely on the model's training data, RAG systems fetch relevant documents and use them to generate more accurate, up-to-date, and contextually relevant responses.

The Core Concept

Think of RAG like an open-book exam:

• Without RAG (Closed Book): The LLM answers questions using only what it memorized during training
• With RAG (Open Book): The LLM can look up relevant information before answering

Etymology and History

The term "Retrieval-Augmented Generation" was coined by Lewis et al. in their 2020 paper "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks".

Key milestones:

• 2020: Original RAG paper published by Facebook AI Research
• 2021: Dense Passage Retrieval (DPR) improvements
• 2022: Vector databases become mainstream
• 2023: LangChain and LlamaIndex popularize RAG pipelines
• 2024: Advanced RAG techniques (HyDE, Multi-Query, etc.)

RAG vs Other Approaches

RAG vs Fine-Tuning

Aspect	RAG	Fine-Tuning
Data updates	Real-time	Requires retraining
Cost	Lower	Higher
Best for	Dynamic knowledge	Behavioral changes
Transparency	Can cite sources	Black box
Setup complexity	Moderate	Higher

RAG vs Prompt Engineering

Aspect	RAG	Prompt Engineering
Context size	Can handle large knowledge bases	Limited by context window
Accuracy	Higher for factual queries	Depends on model knowledge
Use case	Knowledge-intensive tasks	General tasks

The Two Components

1. Retriever

The retriever is responsible for finding relevant information:

# Simplified retriever concept
def retrieve(query, knowledge_base, top_k=5):
    """
    Find the most relevant documents for a query
    """
    # Convert query to embedding
    query_embedding = embed(query)
    
    # Search knowledge base
    results = knowledge_base.similarity_search(
        query_embedding,
        top_k=top_k
    )
    
    return results

2. Generator

The generator uses retrieved context to produce responses:

# Simplified generator concept
def generate(query, retrieved_docs, llm):
    """
    Generate a response using retrieved context
    """
    # Build prompt with context
    context = "\n".join([doc.content for doc in retrieved_docs])
    
    prompt = f"""Based on the following context, answer the question.
    
Context:
{context}

Question: {query}

Answer:"""
    
    # Generate response
    response = llm.generate(prompt)
    
    return response

Real-World Analogy

Imagine you're a customer support agent:

Without RAG:

You answer questions from memory, potentially giving outdated or incorrect information.

With RAG:

Before answering, you quickly search your company's knowledge base, find the relevant policy document, and provide an accurate response with a reference.

When to Use RAG

RAG is ideal when you need:

✅ Up-to-date information - Company documents, news, research
✅ Domain-specific knowledge - Medical, legal, technical docs
✅ Source attribution - Citing where information came from
✅ Private data access - Internal company information
✅ Reduced hallucinations - Grounding responses in facts

RAG might not be the best choice when:

❌ Creative writing - Poetry, fiction, brainstorming
❌ General conversation - Casual chat doesn't need retrieval
❌ Behavioral changes - Teaching the model a new style (use fine-tuning)

Industry Applications

Customer Support

User: "What's your return policy for electronics?"
RAG: Retrieves current return policy → Generates accurate response

Legal Research

User: "Find case law about intellectual property in software"
RAG: Retrieves relevant cases → Summarizes findings

Medical Information

User: "What are the side effects of medication X?"
RAG: Retrieves FDA documents → Provides accurate list

Enterprise Search

User: "What was discussed in last quarter's board meeting?"
RAG: Retrieves meeting minutes → Summarizes key points

Key Terminology

Term	Definition
Retrieval	The process of finding relevant documents
Augmentation	Adding retrieved context to the prompt
Generation	LLM producing a response
Knowledge Base	Collection of documents to search
Embedding	Vector representation of text
Semantic Search	Finding documents by meaning, not keywords

Summary

• RAG combines retrieval and generation for better AI responses
• It solves key LLM limitations like outdated knowledge and hallucinations
• Two main components: Retriever (finds docs) + Generator (produces response)
• Ideal for knowledge-intensive tasks requiring accuracy and citations
• Different from fine-tuning - RAG is for knowledge, fine-tuning for behavior

---

Next Steps

Continue to Chapter 2: Why RAG? to understand the specific problems RAG solves.