Chapter 3: Building Applications with LLMs

Architecture Patterns

1. Simple completion: Direct API calls
2. RAG: Retrieve → Augment → Generate
3. Agents: Plan → Execute → Observe
4. Chains: Compose multiple LLM calls

Error Handling

import time
from openai import RateLimitError, APIError

def call_with_retry(func, max_retries=3):
    for attempt in range(max_retries):
        try:
            return func()
        except RateLimitError:
            wait = 2 ** attempt
            time.sleep(wait)
        except APIError as e:
            if attempt == max_retries - 1:
                raise
    return None

Caching

import hashlib
from functools import lru_cache

@lru_cache(maxsize=1000)
def cached_completion(prompt_hash):
    return client.chat.completions.create(...)

def get_completion(prompt):
    hash_key = hashlib.md5(prompt.encode()).hexdigest()
    return cached_completion(hash_key)

Cost Management

# Track token usage
total_tokens = 0
for response in responses:
    total_tokens += response.usage.total_tokens

# Estimate cost (GPT-4 example)
cost = (total_tokens / 1000) * 0.03  # Per 1K tokens

---

← Chapter 2Quiz →