Chapter 2: Differences from Traditional MLOps

MLOps vs LLMOps Comparison

Aspect	Traditional MLOps	LLMOps
Model Size	MB to GB	GB to TB
Training	Frequent retraining	Fine-tuning or prompting
Input	Structured features	Natural language
Output	Scores/classes	Generated text
Evaluation	Metrics (AUC, F1)	Quality, safety, relevance
Latency	ms	seconds
Cost Driver	Compute	Tokens

Unique LLMOps Challenges

1. Prompt Engineering

Traditional ML doesn't have "prompts" - LLMs require careful prompt design:

# Simple prompt
prompt_v1 = "Summarize this document: {document}"

# Improved prompt with constraints
prompt_v2 = """You are a professional summarizer. 
Summarize the following document in 3 bullet points.
Keep each point under 20 words.
Focus on key facts and figures.

Document: {document}

Summary:"""

2. Non-deterministic Outputs

# Same input, different outputs
for i in range(3):
    response = llm.generate("Write a haiku about Python")
    print(f"Attempt {i+1}: {response}")
    
# Output varies each time!

3. Token-based Pricing

def calculate_cost(input_tokens: int, output_tokens: int, model: str):
    pricing = {
        "gpt-4": {"input": 0.03, "output": 0.06},
        "gpt-3.5-turbo": {"input": 0.0005, "output": 0.0015},
        "claude-3-opus": {"input": 0.015, "output": 0.075},
    }
    
    input_cost = (input_tokens / 1000) * pricing[model]["input"]
    output_cost = (output_tokens / 1000) * pricing[model]["output"]
    
    return input_cost + output_cost

---

Next: Chapter 3: LLM-Specific Challenges