Chapter 2: Tokens and Pricing
Understanding the Economics of LLMs
Using LLM APIs costs money, and understanding how pricing works helps you budget and optimize. The key to understanding costs? Tokens.
What Are Tokens?
A token is the basic unit LLMs process—roughly a word or word piece.
Token Examples
| Text | Tokens | Count |
|---|---|---|
| "Hello" | [Hello] | 1 |
| "Hello!" | [Hello] [!] | 2 |
| "ChatGPT" | [Chat] [G] [PT] | 3 |
| "unbelievable" | [un] [believ] [able] | 3 |
| "こんにちは" | [こんに] [ち] [は] | 3 |
Rules of Thumb
- 1 token ≈ 4 characters in English
- 1 token ≈ ¾ of a word
- 100 tokens ≈ 75 words
- 1 page ≈ 500-750 tokens
How Pricing Works
Pay Per Token
Most LLM APIs charge per 1,000 or 1,000,000 tokens:
| Model | Input Price (per 1M) | Output Price (per 1M) |
|---|---|---|
| GPT-3.5-turbo | $0.50 | $1.50 |
| GPT-4 Turbo | $10 | $30 |
| GPT-4o | $5 | $15 |
| Claude 3 Haiku | $0.25 | $1.25 |
| Claude 3 Sonnet | $3 | $15 |
| Claude 3 Opus | $15 | $75 |
Prices as of late 2024; check provider websites for current pricing.
Input vs. Output Tokens
| Type | What It Is | Usually Costs |
|---|---|---|
| Input tokens | Your prompt + context | Less |
| Output tokens | AI's response | More |
Why the difference? Generating output requires more computation than reading input.
Cost Estimation
Example Calculation
Scenario: Customer support bot using GPT-4 Turbo
| Item | Tokens |
|---|---|
| System prompt | 200 |
| Conversation history (avg) | 500 |
| User question | 50 |
| Total Input | 750 |
| AI response (avg) | 200 |
| Total Output | 200 |
Cost per interaction:
- Input: 750 tokens × $10/1M = $0.0075
- Output: 200 tokens × $30/1M = $0.006
- Total: ~$0.01 per conversation
At 10,000 conversations/month: ~$100
Free Tiers and Limits
Consumer Products
| Product | Free Tier |
|---|---|
| ChatGPT | Unlimited GPT-3.5 |
| Claude | Limited daily messages |
| Gemini | Limited queries |
API Free Tiers
| Provider | Free Credits |
|---|---|
| OpenAI | $5-18 for new accounts |
| Anthropic | $5 for new accounts |
| Google AI | Generous free tier |
Cost Optimization Strategies
1. Choose the Right Model
| Task Complexity | Recommended Model | Why |
|---|---|---|
| Simple tasks | GPT-3.5, Haiku | Cheap and fast |
| Complex reasoning | GPT-4, Sonnet | Better quality |
| Critical tasks | GPT-4, Opus | Highest capability |
2. Optimize Prompts
Bad (wasteful):
"I would like you to please help me by writing a professional email response to my colleague John about rescheduling our meeting that was supposed to happen on Tuesday but I need to move it to Wednesday if possible, and please make it sound professional and friendly at the same time."
Good (efficient):
"Write a professional, friendly email to John rescheduling Tuesday's meeting to Wednesday."
3. Use Caching
If the same question is asked frequently, cache the response instead of calling the API again.
4. Limit Output Length
Set max_tokens to prevent unnecessarily long responses.
5. Batching
Combine multiple small requests into fewer larger ones when possible.
Understanding Rate Limits
What Are Rate Limits?
Limits on how many requests or tokens you can use in a time period.
Common Limits
| Limit Type | Free Tier | Paid Tier |
|---|---|---|
| Requests/minute | 3-20 | 60-500+ |
| Tokens/minute | 10K-40K | 60K-300K+ |
| Tokens/day | 150K-200K | Often unlimited |
Rate Limit Errors
When you hit limits:
- You get a "429 Too Many Requests" error
- Wait and retry
- Or upgrade your tier
Comparing Provider Economics
OpenAI
- Pros: Predictable pricing, good documentation
- Cons: Can get expensive at scale
Anthropic
- Pros: Competitive pricing, Haiku is very cheap
- Cons: Fewer model options
Google
- Pros: Very generous free tier
- Cons: Pricing can be complex
Open Source (Llama)
- Pros: Free to run (no API costs)
- Cons: Requires your own hardware/hosting
Real Cost Examples
Personal Blog Writer Bot
- 10 articles/week × 1,000 words each
- Using GPT-3.5 turbo
- Cost: ~$0.50-1/month
Customer Support (Medium Business)
- 1,000 conversations/day
- Using Claude Haiku
- Cost: ~$50-100/month
Enterprise Document Analysis
- 10,000 documents/day
- Using GPT-4 for accuracy
- Cost: ~$3,000-5,000/month
Hidden Costs
Don't forget about:
| Cost | Description |
|---|---|
| Development time | Building and maintaining the integration |
| Monitoring | Tracking usage and errors |
| Quality assurance | Reviewing outputs |
| Infrastructure | Servers, databases, etc. |
Key Takeaways
✅ Tokens are the billing units—roughly ¾ word each
✅ Input tokens are cheaper than output tokens
✅ Model choice dramatically affects cost
✅ Optimize prompts to reduce token usage
✅ Free tiers are good for learning and small projects
✅ Open source models trade API costs for hosting costs
Think About It
- Why do you think output tokens cost more than input tokens?
- How would you decide between a cheap model and an expensive one for a new project?
- What might happen if you didn't monitor your API usage?
Next Steps
Now let's explore what LLMs can't do in Chapter 3: Limitations and Hallucinations.