Lambda Troubleshooting
Common Errors
Overview
┌─────────────────────────────────────────────────────────────┐
│ COMMON LAMBDA ERRORS │
├─────────────────────────────────────────────────────────────┤
│ │
│ CATEGORY ERROR TYPICAL CAUSE │
│ ───────── ────── ───────────── │
│ │
│ PERMISSIONS AccessDenied Missing IAM role │
│ ResourceNotFound Incorrect ARN │
│ │
│ EXECUTION Timeout Code too slow │
│ OutOfMemory Insufficient RAM │
│ HandlerNotFound Misconfigured handler │
│ │
│ CODE ImportError Missing module │
│ SyntaxError Syntax error │
│ RuntimeError Bug in code │
│ │
│ CONFIGURATION InvalidParameter Bad config │
│ TooManyRequests Throttling │
│ │
└─────────────────────────────────────────────────────────────┘
Timeout
Symptoms and Solutions
┌─────────────────────────────────────────────────────────────┐
│ TIMEOUT │
├─────────────────────────────────────────────────────────────┤
│ │
│ SYMPTOM: │
│ "Task timed out after X seconds" │
│ │
│ POSSIBLE CAUSES: │
│ ┌─────────────────────────────────────────────────────┐ │
│ │ │ │
│ │ 1. Operation too long │ │
│ │ • Slow external API request │ │
│ │ • Processing large files │ │
│ │ • Intensive computation │ │
│ │ │ │
│ │ 2. Timeout set too short │ │
│ │ • Default: 3 seconds │ │
│ │ • Max: 15 minutes │ │
│ │ │ │
│ │ 3. Network blocking │ │
│ │ • Misconfigured VPC │ │
│ │ • Missing NAT Gateway │ │
│ │ • Blocking Security Group │ │
│ │ │ │
│ │ 4. Code in infinite wait │ │
│ │ • Unresponsive DB connection │ │
│ │ • Infinite loop │ │
│ │ │ │
│ └─────────────────────────────────────────────────────┘ │
│ │
│ SOLUTIONS: │
│ ✅ Increase timeout (if justified) │
│ ✅ Optimize the code │
│ ✅ Use timeouts in code (requests, DB) │
│ ✅ Check network config (VPC, NAT) │
│ ✅ Break into smaller functions │
│ │
└─────────────────────────────────────────────────────────────┘
Code with Timeout
import requests
import boto3
# ❌ BAD: No timeout
response = requests.get('https://api.example.com/data')
# ✅ GOOD: With timeout
response = requests.get('https://api.example.com/data', timeout=5)
# ✅ GOOD: Check remaining time
def lambda_handler(event, context):
remaining = context.get_remaining_time_in_millis()
if remaining < 5000: # Less than 5 seconds
return {'error': 'Not enough time remaining'}
# Continue processing...
Out of Memory
┌─────────────────────────────────────────────────────────────┐
│ OUT OF MEMORY │
├─────────────────────────────────────────────────────────────┤
│ │
│ SYMPTOM: │
│ "Runtime exited with error: signal: killed" │
│ "Process exited before completing request" │
│ │
│ CAUSES: │
│ • Loading large files into memory │
│ • Memory leaks │
│ • Configured RAM too low │
│ │
│ DIAGNOSTIC: │
│ ┌─────────────────────────────────────────────────────┐ │
│ │ │ │
│ │ In CloudWatch logs: │ │
│ │ │ │
│ │ REPORT RequestId: xxx │ │
│ │ Duration: 1500.00 ms │ │
│ │ Billed Duration: 1500 ms │ │
│ │ Memory Size: 128 MB │ │
│ │ Max Memory Used: 127 MB ← Close to max! │ │
│ │ │ │
│ └─────────────────────────────────────────────────────┘ │
│ │
│ SOLUTIONS: │
│ ✅ Increase allocated memory │
│ ✅ Process files in streaming │
│ ✅ Release resources after use │
│ ✅ Use temporary files (/tmp) │
│ │
└─────────────────────────────────────────────────────────────┘
Memory Optimized Code
import json
# ❌ BAD: Loads everything into memory
def process_large_file_bad(bucket, key):
s3 = boto3.client('s3')
response = s3.get_object(Bucket=bucket, Key=key)
data = response['Body'].read() # All in RAM!
return json.loads(data)
# ✅ GOOD: Streaming
def process_large_file_good(bucket, key):
s3 = boto3.client('s3')
response = s3.get_object(Bucket=bucket, Key=key)
# Process line by line
for line in response['Body'].iter_lines():
process_line(line)
Permission Errors
┌─────────────────────────────────────────────────────────────┐
│ IAM ERRORS │
├─────────────────────────────────────────────────────────────┤
│ │
│ "AccessDeniedException" │
│ "User is not authorized to perform..." │
│ │
│ CHECKLIST: │
│ │
│ ☐ Does the execution role have the correct permissions? │
│ aws lambda get-function --function-name xxx │
│ → Check "Role" ARN │
│ │
│ ☐ Is the IAM policy correct? │
│ aws iam get-role-policy --role-name xxx │
│ → Check allowed actions │
│ │
│ ☐ Are resources correctly specified? │
│ → Exact ARN vs wildcard (*) │
│ │
│ ☐ No explicit Deny blocking? │
│ → Check SCP, policies │
│ │
│ EXAMPLE OF CORRECT POLICY: │
│ ┌─────────────────────────────────────────────────────┐ │
│ │ { │ │
│ │ "Effect": "Allow", │ │
│ │ "Action": [ │ │
│ │ "s3:GetObject", │ │
│ │ "s3:PutObject" │ │
│ │ ], │ │
│ │ "Resource": "arn:aws:s3:::my-bucket/*" │ │
│ │ } │ │
│ └─────────────────────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────┘
Cold Start
┌─────────────────────────────────────────────────────────────┐
│ COLD START │
├─────────────────────────────────────────────────────────────┤
│ │
│ "Cold Start" is the initialization time when Lambda │
│ creates a new execution environment. │
│ │
│ TIMELINE: │
│ ┌─────────────────────────────────────────────────────┐ │
│ │ │ │
│ │ COLD START (1st invocation) │ │
│ │ ├── Code download (50-200ms) │ │
│ │ ├── Runtime startup (100-500ms) │ │
│ │ ├── Initialization (your init code) │ │
│ │ └── Handler execution │ │
│ │ │ │
│ │ WARM (subsequent invocations) │ │
│ │ └── Handler execution only (~10ms) │ │
│ │ │ │
│ └─────────────────────────────────────────────────────┘ │
│ │
│ FACTORS INCREASING COLD START: │
│ • Large package (many dependencies) │
│ • Heavy runtime (Java > Python > Node) │
│ • VPC (adds 1-10 seconds!) │
│ • Complex initialization │
│ │
│ OPTIMIZATIONS: │
│ ✅ Provisioned Concurrency (keeps instances warm) │
│ ✅ Reduce package size │
│ ✅ Initialize outside the handler │
│ ✅ Avoid VPC if not necessary │
│ ✅ Use lightweight runtimes │
│ │
└─────────────────────────────────────────────────────────────┘
CloudWatch Logs
Reading Logs
┌─────────────────────────────────────────────────────────────┐
│ CLOUDWATCH LOGS │
├─────────────────────────────────────────────────────────────┤
│ │
│ Location: │
│ /aws/lambda/<function-name> │
│ │
│ Log format: │
│ ┌─────────────────────────────────────────────────────┐ │
│ │ │ │
│ │ START RequestId: abc-123 │ │
│ │ [INFO] Processing event... │ │
│ │ [ERROR] Failed to connect to database │ │
│ │ END RequestId: abc-123 │ │
│ │ REPORT RequestId: abc-123 │ │
│ │ Duration: 1234.56 ms │ │
│ │ Billed Duration: 1300 ms │ │
│ │ Memory Size: 256 MB │ │
│ │ Max Memory Used: 128 MB │ │
│ │ Init Duration: 234.56 ms ← Cold start! │ │
│ │ │ │
│ └─────────────────────────────────────────────────────┘ │
│ │
│ CLI Commands: │
│ # View latest logs │
│ aws logs tail /aws/lambda/my-function --follow │
│ │
│ # Filter errors │
│ aws logs filter-log-events \ │
│ --log-group-name /aws/lambda/my-function \ │
│ --filter-pattern "ERROR" │
│ │
└─────────────────────────────────────────────────────────────┘
Diagnostic Method
┌─────────────────────────────────────────────────────────────┐
│ DIAGNOSTIC METHOD │
├─────────────────────────────────────────────────────────────┤
│ │
│ 1. IDENTIFY THE SYMPTOM │
│ • Exact error message │
│ • Returned HTTP code │
│ • Observed behavior │
│ │
│ 2. CHECK THE LOGS │
│ • CloudWatch Logs │
│ • X-Ray traces (if enabled) │
│ • Application logs │
│ │
│ 3. CHECK THE CONFIGURATION │
│ • Timeout │
│ • Memory │
│ • Environment variables │
│ • VPC / Security Groups │
│ │
│ 4. CHECK PERMISSIONS │
│ • Execution role │
│ • Attached policies │
│ • Resource policies │
│ │
│ 5. TEST IN ISOLATION │
│ • Invoke manually │
│ • Simplify the payload │
│ • Disable features one by one │
│ │
│ 6. FIX AND VALIDATE │
│ • Apply the fix │
│ • Test with the same scenario │
│ • Document the solution │
│ │
└─────────────────────────────────────────────────────────────┘
Summary of Solutions
| Error | Cause | Solution |
|---|---|---|
| Timeout | Slow code | Optimize, increase timeout |
| OutOfMemory | Insufficient RAM | Increase memory, streaming |
| AccessDenied | Missing IAM | Add permissions |
| Cold Start | Initialization | Provisioned Concurrency |
| Handler not found | Wrong name | Check handler config |
Next section: Troubleshooting Lab