Lab: Lambda Troubleshooting

Objective

Diagnose and fix failing Lambda functions using a methodical approach.

Exercise 1: Timeout

Problematic Code

# buggy_timeout.py
import time
import json

def lambda_handler(event, context):
    """
    This function has a timeout issue
    """
    print("Starting processing...")
    
    # Simulating a long process
    data = event.get('data', [])
    
    for item in data:
        # Very slow processing
        time.sleep(2)  # 2 seconds per item!
        print(f"Processed: {item}")
    
    return {
        'statusCode': 200,
        'body': json.dumps({'processed': len(data)})
    }

Configuration

• Timeout: 3 seconds
• Payload: {"data": ["a", "b", "c", "d", "e"]}

Questions

1. Why does this function fail?
2. Calculate the total time needed
3. Propose 2 different solutions

💡 Solution

1. Why:

• 5 items × 2 seconds = 10 seconds minimum
• Timeout = 3 seconds
• The function is interrupted before finishing

2. Time needed:

• Processing: 5 × 2s = 10s
• Lambda overhead: ~100ms
• Total: ~10.1 seconds

3. Solutions:

Solution A: Increase the timeout

aws lambda update-function-configuration \
    --function-name buggy-timeout \
    --timeout 30

Solution B: Optimize the code

import json
from concurrent.futures import ThreadPoolExecutor

def process_item(item):
    # Processing (simulated with reduced sleep)
    return f"Processed: {item}"

def lambda_handler(event, context):
    data = event.get('data', [])
    
    # Parallel processing
    with ThreadPoolExecutor(max_workers=5) as executor:
        results = list(executor.map(process_item, data))
    
    return {
        'statusCode': 200,
        'body': json.dumps({'results': results})
    }

Exercise 2: Permission Error

Code

# buggy_permissions.py
import json
import boto3

def lambda_handler(event, context):
    """
    This function attempts to read from S3
    """
    s3 = boto3.client('s3')
    
    bucket = event['bucket']
    key = event['key']
    
    try:
        response = s3.get_object(Bucket=bucket, Key=key)
        content = response['Body'].read().decode('utf-8')
        
        return {
            'statusCode': 200,
            'body': json.dumps({'content': content[:100]})
        }
    except Exception as e:
        return {
            'statusCode': 500,
            'body': json.dumps({'error': str(e)})
        }

Error Received

{
  "statusCode": 500,
  "body": "{\"error\": \"An error occurred (AccessDenied) when calling the GetObject operation: Access Denied\"}"
}

Questions

1. What is the cause of the error?
2. What IAM policy should be added?
3. Write the AWS CLI command to fix it

💡 Solution

1. Cause: The Lambda execution role does not have the s3:GetObject permission

2. IAM policy to add:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetObject"
            ],
            "Resource": "arn:aws:s3:::my-bucket/*"
        }
    ]
}

3. CLI Commands:

# Create the policy
cat > s3-policy.json << 'EOF'
{
    "Version": "2012-10-17",
    "Statement": [{
        "Effect": "Allow",
        "Action": ["s3:GetObject"],
        "Resource": "arn:aws:s3:::my-bucket/*"
    }]
}
EOF

# Attach to the role
aws iam put-role-policy \
    --role-name lambda-role \
    --policy-name S3ReadAccess \
    --policy-document file://s3-policy.json

Exercise 3: Out of Memory

Problematic Code

# buggy_memory.py
import json

def lambda_handler(event, context):
    """
    This function has a memory issue
    """
    size = event.get('size', 1000000)
    
    # Create a huge list in memory
    data = []
    for i in range(size):
        data.append({
            'id': i,
            'name': f'Item {i}',
            'description': 'A' * 1000  # 1KB per item
        })
    
    return {
        'statusCode': 200,
        'body': json.dumps({'count': len(data)})
    }

Configuration

• Memory: 128 MB
• Payload: {"size": 100000} (100K items × ~1KB = ~100MB)

Questions

1. Why does the function fail?
2. What memory do you recommend?
3. How to optimize without increasing memory?

💡 Solution

1. Why:

• 100,000 items × ~1KB = ~100MB of data
• Python/Lambda overhead: ~30-50MB
• Total: ~130-150MB > 128MB configured

2. Recommended memory:

• Minimum: 256 MB (with margin)
• Recommended: 512 MB (for safety)

aws lambda update-function-configuration \
    --function-name buggy-memory \
    --memory-size 512

3. Optimization without more memory:

import json

def lambda_handler(event, context):
    size = event.get('size', 1000000)
    
    # Batch processing with generator
    def process_batch(start, end):
        for i in range(start, end):
            yield {'id': i, 'processed': True}
    
    # Process in chunks of 10000
    batch_size = 10000
    total_processed = 0
    
    for batch_start in range(0, size, batch_size):
        batch_end = min(batch_start + batch_size, size)
        
        # Process the batch (does not keep all in memory)
        for item in process_batch(batch_start, batch_end):
            total_processed += 1
        
        # Free memory between batches
    
    return {
        'statusCode': 200,
        'body': json.dumps({'processed': total_processed})
    }

Exercise 4: Handler Not Found

Configuration

• File: my_function.py
• Configured handler: lambda_function.handler

Error

{
  "errorMessage": "Unable to import module 'lambda_function': No module named 'lambda_function'",
  "errorType": "Runtime.ImportModuleError"
}

Questions

1. What is the configuration error?
2. What is the correct handler value?
3. List 3 common handler errors

💡 Solution

1. Error: The handler is configured for lambda_function.handler but the file is named my_function.py

2. Correct handler:

my_function.handler

Format: <file_name_without_extension>.<function_name>

3. Common errors:

1. Wrong file name (lambda_function vs my_function)
2. Wrong function name (handler vs lambda_handler)
3. File in a subfolder (src/handler.main instead of handler.main)
4. Extension mistakenly included (handler.py.main)
5. Incorrect case (Handler vs handler)

Exercise 5: Complete Diagnostic

Scenario

A Lambda function intermittently returns errors. Here is the available information:

CloudWatch Logs:

START RequestId: abc-123
[INFO] Connecting to database...
END RequestId: abc-123
REPORT RequestId: abc-123 Duration: 29987.45 ms Billed Duration: 30000 ms Memory Size: 256 MB Max Memory Used: 89 MB Init Duration: 1234.56 ms

START RequestId: def-456
[INFO] Connecting to database...
[ERROR] Connection timed out
END RequestId: def-456
REPORT RequestId: def-456 Duration: 30000.00 ms Billed Duration: 30000 ms Memory Size: 256 MB Max Memory Used: 67 MB

Configuration:

• Timeout: 30 seconds
• Memory: 256 MB
• VPC: Yes (private subnet)

Questions

Fill in the diagnostic table:

Element	Observation	Potential Problem	Action
Duration	?	?	?
Memory	?	?	?
Init Duration	?	?	?
Error message	?	?	?
VPC	?	?	?

💡 Solution

Element	Observation	Potential Problem	Action
Duration	~30s (timeout)	Slow/blocked DB connection	DB timeout + retry
Memory	67-89 MB / 256 MB	OK, no memory issue	-
Init Duration	1234ms	High cold start	Optimize or Provisioned
Error message	Connection timed out	DB inaccessible	Check SG, NAT
VPC	Private subnet	No NAT for internet access?	Add NAT Gateway

Diagnostic: The function is in a private VPC without a NAT Gateway. It cannot reach the external database.

Solutions:

1. Add a NAT Gateway to the private subnet
2. Or place the DB in the same VPC
3. Or use VPC Endpoints if the DB is an AWS service
4. Add connection timeouts in the code

Exercise 6: Write Quality Logs

Code Before (poor logs)

def lambda_handler(event, context):
    print("Starting")
    # ... code ...
    print("Done")
    return result

Exercise

Rewrite the code with quality logs that facilitate troubleshooting:

💡 Solution

import json
import logging

# Logger configuration
logger = logging.getLogger()
logger.setLevel(logging.INFO)

def lambda_handler(event, context):
    # Structured log of the input
    logger.info(json.dumps({
        'action': 'start',
        'request_id': context.aws_request_id,
        'event_keys': list(event.keys()),
        'remaining_time_ms': context.get_remaining_time_in_millis()
    }))
    
    try:
        # Your logic
        user_id = event.get('user_id')
        logger.info(f"Processing user: {user_id}")
        
        result = process_user(user_id)
        
        # Success log
        logger.info(json.dumps({
            'action': 'complete',
            'request_id': context.aws_request_id,
            'user_id': user_id,
            'status': 'success'
        }))
        
        return {'statusCode': 200, 'body': json.dumps(result)}
        
    except Exception as e:
        # Detailed error log
        logger.error(json.dumps({
            'action': 'error',
            'request_id': context.aws_request_id,
            'error_type': type(e).__name__,
            'error_message': str(e),
            'user_id': event.get('user_id')
        }), exc_info=True)
        
        return {'statusCode': 500, 'body': json.dumps({'error': str(e)})}

Deliverable

Document to Produce

1. Solutions to the 6 exercises
2. Completed diagnostic table
3. Corrected code for each bug
4. Best practice recommendations

Evaluation Criteria

Criteria	Points
Correct diagnostic	/25
Appropriate solutions	/25
Functional corrected code	/25
Log quality	/15
Documentation	/10
Total	/100

---

Next week: MODULE 13 - Observability