Lab: Route 53 Failover Configuration
Lab Objective
Design and configure a DNS failover solution with Route 53 to ensure service continuity during an outage.
Part 1: Scenario
Context
GlobalBank is an online bank that must maintain 99.99% availability. In the event of a primary infrastructure failure, customers must be automatically redirected to a backup site.
Target Architecture
┌─────────────────────────────────────────────────────────────┐
│ GLOBALBANK FAILOVER ARCHITECTURE │
├─────────────────────────────────────────────────────────────┤
│ │
│ www.globalbank.com │
│ │ │
│ ▼ │
│ ┌──────────────┐ │
│ │ Route 53 │ │
│ │ │ │
│ │ Health Check │ │
│ └──────┬───────┘ │
│ │ │
│ ┌────┴────┐ │
│ │ │ │
│ ▼ ▼ │
│ PRIMARY SECONDARY │
│ │
│ ┌─────────────────┐ ┌─────────────────┐ │
│ │ us-east-1 │ │ eu-west-1 │ │
│ │ │ │ │ │
│ │ ┌───────────┐ │ │ ┌───────────┐ │ │
│ │ │ ALB │ │ │ │ ALB │ │ │
│ │ └─────┬─────┘ │ │ └─────┬─────┘ │ │
│ │ │ │ │ │ │ │
│ │ ┌─────┴─────┐ │ │ ┌─────┴─────┐ │ │
│ │ │ EC2 ASG │ │ │ │ EC2 ASG │ │ │
│ │ └───────────┘ │ │ └───────────┘ │ │
│ │ │ │ │ │ │ │
│ │ ┌───────────┐ │ │ ┌───────────┐ │ │
│ │ │ RDS Multi │ │ │ │ RDS Read │ │ │
│ │ │ AZ │ │ │ │ Replica │ │ │
│ │ └───────────┘ │ │ └───────────┘ │ │
│ │ │ │ │ │
│ └─────────────────┘ └─────────────────┘ │
│ │
│ Normal: 100% traffic → us-east-1 │
│ Failover: 100% traffic → eu-west-1 │
│ │
└─────────────────────────────────────────────────────────────┘
Requirements
| Requirement | Value |
|---|---|
| Availability | 99.99% |
| RTO (Recovery Time Objective) | < 5 minutes |
| RPO (Recovery Point Objective) | < 1 minute |
| Outage Detection | < 90 seconds |
Part 2: Health Checks Configuration
Exercise 2.1: Create the Primary Health Check
Complete the health check configuration:
aws route53 create-health-check \
--caller-reference globalbank-primary-$(date +%s) \
--health-check-config '{
"Type": "_______________",
"ResourcePath": "_______________",
"FullyQualifiedDomainName": "primary-alb.us-east-1.elb.amazonaws.com",
"Port": ___,
"RequestInterval": ___,
"FailureThreshold": ___,
"EnableSNI": true,
"Regions": ["us-east-1", "us-west-2", "eu-west-1"]
}'
💡 Solution
aws route53 create-health-check \
--caller-reference globalbank-primary-$(date +%s) \
--health-check-config '{
"Type": "HTTPS",
"ResourcePath": "/health",
"FullyQualifiedDomainName": "primary-alb.us-east-1.elb.amazonaws.com",
"Port": 443,
"RequestInterval": 10,
"FailureThreshold": 2,
"EnableSNI": true,
"Regions": ["us-east-1", "us-west-2", "eu-west-1"]
}'
Explanation:
RequestInterval: 10= check every 10 seconds (fast)FailureThreshold: 2= 2 consecutive failures = unhealthy- Max detection time = 10s × 2 = 20 seconds
- 3 regions to avoid false positives
Exercise 2.2: Calculate Detection Time
With the following parameters, calculate the maximum time to detect an outage:
| Parameter | Value |
|---|---|
| Request Interval | 10 seconds |
| Failure Threshold | 2 |
| Number of checkers | 3 regions |
Max detection time = ?
💡 Solution
Max detection time = Request Interval × Failure Threshold
= 10s × 2
= 20 seconds
Note: Route 53 considers the health check "unhealthy" when
at least 18% of checkers (1 out of 3 minimum) report failures.
Typical effective time: 15-25 seconds
Part 3: Failover Configuration
Exercise 3.1: Primary Record
aws route53 change-resource-record-sets \
--hosted-zone-id Z1234567890ABC \
--change-batch '{
"Changes": [{
"Action": "CREATE",
"ResourceRecordSet": {
"Name": "www.globalbank.com",
"Type": "A",
"SetIdentifier": "_______________",
"Failover": "_______________",
"AliasTarget": {
"HostedZoneId": "Z35SXDOTRQ7X7K",
"DNSName": "primary-alb.us-east-1.elb.amazonaws.com",
"EvaluateTargetHealth": ___
},
"HealthCheckId": "xxx-primary-health-check"
}
}]
}'
💡 Solution
aws route53 change-resource-record-sets \
--hosted-zone-id Z1234567890ABC \
--change-batch '{
"Changes": [{
"Action": "CREATE",
"ResourceRecordSet": {
"Name": "www.globalbank.com",
"Type": "A",
"SetIdentifier": "primary-us-east-1",
"Failover": "PRIMARY",
"AliasTarget": {
"HostedZoneId": "Z35SXDOTRQ7X7K",
"DNSName": "primary-alb.us-east-1.elb.amazonaws.com",
"EvaluateTargetHealth": true
},
"HealthCheckId": "xxx-primary-health-check"
}
}]
}'
Exercise 3.2: Secondary Record
aws route53 change-resource-record-sets \
--hosted-zone-id Z1234567890ABC \
--change-batch '{
"Changes": [{
"Action": "CREATE",
"ResourceRecordSet": {
"Name": "www.globalbank.com",
"Type": "A",
"SetIdentifier": "_______________",
"Failover": "_______________",
"AliasTarget": {
"HostedZoneId": "Z1234567890DEF",
"DNSName": "secondary-alb.eu-west-1.elb.amazonaws.com",
"EvaluateTargetHealth": ___
}
}
}]
}'
💡 Solution
aws route53 change-resource-record-sets \
--hosted-zone-id Z1234567890ABC \
--change-batch '{
"Changes": [{
"Action": "CREATE",
"ResourceRecordSet": {
"Name": "www.globalbank.com",
"Type": "A",
"SetIdentifier": "secondary-eu-west-1",
"Failover": "SECONDARY",
"AliasTarget": {
"HostedZoneId": "Z1234567890DEF",
"DNSName": "secondary-alb.eu-west-1.elb.amazonaws.com",
"EvaluateTargetHealth": true
}
}
}]
}'
Note: The secondary does not need a HealthCheckId because it is used only when the primary is unhealthy.
Part 4: Test Scenarios
Exercise 4.1: Failover Test Plan
Complete the test plan:
| Step | Action | Expected Result | Verification |
|---|---|---|---|
| 1 | Initial state | ? | ? |
| 2 | Simulate primary failure | ? | ? |
| 3 | Verify failover | ? | ? |
| 4 | Restore primary | ? | ? |
| 5 | Verify failback | ? | ? |
💡 Solution
| Step | Action | Expected Result | Verification |
|---|---|---|---|
| 1 | Initial state | DNS resolves to ALB us-east-1 | dig www.globalbank.com |
| 2 | Stop primary ALB or return 503 | Health check becomes unhealthy | Route 53 Console |
| 3 | Wait 30s | DNS resolves to ALB eu-west-1 | dig www.globalbank.com |
| 4 | Restore primary ALB | Health check becomes healthy | Route 53 Console |
| 5 | Wait for propagation | DNS resolves to ALB us-east-1 | dig www.globalbank.com |
Exercise 4.2: Test Script
#!/bin/bash
# test-failover.sh
DOMAIN="www.globalbank.com"
PRIMARY_IP="52.xxx.xxx.xxx"
SECONDARY_IP="54.xxx.xxx.xxx"
echo "=== Test Failover Route 53 ==="
# Step 1: Verify initial state
echo "1. Initial state..."
CURRENT_IP=$(dig +short $DOMAIN | head -1)
if [ "$CURRENT_IP" == "$PRIMARY_IP" ]; then
echo " ✅ DNS points to PRIMARY"
else
echo " ❌ DNS does not point to PRIMARY: $CURRENT_IP"
exit 1
fi
# Step 2: Simulate failure (manual)
echo "2. Simulate PRIMARY failure now..."
echo " (Stop the ALB or modify /health to return 503)"
read -p " Press Enter when done..."
# Step 3: Wait for failover
echo "3. Waiting for failover (60s max)..."
for i in {1..12}; do
sleep 5
CURRENT_IP=$(dig +short $DOMAIN | head -1)
echo " Check $i: $CURRENT_IP"
if [ "$CURRENT_IP" == "$SECONDARY_IP" ]; then
echo " ✅ Failover succeeded in $((i*5)) seconds!"
break
fi
done
# Step 4: Verify access
echo "4. Test access to secondary..."
HTTP_CODE=$(curl -s -o /dev/null -w "%{http_code}" https://$DOMAIN/health)
if [ "$HTTP_CODE" == "200" ]; then
echo " ✅ Site accessible via SECONDARY"
else
echo " ❌ Site not accessible: HTTP $HTTP_CODE"
fi
echo "=== Test completed ==="
Part 5: Monitoring and Alerts
CloudWatch Alarm Configuration
# Alert when the health check becomes unhealthy
aws cloudwatch put-metric-alarm \
--alarm-name globalbank-primary-unhealthy \
--metric-name HealthCheckStatus \
--namespace AWS/Route53 \
--statistic Minimum \
--period 60 \
--threshold 1 \
--comparison-operator LessThanThreshold \
--dimensions Name=HealthCheckId,Value=xxx-primary-health-check \
--evaluation-periods 1 \
--alarm-actions arn:aws:sns:us-east-1:123456789:ops-alerts \
--alarm-description "GlobalBank primary site is unhealthy"
Recommended Dashboard
| Metric | Source | Alert Threshold |
|---|---|---|
| HealthCheckStatus | Route53 | < 1 |
| HealthCheckPercentageHealthy | Route53 | < 100% |
| RequestCount (primary) | ALB | Info |
| RequestCount (secondary) | ALB | > 0 (active failover!) |
| Latency | ALB | > 1s |
Part 6: Impact Analysis
Exercise: TTL Impact Calculation
If the TTL is 60 seconds and a failover occurs, how long can users still be directed to the failed primary?
💡 Solution
Max TTL impact = Record TTL
= 60 seconds
Scenario:
1. T=0s : User makes DNS request, receives primary IP
2. T=0s : Local cache stores for 60s
3. T=30s : Primary fails, Route 53 switches to secondary
4. T=30s : New DNS request → receives secondary IP ✅
5. T=30s : User with cache → still goes to primary ❌
6. T=60s : Cache expires, new request → secondary ✅
Conclusion:
- Short TTL (60s) = fast failover but more DNS requests
- Long TTL (300s) = fewer requests but slow failover
- For critical DR: TTL 30-60s recommended
Final Deliverable
Document to Produce
- Failover Architecture (diagram)
- Complete Configuration (Health Check + Records)
- Test Plan with results
- Calculations (detection time, TTL impact)
- Monitoring (configured alerts)
Evaluation Criteria
| Criterion | Points |
|---|---|
| Correct architecture | /20 |
| Well-configured Health Check | /20 |
| Correct Failover Records | /20 |
| Complete test plan | /20 |
| Monitoring and alerts | /10 |
| Document quality | /10 |
| Total | /100 |
Next week: MODULE 11 - Lambda