Scaling Policies
Types of Policies
Overview
┌─────────────────────────────────────────────────────────────┐
│ TYPES OF POLICIES │
├─────────────────────────────────────────────────────────────┤
│ │
│ ┌─────────────────────────────────────────────────────┐ │
│ │ │ │
│ │ TARGET TRACKING (recommended) │ │
│ │ ──────────────────────────── │ │
│ │ "Maintain CPU at 50%" │ │
│ │ AWS manages everything automatically │ │
│ │ ✅ Simple ✅ Effective ✅ Recommended │ │
│ │ │ │
│ ├─────────────────────────────────────────────────────┤ │
│ │ │ │
│ │ STEP SCALING │ │
│ │ ──────────── │ │
│ │ "If CPU 60-80% → +1, If CPU 80%+ → +3" │ │
│ │ Proportional response to deviation │ │
│ │ ⚠️ More complex ✅ Fine control │ │
│ │ │ │
│ ├─────────────────────────────────────────────────────┤ │
│ │ │ │
│ │ SIMPLE SCALING (legacy) │ │
│ │ ─────────────────────── │ │
│ │ "If CPU > 70% → +1" │ │
│ │ Mandatory cooldown between actions │ │
│ │ ❌ Legacy ❌ Less responsive │ │
│ │ │ │
│ └─────────────────────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────┘
Target Tracking - Details
Predefined Metrics
┌─────────────────────────────────────────────────────────────┐
│ TARGET TRACKING METRICS │
├─────────────────────────────────────────────────────────────┤
│ │
│ METRIC RECOMMENDED TARGET │
│ ──────── ───────────────── │
│ │
│ ASGAverageCPUUtilization 40-70% │
│ (Average CPU of the group) │
│ │
│ ASGAverageNetworkIn Depends on app │
│ (Incoming network traffic) │
│ │
│ ASGAverageNetworkOut Depends on app │
│ (Outgoing network traffic) │
│ │
│ ALBRequestCountPerTarget 1000-5000 req/target │
│ (Requests per instance via ALB) │
│ │
│ ──────────────────────────────────────────────────────── │
│ │
│ Metric choice: │
│ │
│ • CPU-bound app (compute) → ASGAverageCPUUtilization │
│ • I/O-bound app (network) → ASGAverageNetworkIn/Out │
│ • Web/API → ALBRequestCountPerTarget │
│ │
└─────────────────────────────────────────────────────────────┘
Target Tracking Configuration
# CPU-based policy
aws autoscaling put-scaling-policy \
--auto-scaling-group-name my-asg \
--policy-name cpu-tracking \
--policy-type TargetTrackingScaling \
--target-tracking-configuration '{
"TargetValue": 50.0,
"PredefinedMetricSpecification": {
"PredefinedMetricType": "ASGAverageCPUUtilization"
},
"ScaleOutCooldown": 60,
"ScaleInCooldown": 300
}'
# ALB request-based policy
aws autoscaling put-scaling-policy \
--auto-scaling-group-name my-asg \
--policy-name request-tracking \
--policy-type TargetTrackingScaling \
--target-tracking-configuration '{
"TargetValue": 1000.0,
"PredefinedMetricSpecification": {
"PredefinedMetricType": "ALBRequestCountPerTarget",
"ResourceLabel": "app/my-alb/xxx/targetgroup/my-tg/yyy"
}
}'
Custom Metrics
For specific needs, you can create a custom CloudWatch metric:
# Publish a custom metric from the application
aws cloudwatch put-metric-data \
--namespace "MyApp" \
--metric-name "QueueDepth" \
--value 150 \
--unit Count
# Target Tracking policy on custom metric
aws autoscaling put-scaling-policy \
--auto-scaling-group-name my-asg \
--policy-name queue-tracking \
--policy-type TargetTrackingScaling \
--target-tracking-configuration '{
"TargetValue": 100.0,
"CustomizedMetricSpecification": {
"MetricName": "QueueDepth",
"Namespace": "MyApp",
"Statistic": "Average"
}
}'
Step Scaling - Details
Step Configuration
┌─────────────────────────────────────────────────────────────┐
│ STEP SCALING - EXAMPLE │
├─────────────────────────────────────────────────────────────┤
│ │
│ Alarm : CPUUtilization > 60% │
│ │
│ Steps : │
│ ┌─────────────────────────────────────────────────────┐ │
│ │ │ │
│ │ Deviation from threshold Action │ │
│ │ ────────────────────────── ────── │ │
│ │ │ │
│ │ 60% ──► 70% (+0 to +10) +1 instance │ │
│ │ 70% ──► 80% (+10 to +20) +2 instances │ │
│ │ 80% ──► ∞ (+20 to +∞) +3 instances │ │
│ │ │ │
│ └─────────────────────────────────────────────────────┘ │
│ │
│ Concrete example: │
│ • CPU at 65% → deviation = +5 → +1 instance │
│ • CPU at 75% → deviation = +15 → +2 instances │
│ • CPU at 90% → deviation = +30 → +3 instances │
│ │
│ Advantage: Proportional response to urgency │
│ │
└─────────────────────────────────────────────────────────────┘
CLI Configuration
# 1. Create the CloudWatch alarm
aws cloudwatch put-metric-alarm \
--alarm-name cpu-high \
--metric-name CPUUtilization \
--namespace AWS/EC2 \
--statistic Average \
--period 60 \
--threshold 60 \
--comparison-operator GreaterThanOrEqualToThreshold \
--dimensions Name=AutoScalingGroupName,Value=my-asg \
--evaluation-periods 2 \
--alarm-actions arn:aws:autoscaling:...:scalingPolicy:...:policyName/step-scale-out
# 2. Create the Step Scaling policy
aws autoscaling put-scaling-policy \
--auto-scaling-group-name my-asg \
--policy-name step-scale-out \
--policy-type StepScaling \
--adjustment-type ChangeInCapacity \
--step-adjustments '[
{
"MetricIntervalLowerBound": 0,
"MetricIntervalUpperBound": 10,
"ScalingAdjustment": 1
},
{
"MetricIntervalLowerBound": 10,
"MetricIntervalUpperBound": 20,
"ScalingAdjustment": 2
},
{
"MetricIntervalLowerBound": 20,
"ScalingAdjustment": 3
}
]'
Scheduled Scaling
Common Patterns
┌─────────────────────────────────────────────────────────────┐
│ SCHEDULED SCALING - PATTERNS │
├─────────────────────────────────────────────────────────────┤
│ │
│ PATTERN 1 : Office hours (9am-6pm) │
│ ────────────────────────────────────── │
│ │
│ ┌─────────────────────────────────────────────────────┐ │
│ │ Mon-Fri 9am : min=5, max=20, desired=5 │ │
│ │ Mon-Fri 6pm : min=2, max=5, desired=2 │ │
│ │ Sat-Sun : min=1, max=3 │ │
│ └─────────────────────────────────────────────────────┘ │
│ │
│ PATTERN 2 : E-commerce (peaks and sales) │
│ ──────────────────────────────────────── │
│ │
│ ┌─────────────────────────────────────────────────────┐ │
│ │ Normal : min=3, max=10 │ │
│ │ Black Friday: min=20, max=50 (Nov 28 to 30) │ │
│ │ Christmas : min=15, max=40 (Dec 15-26) │ │
│ └─────────────────────────────────────────────────────┘ │
│ │
│ PATTERN 3 : Batch processing (night) │
│ ──────────────────────────────────── │
│ │
│ ┌─────────────────────────────────────────────────────┐ │
│ │ Day (8am-8pm) : min=2, max=5 │ │
│ │ Night (8pm-8am): min=10, max=30 (batch jobs) │ │
│ └─────────────────────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────┘
Configuration
# Scale up in the morning (Mon-Fri at 8:30am UTC)
aws autoscaling put-scheduled-update-group-action \
--auto-scaling-group-name my-asg \
--scheduled-action-name morning-scale-up \
--recurrence "30 8 * * MON-FRI" \
--min-size 5 \
--max-size 20 \
--desired-capacity 5
# Scale down in the evening (Mon-Fri at 6pm UTC)
aws autoscaling put-scheduled-update-group-action \
--auto-scaling-group-name my-asg \
--scheduled-action-name evening-scale-down \
--recurrence "0 18 * * MON-FRI" \
--min-size 2 \
--max-size 5 \
--desired-capacity 2
# Weekend (Saturday at 12am)
aws autoscaling put-scheduled-update-group-action \
--auto-scaling-group-name my-asg \
--scheduled-action-name weekend-scale-down \
--recurrence "0 0 * * SAT" \
--min-size 1 \
--max-size 3
# One-time event (Black Friday)
aws autoscaling put-scheduled-update-group-action \
--auto-scaling-group-name my-asg \
--scheduled-action-name black-friday \
--start-time "2024-11-28T00:00:00Z" \
--end-time "2024-12-01T00:00:00Z" \
--min-size 20 \
--max-size 50
Combining Policies
Recommended Strategy
┌─────────────────────────────────────────────────────────────┐
│ COMBINATION OF POLICIES │
├─────────────────────────────────────────────────────────────┤
│ │
│ Best practice: Combine multiple types │
│ │
│ ┌─────────────────────────────────────────────────────┐ │
│ │ │ │
│ │ 1. SCHEDULED (base) │ │
│ │ ───────────────── │ │
│ │ Defines min/max based on known patterns │ │
│ │ Ex: More capacity during office hours │ │
│ │ │ │
│ │ + │ │
│ │ │ │
│ │ 2. TARGET TRACKING (fine adjustment) │ │
│ │ ─────────────────────────────── │ │
│ │ Adjusts within min/max limits │ │
│ │ Ex: Maintain CPU at 50% │ │
│ │ │ │
│ │ = │ │
│ │ │ │
│ │ Optimal architecture! │ │
│ │ │ │
│ └─────────────────────────────────────────────────────┘ │
│ │
│ Example: │
│ • Scheduled: 9am → min:5, max:20 │
│ • Target Tracking: CPU 50% │
│ • Result: Between 9am and 6pm, the number of instances │
│ varies between 5 and 20 to maintain CPU at 50% │
│ │
└─────────────────────────────────────────────────────────────┘
Best Practices
Configuration Checklist
┌─────────────────────────────────────────────────────────────┐
│ BEST PRACTICES CHECKLIST │
├─────────────────────────────────────────────────────────────┤
│ │
│ ✅ GENERAL │
│ ──────── │
│ ☐ Min >= 2 (high availability) │
│ ☐ Instances in 2+ AZ │
│ ☐ EC2 AND ELB health checks enabled │
│ ☐ Appropriate termination policy │
│ │
│ ✅ TARGET TRACKING │
│ ───────────────── │
│ ☐ Target value not too aggressive (50-70% CPU) │
│ ☐ Short scale out cooldown (60s) │
│ ☐ Longer scale in cooldown (300s) │
│ ☐ Sufficient warmup for the application │
│ │
│ ✅ INSTANCES │
│ ─────────── │
│ ☐ Idempotent and fast user data │
│ ☐ Pre-configured AMI (reduce startup time) │
│ ☐ Stateless application │
│ ☐ Graceful shutdown (lifecycle hooks) │
│ │
│ ❌ TO AVOID │
│ ───────── │
│ ☐ Min = Max = 1 (no HA) │
│ ☐ Too sensitive thresholds (oscillations) │
│ ☐ Too short cooldown (too frequent scaling) │
│ ☐ No metric monitoring │
│ │
└─────────────────────────────────────────────────────────────┘
Troubleshooting
| Problem | Probable Cause | Solution |
|---|---|---|
| Scaling too slow | Cooldown too long | Reduce scale out cooldown |
| Oscillations | Thresholds too close | Increase gap between thresholds |
| Does not scale | Metrics not received | Check CloudWatch Agent |
| Scale in kills active instances | No grace period | Add lifecycle hooks |
Summary
| Type | Use Case | Complexity |
|---|---|---|
| Target Tracking | 90% of cases, recommended | ⭐ Easy |
| Scheduled | Predictable patterns | ⭐⭐ Medium |
| Step | Proportional response | ⭐⭐⭐ Advanced |
| Combined | Optimal production | ⭐⭐ Medium |
Next section: Lab Auto Scaling