Auto-Scaling

A control loop that adjusts the number of running instances (or container replicas) in response to observed load. It watches metrics (CPU, request rate, queue depth, custom signals), compares them to a target, and scales out (add instances) or in (remove instances) within configured min/max bounds.

Traffic is rarely flat; it has daily peaks, weekly cycles, and unpredictable spikes. Provisioning for peak wastes money off-peak; provisioning for average means falling over during spikes. Auto-scaling matches capacity to demand automatically, controlling cost during quiet periods while absorbing surges without a human in the loop.

Target-tracking: set a target (e.g. 60% CPU or 1000 RPS/instance); the controller adds/removes instances to hold the metric near target. Step/threshold: define rules ('+2 instances if CPU > 80% for 5 min'). Scheduled: scale ahead of known events (Black Friday, market open). Predictive: ML forecasts demand and pre-warms capacity. New instances must pass health checks before the load balancer routes to them; scale-in respects cooldowns and connection draining to avoid killing in-flight work.

• Cost efficiency: pay for peak capacity only during the peak
• Handles spikes and failures without a human reacting at 3am
• Pairs naturally with horizontal scaling of stateless tiers and queue-based workloads

• Reactive scaling lags the spike: new instances take time to boot and warm up, so you eat latency at the front edge
• Bad metrics or thresholds cause thrashing (scale out, scale in, repeat) or runaway scaling that empties your budget
• Only works for stateless/shardable tiers; you can't auto-scale a single stateful primary database

• Instagram Feed →

  The stateless feed-serving tier auto-scales on request rate to ride out daily traffic peaks

• Notification System →

  Worker pools auto-scale on queue depth: a burst of send requests spins up more workers, then scales back down

• Ticketmaster (Seat Booking) →

  Scheduled and predictive scaling ahead of a hot on-sale; the virtual queue smooths the spike auto-scaling can't react to fast enough

• Auto-scaling is reactive and instances aren't instant. For known spikes (on-sales, launches) use scheduled or predictive scaling, or a virtual queue, instead of waiting for the metric to trip.
• Scale on the right metric. CPU is a poor proxy for I/O-bound services; queue depth or request latency is often the real signal.
• Set a sane max. Without an upper bound, a traffic spike (or a retry storm) can scale you into a five-figure bill before anyone notices.
• Cold starts and connection draining matter: new instances need warm-up before taking full traffic, and scale-in must drain in-flight requests or you drop user work.