Curious Pandas
Explorer0 XP

Scaling before it hurts

An autoscaler reacts to the past, and new pods take time to arrive. Watch a spike drop requests while help is still starting up, fix it without paying for the peak all day, then find out why a queue worker never scales at all.

Runs in your browserOn your machine
16 min+40 XP for finishing

A Kubernetes HorizontalPodAutoscaler (HPA) keeps changing how many copies of a pod run, to hold a metric near a target. Its arithmetic fits on one line, from the Kubernetes docs:

desiredReplicas = ceil( currentReplicas × currentMetricValue ÷ desiredMetricValue )

Four pods averaging 100% CPU against a 50% target: 4 × 100 ÷ 50 = 8 pods. Four pods at 25%: 2 pods. The rest of the docs is about the ways that simple line is slowed down on purpose:

  • It runs every 15 seconds by default, not continuously.
  • Inside a tolerance of 10% it does nothing, so it doesn’t twitch at noise.
  • CPU utilization is measured against the pod’s CPU request, not its limit or the node.
  • A pod that isn’t Ready yet is set aside. When the result would be a scale-up, the controller “conservatively assumes that the not-yet-ready pods are consuming 0% of the desired metric”. Slow-starting pods hold back the next scale-up as well as the first.
  • Scaling down waits: it takes “the highest recommendation from within” a 5-minute window, so a brief dip doesn’t throw pods away that the next minute needs.
Visual guideThe autoscaler's loop
The autoscaler's loopEvery 15 seconds the HPA reads the metric, such as CPU as a percentage of each pod's request, and computes the desired count: current replicas times current value over target, rounded up. It sets the Deployment's replica count, which creates new pods - but they serve nothing until they are Ready. Scaling down waits for the highest recommendation of the last five minutes.Metrics APICPU vs requestHPAevery 15 sDeploymentreplicas: Nnew podsuseless until Readydesired countceil(N x now/target)scale-down wait5 min, highest rec.utilizationcomputessets countcreatesgoing down?The autoscaler's loopEvery 15 seconds the HPA reads the metric, such as CPU as a percentage of each pod's request, and computes the desired count: current replicas times current value over target, rounded up. It sets the Deployment's replica count, which creates new pods - but they serve nothing until they are Ready. Scaling down waits for the highest recommendation of the last five minutes.Metrics APICPU vs requestHPAevery 15 sDeploymentreplicas: Nnew podsuseless until Readydesired countceil(N x now/target)scale-down wait5 min, highest rec.utilizationcomputessets countcreatesgoing down?
  1. The Metrics API reports each pod's CPU as a share of what it requested.
  2. Every 15 seconds the HPA reads it and works out the count: current pods times current value over target, rounded up.
  3. It writes that count into the Deployment, which creates pods. They serve nothing until they are Ready - and until then they count as 0% towards the next decision.
  4. Going down is deliberately slow: the highest recommendation of the last five minutes wins, so a brief lull does not undo the work.

The gap that drops requests

Add up the delays: up to 15 seconds before the HPA looks, then however long a new pod takes to pull its image, start, and pass its readiness probe. During that gap, the pods you already have take all of the spike. If the target is 90%, there’s almost no headroom to absorb it, and the overflow is dropped.

Three settings decide how big that gap is. Try them against a spike that quadruples traffic in two minutes:

Autoscalerchecks every 15 s · scales down after 300 s
Traffic (grey) against capacity (green)
Pods: Ready (green), starting (dashed)
0 minspike 10-18 min40 min
requests dropped-
pods on average-
backlog at the end-

Run it once as it is.

Check yourself+10 XP

Four pods are at 90% of their CPU request; the HPA target is 60%. How many pods will it ask for?

Scaling on the wrong thing

CPU is a good signal only when CPU is the work. A worker that spends its time waiting on a slow API, a database or a disk barely uses any CPU, so an HPA on CPU sees an idle service while its queue grows by the minute. Raising the limits doesn’t help, because the metric never asks for more pods. Scale on what the work actually is: queue depth, requests in flight, or consumer lag. On Kubernetes that’s usually KEDA, which can “drive the scaling of any container in Kubernetes based on the number of events needing to be processed” and works alongside the HPA rather than replacing it.

Check yourself+10 XP

A worker's queue is growing, its pods sit at 15% CPU, and the HPA (target 50% CPU) has not scaled. What is the fix?

Do it for real · your machine+50 XP bonus

Everything above ran in your browser. Now do it on a real cluster with tilt lab 05-scale-it (profile observability). You need Docker, kind and Tilt, about 15 GB of free disk and 8 GB of spare RAM. Setup guide · Can my machine run it?

Lab 05 puts an HPA (min 1, max 4, 50% CPU, a 30-second scale-down window) on the hello app. First check it can see anything at all:

kubectl apply -f labs/05-scale-it/hpa.yaml
kubectl -n hello get hpa hello

<unknown>/50% under TARGETS means no metrics, and the HPA will sit there forever without complaint. metrics-server comes with the observability profile. Then load it and watch:

./labs/05-scale-it/load.sh 90
kubectl -n hello get hpa,pods -w

When you’re done, follow the lab’s clean-up. Deleting the HPA doesn’t undo its last decision: the Deployment keeps whatever count it was left at.

Self-reported. We can't see your machine, and we don't want to. This is a bonus, never required.

Sources

Finished reading?

Missions and quizzes above earn their own XP. This marks the lesson done.