sdlc.dev

Kubernetes path · Keeping it healthy · Topic 10 of 13

Autoscaling: HPA and node autoscalers

Not started

Kubernetes scales in two layers. The **HorizontalPodAutoscaler** (HPA) changes how many replicas a Deployment or StatefulSet has, based on metrics. A **node autoscaler** (Cluster Autoscaler or Karpenter) changes how many machines the cluster has, based on whether pods fit. Neither looks at the other directly; they meet through Pending pods and resource requests.

Why it matters: Autoscaling only works when requests, metrics and start-up time cooperate. Set it up blindly and it scales too late, flaps, or scales into a wall of Pending pods.

  1. 1HPA changes the replica count

    Every 15 seconds the HorizontalPodAutoscaler compares a metric with its target and sets replicas to ceil(current x currentValue / target).

  2. 2Utilization is relative to requests

    averageUtilization: 70 means 70% of the CPU request, so wrong requests mean wrong scaling.

  3. 3Scale down slowly

    By default the HPA uses the highest recommendation of the last 5 minutes when scaling down, and ignores changes within a 10% tolerance.

  4. 4Nodes follow pods

    Cluster Autoscaler or Karpenter adds nodes when pods are Pending for lack of room and removes underused ones.

1

How it works

The explanation, with examples you can run.

The HPA loop

apiVersion: autoscaling/v2kind: HorizontalPodAutoscalermetadata:  name: webspec:  scaleTargetRef:    apiVersion: apps/v1    kind: Deployment    name: web  minReplicas: 3  maxReplicas: 30  metrics:    - type: Resource      resource:        name: cpu        target:          type: Utilization          averageUtilization: 70     # percent of the CPU *request*  behavior:    scaleDown:      stabilizationWindowSeconds: 300   # the default, written out

The HPA controller runs every 15 seconds by default. For each metric it computes

desiredReplicas = ceil( currentReplicas x currentMetricValue / desiredMetricValue )

and takes the largest result across metrics, clamped to minReplicas and maxReplicas. If the ratio is within the tolerance (10% by default), it does nothing, which prevents constant small adjustments. For scale-down it uses the highest recommendation from the stabilization window (300 seconds by default), so a short dip does not remove pods that a returning spike needs.

Resource metrics (CPU, memory) come from the metrics.k8s.io API, usually provided by the metrics-server add-on, which has to be installed. Custom or external metrics (requests per second, queue length) come from adapters such as the Prometheus Adapter or KEDA.

Making HPA work well

  • Requests must be right. Utilisation is usage divided by request. If the request is double the real need, 70% target means 35% of what the pod can do, and you over-provision; if too low, you scale late.
  • Choose a metric that leads. CPU is fine for CPU-bound services. For queue workers, scale on queue depth or lag; for I/O-bound APIs, requests per second or concurrency per pod.
  • Start-up time is reaction time. Scaling takes the metric delay, plus scheduling, plus image pull, plus application start and readiness. Keep a minReplicas floor that survives the first minutes of a spike.
  • Do not fight it. Remove replicas from the Deployment manifest you apply (or ignore it in GitOps), or every deploy resets the count chosen by the HPA.
  • Memory is a poor scaling signal for most runtimes, which keep memory after load drops; pods scale up and never come down.

Adding nodes

When pods are Pending because no node has room for their requests, a node autoscaler provisions capacity:

  • Cluster Autoscaler grows and shrinks predefined node groups (cloud auto-scaling groups).
  • Karpenter chooses instance types per batch of pending pods and launches nodes directly.

Both, according to the Kubernetes docs, are sponsored by SIG Autoscaling and also remove or consolidate underused nodes by evicting their pods, which is where PodDisruptionBudgets and graceful shutdown matter again. A new node takes minutes, not seconds, so node scaling is for sustained growth; bursts are absorbed by headroom.

Vertical scaling

The Vertical Pod Autoscaler recommends (or applies) requests based on observed usage. Do not let VPA and HPA act on the same CPU or memory metric for one workload; they will fight. With in-place pod resize stable since 1.35, VPA can apply new values with fewer pod restarts.

2

Check yourself

3 questions. Get them all right to finish this topic.

  1. 1.4 replicas average 90% CPU utilisation against a target of 60%. What does the HPA ask for?
  2. 2.The HPA scaled to 30 replicas but 12 are Pending. What is missing?
  3. 3.Why does the HPA need resource requests on the target's containers to scale on CPU utilisation?

0 of 3 right so far. Get all three to finish this topic.

Go deeper