Why won't it schedule?
A pod stuck in Pending has a reason, and it's never printed where you're looking. Find it, read it, and fix the request rather than the cluster.
You ask for three replicas. Two start. The third sits in Pending and stays there. Nothing crashed and nothing is red. kubectl get pods shows the word Pending and nothing else.
Pending means the pod exists but no node has been chosen for it. The scheduler keeps trying, and each time it fails it records why as an event on the pod. You only see events if you ask for them.
Requests, not usage
The scheduler places pods by what they request, not by what they actually use. Every container can declare requests.cpu and requests.memory. The scheduler adds up the requests already placed on each node and won’t go past the node’s capacity. From the Kubernetes docs:
although actual memory or CPU resource usage on nodes is very low, the scheduler still refuses to place a Pod on a node if the capacity check fails.
That surprises people the first time. A node can sit at 3% CPU and still be “full”, because everything on it asked for a lot. It’s deliberate: a request is a reservation for your peak, not a guess at your average.
Find it, read it, fix it
This cluster has two nodes with 2 CPUs (2000m) each. Someone deployed big with three replicas, each requesting 1500m. Work out why one won’t start, then fix it without scaling down.
Here’s what the scheduler did each time it tried the third pod:
- A pod asks for 1500m. Until a node is chosen it stays Pending - nothing crashed, nothing is red.
- The scheduler adds up what the pods on each node requested. Not what they use.
- Each node has 2000m and has already promised 1500m. 500m is left on each, so neither fits.
- So it writes the reason as an event on the pod and tries again later. kubectl describe pod is where you read it.
A node shows 4% CPU usage in your monitoring, but new pods won't schedule there because of Insufficient cpu. How can both be true?
The scheduler sums requests, not usage. Idle pods with large requests fill a node as surely as busy ones. The fix is usually right-sizing requests, not adding nodes.
Where does the reason a pod is Pending show up?
A Pending pod never started, so it has no logs. The scheduler writes a FailedScheduling event, such as '0/2 nodes are available: 2 Insufficient cpu.', which describe shows at the bottom.
Everything above ran in your browser. Now do it on a real cluster withthe tilt platform. You need Docker, kind and Tilt, about 15 GB of free disk and 8 GB of spare RAM. Setup guide · Can my machine run it?
There’s no dedicated tilt lab for this one; the minimal platform is enough. Ask for more CPU than any laptop has:
kubectl create deployment big --image=nginx --replicas=2
kubectl set resources deployment big --requests=cpu=100 # 100 whole CPUs
kubectl get pods # Pending
kubectl describe pod -l app=big | tail -5
kubectl describe node | grep -A6 "Allocated resources"
kubectl delete deployment bigSources
Missions and quizzes above earn their own XP. This marks the lesson done.