Kubernetes Node Stuck in NotReady

The API server remembers the node, but kubelet stopped reporting. The node itself usually tells you why within one journalctl command.

What you'll see

Root causes

kubelet or container runtime down

The kubelet hasn't posted its lease. Check `systemctl status kubelet containerd` on the node.

CNI plugin missing or broken

The network plugin on that node failed to start, so kubelet marks the network not ready. Events show 'container runtime network not ready'.

Cert or clock problems

Kubelet client cert expired (or the node's clock drifted), so it can't authenticate — node appears dead from the API's view.

Fix it

  1. Check the node's conditions from the API side
    kubectl describe node <node> | grep -A6 Conditions
  2. On the node, read kubelet's own errors
    sudo journalctl -u kubelet --since '10 min ago' --no-pager | tail -40
  3. Verify the runtime is alive
    sudo systemctl status containerd kubelet
  4. For CNI failures, reinstall/roll the plugin on that node
    kubectl -n kube-system rollout restart daemonset/<cni-name>

Field note

A NotReady node still shows pods as Running for a grace period — don't trust `kubectl get pods` alone during an outage; the API view lags reality by up to the pod-eviction timeout.

Common questions

What's the difference between NotReady and SchedulingDisabled?

NotReady means the kubelet stopped reporting (or its checks failed): network, kubelet process, or runtime. SchedulingDisabled (cordoned) is a deliberate state. Uncordoning won't fix a NotReady node — fix what the kubelet is complaining about.

Why does one node flapping NotReady matter cluster-wide?

Pods on that node get evicted after the not-ready toleration window (default 300s), shifting load abruptly. A flapping node churns the whole cluster's scheduling — treat the node's kubelet/service logs as the primary incident.

Ship it right the first time

Kustomize base with probes, PDBs, and zero-downtime rollouts already wired.

Kubernetes Production Blueprints — $27 →

One-time. Yours to modify. Instant download from the NinjaOps template store.

Get new fixes by email

One short email when new fixes and production templates drop. No spam, unsubscribe anytime.