Cases1 case
1 to 1 of 1
IL-0183kubeadm's one-year leaf certificates expired, so kube-apiserver could not reach etcd, restarted in a loop and left the cluster unreachableckck8032026- Symptoms
The cluster could not be reached from outside. On the control plane node, sudo systemctl status kubelet showed TLS handshake errors such as "http: TLS handshake error from 101.198.0.183:19517: tls: client requested unsupported application protocols" and "http: TLS handshake error from 101.198.0.172:36328: tls: client offered only unsupported versions: [302 301]".
- Trigger
The leaf certificates that kubeadm had generated for the cluster, valid for one year by default, expired at 2026-02-20 16:16 UTC.
- Root cause
- Component
- Contributing factor
- Detection
The author could not connect to the cluster from outside and went to the control plane node to read the kubelet logs.
- Remediation
- renewed all expired certificates with sudo kubeadm certs renew all
- restarted containerd and the kubelet so the static control plane pods restarted with the new certificates
- copied the renewed /etc/kubernetes/admin.conf to $HOME/.kube/config
- Lessons
A kubeadm cluster stops working one year after its leaf certificates were issued unless they are renewed. The first errors seen may be TLS handshake failures or a flag error that does not mention certificates; the decisive message is the x509 expiry in the kube-apiserver log for its connection to etcd. The author recommends checking expiry with kubeadm certs check-expiration and renewing from 30 days before. kubeadm upgrade apply renews the certificates too, so a cluster upgraded at least once a year avoids the problem; a cluster run for a long time without upgrades can renew from cron, for example with 0 3 1 1 * root kubeadm certs renew all && systemctl restart kubelet in /etc/cron.d/kubeadm-cert-renew.
This case involves Calico and etcd. kubegap audits a Kubernetes cluster for gaps like these, read-only, from Claude Code or Codex; its checks cite public reports like the ones in this library.