Incident Library

API · MCP 한국어

API and MCP kubegap Corrections and proposals

Incident Library

API · MCP 한국어

Component

Container runtime

컨테이너 런타임 · container-runtime

6 cases · 2017 to 2026

Root cause Capacity exhaustion2 Configuration change2 Dependency failure1 unknown1

Contributing factor Unreviewed defaults3 Unverified assumption2 Automation amplification2 Excessive blast radius2

Symptom Service outage3 Node NotReady2 Connection pool exhaustion1 Credentials unavailable1

Organisation DFDS1 Fyra Labs1 MUSINSA1 Onkar Raghunath Shelke1 Saltside

Cases, by severity

IL-0109 A retried node rollover on a DFDS EKS cluster set the Auto Scaling group back to 15 instances, and the default termination policy removed every healthy worker node, leaving the cluster with no nodesDFDS · 2023 · sev1
Root cause Configuration change
IL-0073 Intermittent Kafka connectivity overloaded Docker daemons, set off a node rescheduling cascade and poisoned the Consul gossip meshTarget · 2019 · sev1
Root cause Dependency failure
With DEBUG logging and synchronous log output, a high-traffic service filled the 64 KB stdout pipe, so request threads blocked in write() while holding database connections and HikariCP ran dry, failing requests in 30-second bursts

View on the graph

Harmful manual action2

Monitoring blind spot2

Single point of failure2

Ignored early signal1

Unpatched known bug1

No configuration tests1

Unrepresentative staging1

High error rate1

Image pull failures1

Intermittent timeouts1

Monitoring outage1

Node disk full1

Sustained high node CPU1

Resources deleted1

Load imbalance1

1

IL-0179

MUSINSA · 2026

Root cause Configuration change

IL-0071 Worker node running out of inodes while disk space looked fine, because containerd kept a full node_modules tree in each of several image layer snapshotsOnkar Raghunath Shelke · 2026

Root cause Capacity exhaustion

IL-0087 After a cold reboot of RKE2 nodes, pods could not get IP addresses because Multus could not reach the Kubernetes API, and further reboots spread the failure to every nodeFyra Labs · 2023

Root cause unknown

IL-0040 Parallel Helm installs saturated small gp2 root volumes on a kops cluster and left all three workers NotReady with the kubelet or Docker downSaltside · 2017

Root cause Capacity exhaustion