Incident Library

API · MCP 한국어

API and MCP kubegap Corrections and proposals

Incident Library

API · MCP 한국어

Component

CoreDNS

CoreDNS · coredns

12 cases · 2019 to 2026

Root cause Capacity exhaustion8 DNS or network fault2 Configuration change1 Software defect1

Contributing factor Unreviewed defaults6 Unknown limit4 Automation amplification3 No caching3

Symptom DNS resolution failures10 High error rate5 High latency4 OOMKilled4

Organisation Adevinta1 Backpackr1 Buildkite1 Cisco ThousandEyes1 DigitalOcean

Cases, by severity

IL-0102 CoreDNS pods on a production EKS cluster ran out of memory under a burst of new pods, endpoints and nodes, taking down the Buildkite web UI, APIs and job dispatchBuildkite · 2026
Root cause Capacity exhaustion
IL-0155 Some DOKS clusters in NYC1 saw intermittent DNS timeouts and NodeNotReady events because CoreDNS ran on shared-CPU Droplet worker nodesDigitalOcean · 2026
Root cause Capacity exhaustion
IL-0104 After a CoreDNS pod was OOMKilled, the talk names three causes: Istio's aggressive DNS cache refresh on one reused client port, kube-proxy not cleaning up conntrack entries of terminated pods, and the 2-minute UDP conntrack timeout

View on the graph

Ignored early signal2

Harmful manual action2

Monitoring blind spot2

Unverified assumption1

Excessive blast radius1

Change during peak hours1

No alert on limit1

No configuration tests1

Missed in review1

Single point of failure1

Service outage3

Autoscaler misbehaviour2

Intermittent connection failures2

Intermittent timeouts2

Stale endpoint routing2

Control plane unresponsive1

Deploys failing1

Image pull failures1

Monitoring outage1

IP address exhaustion1

Node disk full1

Pods stuck in ContainerCreating1

1

UK Ministry of Justice1

Cisco ThousandEyes · 2025

Root cause Software defect

IL-0048 DNS lookups timed out in a large EKS cluster when CoreDNS hit the 1024 packets-per-second link-local limit, driven by sudo hostname lookupsunknown · 2025

Root cause Capacity exhaustion

IL-0082 Losing one CoreDNS pod during a planned EKS node reroll overloaded the remaining CoreDNS pods, causing DNS timeouts and errors across multiple services at RebelMouseRebelMouse · 2025

Root cause Capacity exhaustion

IL-0027 All CoreDNS pods landed on one node, the node ran out of memory, and DNS for the whole service went downBackpackr · 2025

Root cause Capacity exhaustion

IL-0204 An etcd memory spike overloaded the control plane of a Render Frankfurt cluster, and DNS then failed because CoreDNS ran on the rebooted control plane nodes and could not start without the API serverRender · 2024

Root cause Capacity exhaustion

IL-0182 Too few CoreDNS replicas on EKS were overloaded during traffic spikes, raising latency and cutting the network for task pods, and the over-provisioned, tightly spread CoreDNS pods added later blocked Karpenter consolidation이언철 · 2024

Root cause Capacity exhaustion

IL-0019 An EKS Terraform module upgrade shrank node root disks from 100 GB to 20 GB and image pulls failed with no space left on deviceUK Ministry of Justice · 2023

Root cause Configuration change

IL-0083 Intermittent 5xx errors in a multi-tenant Kubernetes cluster came from internal DNS: the node-local dnsmasq hit its 150 concurrent query limit, did not cache cluster.local answers, and ndots search lookups flooded CoreDNSAdevinta · 2023

Root cause DNS or network fault

IL-0049 Partial DNS outage after CoreDNS scaled down and kube-proxy failed to delete stale conntrack entriesPreply · 2020

Root cause DNS or network fault

IL-0001 Cluster-wide DNS outage after CoreDNS pods were OOM-killed again and again under a burst of DNS queriesZalando · 2019

Root cause Capacity exhaustion