Incident Library

Organisation

Fullstaq

Fullstaq · fullstaq

1 case · 2019

Root cause Capacity exhaustion
Component etcdKubernetes API serverCluster authentication provider
Contributing factor Unreviewed defaultsIgnored early signalLimited debug accessHarmful manual actionRetry storm
Symptom Control plane unresponsiveMonitoring outage

Cases

  1. IL-0076 Kubernetes masters went down after Dex filled etcd with AuthRequest objects and deleting its namespace started an endless list-and-timeout loopFullstaq · 2019

    Root cause Capacity exhaustion