Incident Library

Organisation

NU.nl

NU.nl · nu-nl

1 case · 2019

Root cause Configuration change
Component kubeletContainer memory limitPrometheus
Contributing factor Excessive blast radiusUnreviewed defaultsMonitoring blind spotNo alert on limitNo resource requests or limits
Symptom High error rateHigh latencyIntermittent timeoutsMonitoring outageNode NotReadyOOMKilled

Cases

  1. IL-0035 Worker nodes went NotReady after a new ElastAlert rule without memory limits used up node memoryNU.nl · 2019

    Root cause Configuration change