Incident Library

Organisation

Barush Mendez

Barush Mendez · barush-mendez

1 case · 2025

Root cause Software defect
Component GPU driverKubernetes API serverLinux kernel
Contributing factor Change with a delayed effectIgnored early signalMonitoring blind spotSingle point of failure
Symptom Control plane unresponsiveMonitoring outageService outage

Cases

  1. IL-0138 A failed Nvidia driver build left broken kernel packages on the only Kubernetes master node, which hit a kernel panic on reboot after a power outage and took the homelab cluster down for six daysBarush Mendez · 2025

    Root cause Software defect