Incident Library

Organisation

Inflab

Inflab · inflab

1 case · 2025

Root cause Software defect
Component GPU driverKarpenterNode operating system image
Contributing factor Unverified assumptionAutomation amplification
Symptom GPU unavailableNode recreation loop

Cases

  1. IL-0171 Karpenter node auto-repair replaced EKS GPU nodes about 10 minutes after they became Ready, in a loop, because the Bottlerocket NVIDIA AMI lacked libdcgm.so and set AcceleratedHardwareReady to FalseInflab · 2025

    Root cause Software defect