Incident Library

Organisation

Neon

Neon · neon

1 case · 2025

Root cause Capacity exhaustion
Component Cluster autoscalerVPC subnet or pod address rangeAWS VPC CNI
Contributing factor Unverified assumptionUnreviewed defaultsHarmful manual actionNo capacity planningRetry stormIneffective rollback
Symptom IP address exhaustionPods Pending

Cases

  1. IL-0006 VPC subnets ran out of IP addresses because the AWS VPC CNI kept IPs on full nodes, and WARM_IP_TARGET changes then blocked pod starts after every pod deletionNeon · 2025

    Root cause Capacity exhaustion