Cases1 case

1 to 1 of 1

1. IL-0081 API requests timed out during deployment rollouts because the ingress-nginx enable-serial-reloads option held back endpoint updates while a long NGINX reload was running

   Cresta2025

   Symptoms

   During a deployment, traffic into the Kubernetes cluster started hitting dead ends: some API requests timed out and requests were slow and disrupted while the rollout ran.

   Intermittent timeoutsStale endpoint routingHigh latency

   Trigger

   A deployment rollout changed backend endpoints while a long-running NGINX full reload was in progress, with the enable-serial-reloads option enabled on the ingress-nginx controller.

   Root cause

   Software defect

   Component

   Ingress controller

   Contributing factor

   –

   Remediation

   • restarted the NGINX Ingress controller as an attempt to resolve the problem

   Action items

   • Open an issue in kubernetes/ingress-nginx describing how enable-serial-reloads blocks dynamic endpoint updates

   Lessons

   A flag meant only to serialise full reloads also delayed endpoint updates, which opened a window during rollouts in which the ingress routed to pods that no longer existed. Options that change how the ingress controller applies configuration need to be checked against what happens to endpoint updates during a long reload.

   View case

By root cause1