Cases2 cases

1 to 2 of 2

1. IL-0465 IONOS Container Registry image pushes and pulls failed with 504 Gateway Timeout

   IONOS Cloud2026

   Symptoms

   On 4 March 2026 customers of the IONOS Container Registry got 504 Gateway Timeout errors when pushing or pulling container images, and deployments that relied on the registry were blocked. The status page first reported an increased error count on the registry at 09:23 UTC, saying customers might be unable to pull images.

   Image pull failuresHigh error rate

   Trigger

   A migration combined with a temporary capacity constraint placed two critical control plane components on the same proxy instance, despite anti-affinity rules; the shared proxy then reached its concurrent connection limit.

   Root cause

   Capacity exhaustion

   Component

   Container registry and its credentialsManaged Kubernetes control plane and provider APIPod scheduling and placement

   Contributing factor

   No headroomSingle point of failureNo alert on limit

   Remediation

   • Rolled back a component of the Kubernetes control plane to an earlier version
   • Provisioned additional proxy capacity
   • Moved the affected control plane components onto separate proxy instances

   Action items

   • Redesign registry-to-API connectivity so each node uses its own local proxy
   • Adjust alert thresholds to warn before proxy connection use approaches capacity
   • Deploy more infrastructure clusters and redistribute registries so no cluster exceeds safe capacity, with automation that keeps balancing load

   View case
2. IL-0156 Revoking old certificates after a rotation left a certificate key mismatch between API endpoints and the cluster controller, and LKE cluster management and autoscaling in Atlanta returned 5xx errors

   Linode2025

   Symptoms

   From about 02:29 UTC on 24 September 2025, customers of Linode Kubernetes Engine (LKE) in US-Southeast (Atlanta) could not manage their clusters, and autoscaling was impacted. Internal logs showed 5xx errors whenever customers tried to manage LKE clusters associated with the affected cluster controller.

   High error rateAutoscaler misbehaviour

   Trigger

   Old certificates on the LKE cluster controllers were revoked at 02:29 UTC on 24 September 2025, after the expiring certificates had been rotated on 23 September.

   Root cause

   Certificate or credential expiry

   Component

   Managed Kubernetes control plane and provider APICluster autoscaler

   Contributing factor

   Change with a delayed effect

   Remediation

   • deployed a fix for the certificate key mismatch
   • restarted the API endpoints

   Action items

   • Continue investigating the root cause and take preventive actions

   View case

By root cause2