Incident Library

IL-0464 IONOS Cloud · 2026-03-14kubernetes

IONOS Managed Kubernetes control planes went unreachable for minutes at a time as their shared etcd grew and fragmented

IONOS Managed Kubernetes keeps the state of many customer control planes in shared etcd databases. A few clusters stored unusually large amounts of data there, mainly security scanning reports and policy audit records, along with many event and autoscaling objects; on one affected database instance three clusters held about 64% of the stored data. The compaction interval that reclaims unused space could not keep up with the write rate, so old revisions piled up and the database grew past its normal size, and fragmentation added roughly 40% of allocated space beyond the data in use. This pushed the database toward its hard storage limit, where it would have gone read-only and taken down every control plane on that instance. Together these caused stalls during database maintenance cycles that disrupted the Kubernetes API server for several minutes each time.

On 16 March the team identified a resource constraint within etcd as the root cause. It defragmented the affected database instances, reclaiming about 3–4 GiB each, shortened their compaction intervals and rolled changes out to the affected clusters; DBaaS, the AI Model Hub and the Container Registry recovered during the day, and the final changes were applied to all affected clusters by 19:59 UTC. The incident was marked resolved on 17 March.

Severity
–
Scope
A subset of customer Managed Kubernetes control planes periodically unreachable; DBaaS (PostgresDB on Kubernetes), the AI Model Hub and the Container Registry were also affected
Users affected
–
Region
DE/FRA (Managed Kubernetes)
Time to detect
–
Time to resolve
–

Symptoms

From 14 to 16 March 2026 a subset of customer control planes on IONOS Managed Kubernetes (MK8s) became unavailable again and again, each time for several minutes. While a cluster was unreachable, operations that depend on the Kubernetes API, such as deployments, scaling and health checks, did not work reliably.

On 16 March the incident widened to other services: an increased error count from PostgresDB on Kubernetes in DBaaS, the AI Model Hub, and the Container Registry, where customers had trouble pulling and pushing images. The stalls came during database maintenance cycles of the shared control plane database, and the team described the remaining problem as recurring load spikes.

Symptom Control plane unresponsiveImage pull failures

Root cause

Root cause Capacity exhaustion

Component etcdKubernetes API serverManaged Kubernetes control plane and provider API
Contributing factor Excessive blast radius
Trigger
–

Detection

–

Remediation

Action items

Lessons

–

Sources

Related cases

Check your cluster for this

kubegap audits a Kubernetes cluster for conditions like the ones in this case, etcd, Kubernetes API server, Managed Kubernetes control plane and provider API among them, read-only, from Claude Code or Codex. Its checks cite public reports like this one.

Open kubegap

For AI tools and scripts

The API needs no key. The MCP server offers search_incidents, get_incident, list_incidents, get_topic and vocab; both are explained on API and MCP.