Incident Library
Applied filters ComponentManaged Kubernetes control plane and provider APIFactorExcessive blast radiusOrganisationIONOS CloudClear allEvery case under the filters, with no sentence; newest first.
Suggested filters Search, and the values found in your words appear here.

Graph

  • Root cause
  • Component
  • Symptom
  • Contributing factor
  • Case

Cases1 case

1 to 1 of 1

  1. Symptoms

    From 14 to 16 March 2026 a subset of customer control planes on IONOS Managed Kubernetes (MK8s) became unavailable again and again, each time for several minutes. While a cluster was unreachable, operations that depend on the Kubernetes API, such as deployments, scaling and health checks, did not work reliably.

    Root cause
    Component
    Contributing factor
    Remediation
    • Defragmented the affected database instances, reclaiming about 3–4 GiB per instance
    • Shortened the database compaction intervals on the affected instances
    • Deployed a fix for the affected AI Model Hub database services
    • Applied final changes to all affected clusters
    Action items
    • Expand monitoring dashboards to show control plane health per database instance
    • Move the highest-volume clusters to dedicated database backends
    • Make incidents of this kind trigger immediate paging
    • Roll the updated compaction interval out to all shared control plane database instances
    • Run automated, scheduled defragmentation jobs on all instances
    • Rebalance clusters across database instances to reduce concentration
    • Contact customers whose clusters write disproportionate data, for example to store large security scan reports outside the control plane database
    • Introduce per-tenant resource quotas and key-count limits on the shared control plane database
    • Re-architect the shared control plane for stronger isolation, with dedicated database instances and better vertical scaling
    • Evaluate full database sharding for customers with consistently high data volumes
    • Hold regular drills on database resource exhaustion and recovery
Check your cluster for this

This case involves etcd and Kubernetes API server. kubegap audits a Kubernetes cluster for gaps like these, read-only, from Claude Code or Codex; its checks cite public reports like the ones in this library.

By root cause1