IL-0349 IONOS Cloud · 2026-01-15networking

IONOS Private Cloud in Berlin lost management access, storage and VM internet routing after a VLAN change

IONOS Cloud planned a maintenance on 15 January 2026 to introduce a new VLAN on the management devices in Berlin. The command that was run overwrote the existing VLANs on the trunk between the core and management devices instead of appending the new one, leaving only the new VLAN active. Access to several management interfaces was lost, the internal storage clusters that had interfaces on those management devices were disconnected, several VMs and Edge Nodes went read-only, and the NSX Edge Nodes lost external routing, cutting the VMs off from the internet. Restarting the affected services near the end of recovery caused a brief second outage. The status page reported the impact at 08:21 UTC, a fix and monitoring at 08:37 UTC and resolution at 09:25 UTC. IONOS planned an audit of the management device architecture for better segmentation and safeguards against unintended VLAN changes.

Severity: –
Scope: VMware-based Private Clouds: management VPN access, traffic to and from VMs, internal storage clusters, VMs and Edge Nodes in read-only mode, and external routing through NSX Edge Nodes
Users affected: –
Region: Berlin (Location DE/TXL)
Time to detect: –
Time to resolve: –

Symptoms

On 15 January 2026, network services for VMware-based Private Clouds in IONOS Cloud's Berlin location stopped working: customers lost management VPN access and traffic to and from their virtual machines. Inside IONOS, several management interfaces in the Berlin region could no longer be reached.

The loss spread from the management network to what depended on it. Internal storage clusters whose interfaces sat on the affected management devices were disconnected, and several VMs and Edge Nodes switched to read-only mode to protect their data. The NSX Edge Nodes, which connect the virtual networks to the physical network, lost external routing, so the VMs lost their internet connectivity. Late in the recovery, restarting the affected storage services caused a second, brief outage.

Symptom Network unreachable Storage disconnected or stale mounts

Root cause

Root cause Operator error

Component Network switch Transit gateway or network interconnect

Contributing factor Excessive blast radius No safeguard on destructive change

Trigger: During a planned maintenance window to add a new VLAN to the management devices, a human error in the command replaced the existing VLANs on the trunk between the core and management devices instead of adding to them, so only the new VLAN stayed active.

Detection

–

Remediation

Restarted the affected storage services in the final stage of recovery, which caused a brief second outage

Action items

Audit the management device architecture to segment it better (target Q3 2026)
Evaluate and implement ways to prevent unintended VLAN changes, such as improved configuration safeguards or automation scripts (target Q1 2026)

Lessons

–

Sources

https://status.ionos.cloud/incidents/hgt9fqjj7vvm · official-status

Related cases

IL-0568 A layer 2 loop after an access switch change broke network traffic across a whole DeNA site, with mass MACFLAP logsDeNA · 2026
same root cause · shared Network switch

Check your cluster for this

kubegap audits a Kubernetes cluster for conditions like the ones in this case, Network switch, Transit gateway or network interconnect among them, read-only, from Claude Code or Codex. Its checks cite public reports like this one.

Open kubegap

For AI tools and scripts

This case as JSON: https://incidentlib-api.benchlet.com/cases/IL-0349
The cases most like it: https://incidentlib-api.benchlet.com/cases/IL-0349/similar
MCP server (Streamable HTTP, no key): https://incidentlib-mcp.benchlet.com/mcp

The API needs no key. The MCP server offers search_incidents, get_incident, list_incidents, get_topic and vocab; both are explained on API and MCP.