IL-0425 Linode · 2026-06-23ai-infra

GPU Linodes in all regions failed to boot intermittently when several started at once after a global software deployment

After a global software deployment, Linode found on June 23, 2026 that GPU Linodes were failing to boot from time to time. The investigation showed that the software update made GPU servers that started at exactly the same time block one another from loading, and the platform did not retry the boot automatically. The conflict only appears under heavy, simultaneous load, so Linode's standard pre-release testing did not catch it. On June 24 at 20:32 UTC Linode deployed a hotfix directly to every active GPU host in its fleet, and after a day of monitoring without further boot failures the incident was closed on June 25. The postmortem, posted on June 26, says the investigation was still ongoing.

Severity: –
Scope: GPU Linodes that booted at the same time as other GPU Linodes failed to boot intermittently, causing local disruption or higher error rates, mostly during automated node recycling and scaling events
Users affected: –
Region: All regions (the status page lists 43)
Time to detect: –
Time to resolve: –

Symptoms

On June 23, 2026, Linode identified intermittent boot failures on GPU Linodes in all regions. Only instances that booted at the same time as other GPU Linodes were affected, and customers could see local disruption or higher error rates, mostly during automated node recycling and scaling events. The Linode status page posted the issue as an emerging service issue on June 24 at 18:53 UTC, posted an update at 20:00 UTC, moved to monitoring at 21:12 UTC after the fix at 20:32 UTC, and marked the incident resolved on June 25 at 19:29 UTC after no further boot failures were seen.

Symptom GPU unavailable High error rate

Root cause

Root cause Faulty deployment

Component –

Contributing factor Unrepresentative staging

Trigger: A global software deployment introduced a conflict that showed up when several GPU servers started at exactly the same moment.

Detection

–

Remediation

Deployed a hotfix directly to all active GPU hosts in the fleet at 20:32 UTC on June 24, 2026

Action items

Ship a full fix in the next software release, rolling out globally over the following week
Procure dedicated GPU testing hardware for the development cloud so concurrent GPU workloads are simulated before updates reach production

Lessons

The conflict only happens when many GPU servers start at once under heavy load, which the standard pre-release testing did not reproduce; testing needs hardware that can simulate concurrent GPU workloads before an update reaches production.

Sources

https://status.linode.com/incidents/1h0hwggm668b · official-status · archived copy

Related cases

–

Check your cluster for this

kubegap audits a Kubernetes cluster for conditions like the ones in this case, read-only, from Claude Code or Codex. Its checks cite public reports like this one.

Open kubegap

For AI tools and scripts

This case as JSON: https://incidentlib-api.benchlet.com/cases/IL-0425
The cases most like it: https://incidentlib-api.benchlet.com/cases/IL-0425/similar
MCP server (Streamable HTTP, no key): https://incidentlib-mcp.benchlet.com/mcp

The API needs no key. The MCP server offers search_incidents, get_incident, list_incidents, get_topic and vocab; both are explained on API and MCP.