Incident Library
Applied filters Root causeDNS or network faultSymptomNetwork unreachableClear allEvery case under the filters, with no sentence; newest first.
Suggested filters Search, and the values found in your words appear here.

Graph

  • Root cause
  • Component
  • Symptom
  • Contributing factor
  • Case

Cases1 case

1 to 1 of 1

  1. IL-0423 A core switch failure in Paperspace's NY2 region cut VMs off their network file system and left them read-only
    Paperspace2024
    Symptoms

    On 16 June 2024, from around 10:30 UTC, users could not interact with Gradient notebooks and Core VMs in Paperspace's NY2 region. A large number of virtual machines lost network access to the Network File System they resided on and went into Read-Only (RO) mode, so customers could not write to them. Network connectivity came back by 18:20 UTC, but the affected VMs stayed unavailable in RO mode until the next day, and after Core was fixed the Gradient platform still ran with degraded performance while it recovered.

    Storage disconnected or stale mountsNetwork unreachableService outage
    Trigger

    A core switch in NY2 failed, and the resulting spike in traffic degraded network performance.

    Root cause
    DNS or network fault
    Component
    Shared network filesystem
    Contributing factor
    –
    Remediation
    • Restored network connectivity to the affected VMs by 18:20 UTC on 16 June
    • Returned the affected VMs from Read-Only to Read-Write mode between 5:58 and 12:23 UTC on 17 June
    Action items
    • Redesign the network
    • Install new equipment
    View case
Check your cluster for this

This case involves Shared network filesystem. kubegap audits a Kubernetes cluster for gaps like these, read-only, from Claude Code or Codex; its checks cite public reports like the ones in this library.

Open kubegap

By root cause1

DNS or network fault1 case

Component
Shared network filesystem
Contributing factor
–
Organisation
Paperspace
  1. IL-0423 A core switch failure in Paperspace's NY2 region cut VMs off their network file system and left them read-only
    Paperspace2024