Incident Library
Applied filters Root causeConfiguration changeComponentCloud load balancerSymptomLoad imbalanceClear allEvery case under the filters, with no sentence; newest first.
Suggested filters Search, and the values found in your words appear here.

Graph

  • Root cause
  • Component
  • Symptom
  • Contributing factor
  • Case

Cases1 case

1 to 1 of 1

  1. IL-0543 Twingate API traffic piled onto one backend behind the global load balancer, failing core services for 29 minutes
    Twingate2025
    Symptoms

    At 20:45 UTC on 28 May 2025, most of Twingate's API traffic started going to a single backend service, which was overwhelmed and returned elevated error rates, so API calls failed widely across core services.

    Misrouted requestsLoad imbalanceHigh error rate
    Trigger

    A rollout that updated the routing rules of Twingate's global load balancer.

    Root cause
    Configuration change
    Component
    Cloud load balancer
    Contributing factor
    Unrepresentative stagingCircular dependency
    Detection

    The incident was detected at 20:48 UTC, three minutes after API traffic was first misrouted, and triage began.

    Remediation
    • Started a rollback of the routing change at 20:52 UTC; the automated rollback stalled on infrastructure dependencies
    • Applied a manual fix with local tooling at 21:09 UTC, which restored service by 21:14 UTC
    Action items
    • Correct the routing rule
    • Add alerts that detect backend traffic anomalies in lower environments
    • Improve resource scaling thresholds
    • Decouple infrastructure tooling from service dependencies so recovery has faster options
    • Introduce backend sharding to isolate customer workloads and reduce the impact of incidents
    View case
Check your cluster for this

This case involves Cloud load balancer. kubegap audits a Kubernetes cluster for gaps like these, read-only, from Claude Code or Codex; its checks cite public reports like the ones in this library.

Open kubegap

By root cause1

Configuration change1 case

Component
Cloud load balancer
Contributing factor
Circular dependencyUnrepresentative staging
Organisation
Twingate
  1. IL-0543 Twingate API traffic piled onto one backend behind the global load balancer, failing core services for 29 minutes
    Twingate2025