Incident Library
Applied filters Root causeCloud provider outageFactorOverlapping failuresOrganisationSpotifyClear allEvery case under the filters, with no sentence; newest first.
Suggested filters Search, and the values found in your words appear here.

Graph

  • Root cause
  • Component
  • Symptom
  • Contributing factor
  • Case

Cases1 case

1 to 1 of 1

  1. IL-0252 Spotify users who were logged out could not log back in during a global outage of service discovery
    Spotify2022
    Symptoms

    At 18:12 UTC on 8 March 2022, reports began to surface of users being logged out of the different Spotify client apps. Anyone who was logged out of a Spotify app could not log back in, and the outage was global.

    High error rate
    Trigger

    Google Cloud Traffic Director, the xDS-based traffic control plane and discovery system that some Spotify services use, had an outage.

    Root cause
    Cloud provider outage
    Component
    Service discoveryHTTP and RPC client library
    Contributing factor
    Overlapping failures
    Detection

    Reports of users being logged out of the client apps started to surface at 18:12 UTC.

    Remediation
    • Rolled out configuration changes that moved the affected systems back to the DNS-based service discovery
    Action items
    • Work with Google Cloud to understand how the Traffic Director issues led to a large outage for Spotify users
    • Add monitoring and alerting that catches similar service discovery problems earlier
    • Add safety nets for monitoring, automatic error detection and self-recovery
    View case
Check your cluster for this

This case involves HTTP and RPC client library and Service discovery. kubegap audits a Kubernetes cluster for gaps like these, read-only, from Claude Code or Codex; its checks cite public reports like the ones in this library.

Open kubegap

By root cause1

Cloud provider outage1 case

Component
HTTP and RPC client libraryService discovery
Contributing factor
Overlapping failures
Organisation
Spotify
  1. IL-0252 Spotify users who were logged out could not log back in during a global outage of service discovery
    Spotify2022