Incident Library
Applied filters Root causeDependency failureComponentDatabaseSymptomHigh error rateClear allEvery case under the filters, with no sentence; newest first.
Suggested filters Search, and the values found in your words appear here.

Graph

  • Root cause
  • Component
  • Symptom
  • Contributing factor
  • Case

Cases1 case

1 to 1 of 1

  1. IL-0408 OpenRouter's API returned 500s and false 401 "User not found" errors when its API key cache dropped and came back empty
    OpenRouter2026
    Symptoms

    At 5:27 AM UTC on 17 February 2026 part of OpenRouter's users started getting 500 errors on all API endpoints, and from 5:40 AM many requests failed with 401 "User not found" errors although their API keys were valid. The chatroom was degraded at the same time. The same pattern came back at 7:36 AM UTC on 19 February.

    High error rateService outageDatabase overload
    Trigger

    The third-party caching layer in front of API key lookups dropped all of its connections to OpenRouter's database and began returning errors.

    Root cause
    Dependency failure
    Component
    In-memory cacheDatabase
    Contributing factor
    Dependence on a warm cacheOverlapping failures
    Detection

    An internal alert fired one minute after impact began on 17 February, at 5:28 AM UTC, and two minutes after it began on 19 February, at 7:38 AM UTC, and the response team came online to investigate.

    Remediation
    • Deployed WAF rules to block the denial-of-service attack running at the same time on 17 February
    Action items
    • Add circuit breakers that detect caching layer failures, limit their blast radius and repopulate a cold cache in a controlled way
    • Return 503 instead of 401 when an authorization lookup cannot complete because of an infrastructure problem
    • The caching provider fixes the failure mode that caused both incidents
    • Roll out a fallback caching mechanism so OpenRouter survives even extended downtime of the caching layer
    Lessons

    Answering an infrastructure failure with an authentication error made customers debug API keys that were fine, so error codes have to say what actually failed. Attributing the first outage to a concurrent denial-of-service attack, because the caching dependency had always been reliable, left the real fault in place until it recurred two days later. OpenRouter holds that redundancy against failures in its dependencies is its own responsibility, not the provider's.

    View case
Check your cluster for this

This case involves In-memory cache and Database. kubegap audits a Kubernetes cluster for gaps like these, read-only, from Claude Code or Codex; its checks cite public reports like the ones in this library.

Open kubegap

By root cause1

Dependency failure1 case

Component
In-memory cacheDatabase
Contributing factor
Overlapping failuresDependence on a warm cache
Organisation
OpenRouter
  1. IL-0408 OpenRouter's API returned 500s and false 401 "User not found" errors when its API key cache dropped and came back empty
    OpenRouter2026