Incident Library
  • Root cause
  • Component
  • Symptom
  • Contributing factor
  • Case

Cases1 case

1 to 1 of 1

  1. IL-0393 Load on the BBC programme metadata database surged and metadata requests failed, leaving BBC iPlayer down over a weekend
    BBC2014
    Symptoms

    At 9.30 on Saturday 19 July 2014, the load on the database of the BBC system that provides programme and clip metadata rose sharply, and many requests for metadata to its application servers started to fail. BBC iPlayer, BBC iPlayer Radio and audio and video playback on other parts of bbc.co.uk were affected.

    Database overloadHigh error rateService outage
    Root cause
    unknown
    Component
    Database
    Contributing factor
    No cachingOverlapping failures
    Remediation
    • Restored the failed pool of the caching layer first
    • Switched the BBC homepage to its emergency mode until the cache was stable
    • Did significant remedial work on the metadata system from Saturday afternoon into the evening
    • Ran the service in a reduced state through the weekend and did the full restoration on Monday morning
    Action items
    • Complete the forensics on the root causes and put measures in place to reduce the chance of similar interruptions
    View case
Check your cluster for this

This case involves Database. kubegap audits a Kubernetes cluster for gaps like these, read-only, from Claude Code or Codex; its checks cite public reports like the ones in this library.

Open kubegap

By root cause1

unknown1 case

Component
Database
Contributing factor
No cachingOverlapping failures
Organisation
BBC
  1. IL-0393 Load on the BBC programme metadata database surged and metadata requests failed, leaving BBC iPlayer down over a weekend
    BBC2014