Incident Library

Organisation

Twilio

Twilio · twilio

1 case · 2013

Root cause Configuration change
Component In-memory cacheDatabase replication
Contributing factor Harmful manual actionRetry stormUnsafe failure behaviour
Symptom Data lossDatabase overloadWrites rejected

Cases

  1. IL-0215 Twilio's billing Redis master lost all balances and went read-only, and auto-recharge charged customer cards repeatedlyTwilio · 2013

    Root cause Configuration change