Incident Library

Organisation

Woowa Brothers

Woowa Brothers · woowa-brothers

3 cases · 2016 to 2020

Root cause Capacity exhaustion1Operator error1Software defect1
Component Database2Database backups and snapshots1Client-side DNS resolver and cache1NAT gateway1On-premises load balancer1
Contributing factor Unpatched known bug1Unknown limit1
Symptom High error rate2Data loss1Intermittent connection failures1Outbound connection exhaustion1Service outage1

Cases

  1. IL-0455 Calls from an AWS service through NAT Gateway to an on-premises service returned 5xx errors every busy weekendWoowa Brothers · 2020

    Root cause Capacity exhaustion

  2. IL-0573 A developer's mistake dropped nine main tables of the Baemin billing MariaDB database and payments briefly stoppedWoowa Brothers · 2016

    Root cause Operator error

  3. IL-0454 Baemin billing servers could not open new MariaDB connections at the start of a card-discount eventWoowa Brothers · 2016

    Root cause Software defect