Incident Library
Applied filters Root causeDNS or network faultComponentDatabase replicationSymptomIntermittent timeoutsClear allEvery case under the filters, with no sentence; newest first.
Suggested filters Search, and the values found in your words appear here.

Graph

  • Root cause
  • Component
  • Symptom
  • Contributing factor
  • Case

Cases1 case

1 to 1 of 1

  1. IL-0590 F5 load balancers cached a path MTU of 296 after Rackspace reboots, and Mailgun connections failed intermittently
    Mailgun2014
    Symptoms

    On Saturday, September 27, 2014 at 6:50 p.m. UTC, Mailgun saw that all its database replicas and the internal traffic between its environments in all regions were broken and reporting timeouts. Once Rackspace began rebooting cloud instances in its Chicago data center (ORD), Mailgun customers had intermittent connection failures.

    Intermittent connection failuresIntermittent timeoutsPacket loss
    Trigger

    Rackspace, Mailgun's parent company and hosting provider, rebooted cloud instances in its Chicago data center (ORD), and the cloud maintenance triggered an error condition on Mailgun's F5 load balancers.

    Root cause
    DNS or network fault
    Component
    Cloud load balancerDatabase replication
    Contributing factor
    –
    Detection

    Mailgun noticed at 6:50 p.m. UTC on September 27 that all database replicas and the internal traffic between its environments were broken and reporting timeouts.

    Remediation
    • The Rackspace networking team cleared the cached path MTU on Mailgun's F5s and on some Rackspace dedicated customers' F5s
    • Set up a new virtual IP for Mailgun services
    • Changed Mailgun's DNS settings, which forced the remote F5s to clear their caches
    Action items
    • Finish the root cause analysis of the F5 path MTU problem with Rackspace
    • Make sure maintenance of Mailgun's cloud infrastructure has no impact on Mailgun customers
    View case
Check your cluster for this

This case involves Cloud load balancer and Database replication. kubegap audits a Kubernetes cluster for gaps like these, read-only, from Claude Code or Codex; its checks cite public reports like the ones in this library.

Open kubegap

By root cause1

DNS or network fault1 case

Component
Cloud load balancerDatabase replication
Contributing factor
–
Organisation
Mailgun
  1. IL-0590 F5 load balancers cached a path MTU of 296 after Rackspace reboots, and Mailgun connections failed intermittently
    Mailgun2014