Incident Library

Organisation

MindTickle

MindTickle · mindtickle

2 cases · 2019 to 2020

Root cause DNS or network fault2
Component conntrack table1Client-side DNS resolver and cache1Linux kernel1AWS VPC CNI1
Contributing factor Unreviewed defaults2Unverified assumption1
Symptom High latency2DNS resolution failures1Intermittent connection failures1Intermittent timeouts1

Cases

  1. IL-0056 Calls from EKS pods to services behind an NLB in another VPC failed intermittently because the AWS VPC CNI did source NAT on the nodeMindTickle · 2020

    Root cause DNS or network fault

  2. IL-0047 Pod-to-pod requests delayed by 5, 10 or 15 seconds because of DNS lookups lost to a conntrack raceMindTickle · 2019

    Root cause DNS or network fault