Incident Library

Organisation

Hyperconnect

Hyperconnect · hyperconnect

2 cases · 2022 to 2023

Root cause Software defect2
Component Container CPU limit1Language runtime settings1Telemetry agent1
Contributing factor Limited debug access1No resource requests or limits1
Symptom Container crash loop1Memory pressure1

Cases

  1. IL-0174 After an upgrade to Fluent Bit 2.0, logging sidecars that used the Kafka output plugin kept restarting with SIGSEGV because the plugin read a corrupted record as a map without checking its typeHyperconnect · 2023

    Root cause Software defect

  2. IL-0175 Lowering a JVM pod's CPU request from 2 to 1 pushed memory past the 90% alert because the JVM treats a cgroup v1 cpu.shares value of exactly 1024 as unset and sized Reactor Netty threads from the node's 36 CPUsHyperconnect · 2022

    Root cause Software defect