Incident Library

Organisation

Google

Google · google

1 case · 2024

Root cause Hardware failure
Component NIC, cable and switch port linkTPU
Contributing factor –
Symptom Throughput degradationTraining job failure

Cases

  1. IL-0418 Machine, ICI link and optical circuit switch failures interrupted training jobs on Google's TPU v4 supercomputersGoogle · 2024

    Root cause Hardware failure