Incident Library

Organisation

Databricks

Databricks · databricks

1 case · 2026

Root cause Dependency failure
Component InfiniBand fabricNCCLPyTorch
Contributing factor Unreviewed defaultsMonitoring blind spot
Symptom Link flappingNCCL timeoutTraining job failure

Cases

  1. IL-0023 A training run crashed with an NCCL timeout after one InfiniBand port went down once and recoveredDatabricks · 2026

    Root cause Dependency failure