Incident Library
Applied filters ComponentCI systemOrganisationGitHubClear allEvery case under the filters, with no sentence; newest first.
Suggested filters Search, and the values found in your words appear here.

Graph

  • Root cause
  • Component
  • Symptom
  • Contributing factor
  • Case

Cases1 case

1 to 1 of 1

  1. IL-0471 GitHub Actions runs failed or sat queued for 9 hours after a routine deployment's pod replacement saturated capacity
    GitHub2026
    Symptoms

    From 15:05 UTC on 6 August 2026 GitHub Actions workflow runs failed to start, failed partway through or stayed queued for a long time, on both GitHub-hosted and self-hosted runners. Some requests to the Actions REST API returned errors, and some customers saw unexpected rate limiting in their workflows.

    High error rateHigh latency
    Trigger

    A routine deployment to an internal Actions service that processes events and generates Actions jobs replaced pods, and the remaining capacity became saturated.

    Root cause
    Capacity exhaustion
    Component
    CI system
    Contributing factor
    No headroomExcessive blast radiusHarmful manual action
    Remediation
    • Expanded capacity of the affected services
    • Throttled incoming webhook-triggered work
    • Increased processing capacity for the backlog of affected events
    • Deployed changes that stopped runners from repeatedly trying to acquire invalid jobs
    • Deployed a fix for self-hosted runners that were not picking up jobs and enabled it incrementally
    • Paused migrations with GitHub Enterprise Importer
    • Restored webhook-triggered Actions workflows to full throughput
    • Rolled back the mitigation that left some ARC runners offline, and recovered those runners by hand
    • Shared CLI and UI steps for customers to clear jobs that could not be retried or canceled
    Action items
    • Improve deployment and capacity safeguards for the affected services
    • Strengthen monitoring for the conditions that preceded the incident
    • Improve the resiliency and recovery of queued work and runner assignment
    • Add automatic recovery for self-hosted runners in upcoming Runner and ARC releases
    • Reduce the risk of cascading failures and speed up recovery during large Actions disruptions
    View case
Check your cluster for this

This case involves CI system. kubegap audits a Kubernetes cluster for gaps like these, read-only, from Claude Code or Codex; its checks cite public reports like the ones in this library.

Open kubegap

By root cause1

Capacity exhaustion1 case

Component
CI system
Contributing factor
Excessive blast radiusHarmful manual actionNo headroom
Organisation
GitHub
  1. IL-0471 GitHub Actions runs failed or sat queued for 9 hours after a routine deployment's pod replacement saturated capacity
    GitHub2026