Incident Library
Applied filters ComponentNode poolSymptomScale-up failureOrganisationLinodeClear allEvery case under the filters, with no sentence; newest first.
Suggested filters Search, and the values found in your words appear here.

Graph

  • Root cause
  • Component
  • Symptom
  • Contributing factor
  • Case

Cases2 cases

1 to 2 of 2

  1. Symptoms

    Linodes deployed with StackScripts failed to deploy and could not boot. Every newly created Linode Kubernetes Engine (LKE) and LKE-E node relies on StackScripts, so LKE autoscaling and node recycling failed as well, in all regions.

    Trigger

    A recent update on the compute hosts removed a credential parser that is used when decoding StackScripts.

    Root cause
    Component
    Contributing factor
    –
    Detection

    The issue was identified at 18:18 UTC on 2026-05-15. The first status post went up at 00:42 UTC on 2026-05-16.

    Remediation
    • tested and applied a rollback starting at 01:10 UTC on 2026-05-16, and failures dropped from 01:20 UTC
    Lessons

    Removing a credential parser from the compute hosts broke the boot of every Linode deployed with StackScripts, and every new LKE and LKE-E node is one of them, so node creation, autoscaling and node recycling stopped in all regions. The rollback reduced failures ten minutes after testing began.

  2. Symptoms

    From 11:45 UTC on 16 April 2026, customers of Linode Kubernetes Engine (LKE) and Linode Kubernetes Engine Enterprise (LKE-E) could not deploy new clusters or add new nodes to existing clusters. Because no new node could be deployed, node recycling and cluster autoscaling did not work either. The problem hit every region where LKE runs.

    Trigger

    A credential expired, which caused authentication failures when LKE and LKE-E deployed clusters and nodes. One status update describes it as an expired certificate.

    Root cause
    Component
    Contributing factor
    –
    Detection

    The first status page update, saying the team was investigating an emerging issue affecting LKE and LKE-E in all regions, was posted at 13:20 UTC.

    Remediation
    • created new credentials and deployed them, rotating the expired credential (certificate)
    Action items
    • Investigate what led to the credential expiry and take preventive actions
    Lessons

    One expired credential stopped every operation that needs a new node, in all regions at once: new clusters, added nodes, node recycling and autoscaling.

Check your cluster for this

These 2 cases involve Node pool and Cluster autoscaler. kubegap audits a Kubernetes cluster for gaps like these, read-only, from Claude Code or Codex; its checks cite public reports like the ones in this library.

By root cause2