Incident Library
Applied filters Root causeFaulty deploymentComponentService discoverySymptomIntermittent connection failuresClear allEvery case under the filters, with no sentence; newest first.
Suggested filters Search, and the values found in your words appear here.

Graph

  • Root cause
  • Component
  • Symptom
  • Contributing factor
  • Case

Cases1 case

1 to 1 of 1

  1. IL-0371 Half of new connections to Render Postgres databases with point-in-time recovery failed while their backup jobs ran
    Render2023
    Symptoms

    After Point-in-time Recovery (PITR) was released to eligible new Render Postgres databases on 21 April 2023, 50% of new connections to PITR-enabled databases failed while a PITR backup job was running. A backup takes 15 to 45 minutes depending on database size, and existing connections, such as those held through connection pooling, kept working.

    Intermittent connection failuresMisrouted requests
    Trigger

    Point-in-time Recovery was released to eligible new databases on 21 April 2023, and its backup job carried an incorrect networking configuration.

    Root cause
    Faulty deployment
    Component
    DatabaseDatabase backups and snapshotsService discovery
    Contributing factor
    –
    Remediation
    • Disabled backups for affected databases until they could be scheduled in low-traffic hours
    • Changed the backup job's network labels so the routing layer no longer picks it up as a database
    Action items
    • Tighten the process for sharing incident updates so problems are not declared resolved too early
    • Use more specific network labels for the database routing layer so ineligible IPs are never added
    • Run a skipped asynchronous job in the next scheduled window instead of immediately
    • Use open source benchmarking software to reproduce issues customers notice
    View case
Check your cluster for this

This case involves Database and Database backups and snapshots. kubegap audits a Kubernetes cluster for gaps like these, read-only, from Claude Code or Codex; its checks cite public reports like the ones in this library.

Open kubegap

By root cause1

Faulty deployment1 case

Component
DatabaseDatabase backups and snapshotsService discovery
Contributing factor
–
Organisation
Render
  1. IL-0371 Half of new connections to Render Postgres databases with point-in-time recovery failed while their backup jobs ran
    Render2023