Incident Library

Applied filters ComponentDatabaseSymptomConnection pool exhaustionOrganisationStorjClear allEvery case under the filters, with no sentence; newest first.

Suggested filters Search, and the values found in your words appear here.

Graph

Root cause
Component
Symptom
Contributing factor
Case

Cases1 case

1 to 1 of 1

IL-0557 Database transaction locks piled up on Storj's US1 satellite, and uploads and downloads failed with timeouts for 2 hours
Storj2025
Symptoms
From 21:04 UTC on 7 August 2025, database transaction locks on the Storj US1 satellite started to increase, and upload and download requests began to fail. The on-call team was paged at 21:14 UTC.
Lock contention or deadlock Intermittent timeouts Connection pool exhaustion High error rate
Trigger
An unusually high volume of concurrent uploads to the same objects reached the US1 satellite.
Root cause
Capacity exhaustion
Component
Database
Contributing factor
–
Detection
The on-call team was paged at 21:14 UTC, ten minutes after database transaction locks began to rise.
Remediation
Reset the state of connections periodically to keep contention errors from growing in a cascade
Action items
Remove the sources of contention between database transactions with code changes
Add monitoring and alerts for increased levels of database locks
Look for further process and system improvements, including tools that reduce or remove the need for manual maintenance
View case

Check your cluster for this

This case involves Database. kubegap audits a Kubernetes cluster for gaps like these, read-only, from Claude Code or Codex; its checks cite public reports like the ones in this library.

Open kubegap

By root cause1

Capacity exhaustion1 case

Component: Database
Contributing factor: –
Organisation: Storj

IL-0557 Database transaction locks piled up on Storj's US1 satellite, and uploads and downloads failed with timeouts for 2 hours
Storj2025