Cases1 case

1 to 1 of 1

1. IL-0574 Aurora MySQL writer CPU stuck near 100% minutes after Safie pointed production at a new Amazon RDS Proxy

   Safie2026

   Symptoms

   A few minutes after the production endpoint was switched to RDS Proxy, Safie's alert monitoring system fired on errors, and the CPU use of the Aurora MySQL writer was stuck near 100%.

   Database overloadHigh error rate

   Trigger

   The production application endpoints were switched to the new RDS Proxy all at once, with MaxConnectionsPercent at 21%, MaxIdleConnectionsPercent at 10% and ConnectionBorrowTimeout at 7 seconds, the same as the application's socket timeout, after a week in staging had shown no problem.

   Root cause

   Configuration change

   Component

   Database proxy and connection poolerDatabase

   Contributing factor

   Retry stormUnrepresentative stagingNo staged rollout

   Detection

   Safie's alert monitoring system fired on errors a few minutes after the endpoint was switched to RDS Proxy.

   Remediation

   • Rolled back the ECS Fargate tasks that used RDS Proxy to their previous version

   Action items

   • Raise MaxConnectionsPercent to 62% and MaxIdleConnectionsPercent to 31%
   • Choose between direct Aurora and RDS Proxy with a feature flag evaluated per ECS task, so the switch can be made in stages and undone without a deployment
   • Move read requests first and write requests afterwards
   • Raise the share in steps of 1%, 10%, 25%, 50%, 75% and 100%, checking proxy borrow wait, new backend connections, Aurora connections, CPU and latency and application errors before each step

   Lessons

   The failure and the later problems were not bugs in RDS Proxy; the application's connection pool implementation and settings were exposed by putting a proxy in between. With a proxy, backend connections are shared, so reuse depends on matching session state, closed transactions and connection lifetimes within the database's own timeouts. Before introducing it, the team should have checked what the application pool runs when it returns a connection, how it sets connection lifetime and what it runs on connect, more than the proxy settings themselves.

   View case

By root cause1