IL-0357 Vercel · 2025-10-20awsnetworking

Up to 22% of Vercel static file requests failed when part of its global cache was lost during the AWS us-east-1 outage

Vercel runs its Compute, CDN and Firewall services across 19 AWS regions, and its serving stack is designed to survive the loss of any one region. When AWS us-east-1 (iad1) failed on 20 October 2025, the first alerts fired at 06:55 UTC. At 07:15 the team decided to move traffic out of iad1 into neighbouring regions, re-routing incrementally from 07:20 until 07:44, and from 07:25 it re-routed function invocations for customers with backup or secondary regions configured. At 07:45 a cascading failure cost Vercel a percentage of its global caching infrastructure, the Regional Cache service in its CDN regions, and static file serving failures peaked at about 22% of traffic until service was restored at 08:18. Vercel states that the root cause of this cascading failure has already been mitigated. Function invocations for customers using iad1 as their only region were restored at 09:21.

The AWS outage also caused a second, independent incident on the Vercel side, a disruption of the control plane after a feature flag provider outage, which did not affect production traffic and is not covered here.

Severity: –
Scope: Static file serving from Vercel's CDN failed for up to about 22% of traffic; function invocations had elevated errors, and requests reaching iad1 directly were disrupted until re-routed
Users affected: –
Region: global, with us-east-1 (iad1) as the failed region
Time to detect: –
Time to resolve: 33m

Symptoms

On 20 October 2025 at 07:45 UTC, while Vercel was moving traffic away from its us-east-1 region (iad1) during the AWS us-east-1 outage, a cascading failure took out a percentage of Vercel's global caching infrastructure. Failures of static file serving peaked at about 22% of traffic, the largest impact of the whole event, until service was restored at 08:18 UTC.

The first automated alerts for errors across Vercel services fired at 06:55 UTC and paged the on-call engineers. At first only minor disruption of traffic was visible. By 07:07 the on-call engineers judged that AWS-managed services likely had a major issue and paged the most senior responders. Function invocations showed elevated errors, and users connecting directly to iad1 saw disruption until their traffic was moved, which started at 07:20 and finished at 07:44.

Following a request through the serving path, Vercel DNS, the Anycast CDN addresses, the L3 to L5 DDoS protection, TLS termination and the L7 firewall all worked as expected, and global deployment metadata worked except for requests that hit iad1 directly. The failing part was Regional Cache, the component in each Vercel CDN region that caches static assets and cacheable dynamic requests. Its routing metadata originates in us-east-1 S3 and is meant to be persisted globally on first use, and the cascading failure of the regional cache service widened the disruption. ISR content cached at the edge kept serving, while ISR content with us-east-1 as its origin that was not yet cached at the edge was temporarily disrupted.

Symptom High error rate

Root cause

Root cause Cloud provider outage

Component CDN and edge network Object storage

Contributing factor –

Trigger: The AWS us-east-1 region (Vercel's iad1) failed on 20 October 2025, and Vercel began re-routing traffic and function invocations away from it; at 07:45 UTC a cascading failure followed in the global caching infrastructure.

Detection

The first automated alerts, for errors across Vercel services, fired at 06:55 UTC at the start of the us-east-1 disruption, 50 minutes before the cache failure began; incidents were created automatically and on-call engineers were paged.

Remediation

Re-routed end-user traffic away from iad1 into neighbouring data centers
Re-routed function invocations to configured backup or secondary regions
Restored static file serving at 08:18 UTC and mitigated the root cause of the cascading cache failure

Action items

Write a detailed technical postmortem leading to systematic mitigation of the failure modes these disruptions exposed
Continue and intensify rehearsals of major regional disruptions

Lessons

–

Sources

https://vercel.com/blog/update-regarding-vercel-service-disruption-on-october-20-2025 · blog · archived copy

Related cases

IL-0438 Builder Work became almost entirely unreachable with 504 errors when the Amazon CloudFront VPC Origins feature failedANDPAD · 2026
same root cause · shared CDN and edge network
IL-0368 PlanetScale's control plane went down and some database branches failed queries during the AWS us-east-1 outagePlanetScale · 2025
same root cause · shared Object storage
IL-0584 Cloud CDN at Osaka served corrupted JavaScript files, and a single-page app failed with a SyntaxError in western JapanMonicle · 2024
same root cause · shared CDN and edge network

Check your cluster for this

kubegap audits a Kubernetes cluster for conditions like the ones in this case, CDN and edge network, Object storage among them, read-only, from Claude Code or Codex. Its checks cite public reports like this one.

Open kubegap

For AI tools and scripts

This case as JSON: https://incidentlib-api.benchlet.com/cases/IL-0357
The cases most like it: https://incidentlib-api.benchlet.com/cases/IL-0357/similar
MCP server (Streamable HTTP, no key): https://incidentlib-mcp.benchlet.com/mcp

The API needs no key. The MCP server offers search_incidents, get_incident, list_incidents, get_topic and vocab; both are explained on API and MCP.