IL-0394 Cloudflare · 2026-01-22networking
A Cloudflare router in Miami leaked IPv6 BGP routes to peers and providers for 25 minutes, congesting the backbone
Cloudflare no longer needed to forward some IPv6 traffic through Miami toward its Bogotá data center, so a change through its policy automation platform removed the prefix list 6-BOG04-SITE-LOCAL from export policy terms such as 6-TELIA-ACCEPT-EXPORT, 6-PUBLIC-PEER-OUT and 6-PRIVATE-PEER-ANYCAST-OUT on a Miami edge router. With the prefix list gone, each term only matched "from route-type internal" and accepted the route. In JunOS and JunOS EVO, route-type internal matches any non-external route type, including IBGP routes, so every IPv6 prefix that Cloudflare redistributes internally across its backbone was advertised to all BGP neighbors in Miami. The result was a mixture of Type 3 and Type 4 route leaks as defined in RFC7908; Cloudflare notes that it is very similar to an outage it had in 2020.
The leak caused congestion on the Miami to Atlanta backbone, loss and latency for some customer traffic, and discarded traffic for the leaked networks. The network team began investigating at 20:40 UTC and raised an incident at 20:44. At 20:50 a network operator reverted the configuration by hand and paused the automation for that router, ending the impact after 25 minutes. The triggering change was reverted in the code repository at 21:47, the automation was confirmed healthy at 22:07, and it was unpaused on the Miami router at 22:40.
- Severity
- –
- Scope
- IPv6 traffic: congestion on the backbone between Miami and Atlanta, elevated loss and higher latency for some Cloudflare customer traffic, and about 12Gbps at peak discarded for external networks whose prefixes leaked
- Users affected
- Some Cloudflare customers and external networks whose traffic was funnelled through Miami
- Region
- Miami
- Time to detect
- 15m
- Time to resolve
- 25m
Symptoms
On 22 January 2026 from 20:25 UTC, a Cloudflare edge router in Miami sent unintended BGP advertisements of IPv6 prefixes to its transit providers and peers. External networks' traffic was funnelled into the Miami data center, the backbone between Miami and Atlanta became congested, and some Cloudflare customer traffic saw elevated loss and higher latency.
The leaked updates came from AS13335 and are visible in MRT data with the monocle tool or RIPE BGPlay. For example, the prefix 2a03:2880:f077::/48 appeared with the AS path 64112 22850 174 3356 13335 32934, meaning Cloudflare took a route received from Meta (AS32934), a peer, and advertised it to Lumen (AS3356), a transit provider.
Traffic for the networks whose prefixes leaked was discarded by firewall filters on Cloudflare's routers, which only accept traffic for Cloudflare services and customers; at peak about 12Gbps entering the Miami router was discarded. Only IPv6 traffic was affected.
Symptom Packet lossHigh latency
Root cause
Root cause Configuration change
- Trigger
- At 20:25 UTC Cloudflare's policy automation pushed a change, merged at 19:52 UTC, that removed the BGP announcements from Miami for a data center in Bogotá, Colombia, by deleting the prefix-list 6-BOG04-SITE-LOCAL from several export policy terms.
Detection
The network team began investigating unintended route advertisements from Miami at 20:40 UTC, and an incident was raised at 20:44 UTC.
Remediation
- A network operator reverted the bad configuration change on the Miami router by hand
- Paused the automation for the router so it could not run again
- Reverted the triggering change in the network automation code repository
- Unpaused the automation on the Miami router after confirming it ran without the routing policy bug
Action items
- Patch the failure in the routing policy automation that caused the leak
- Add BGP community-based safeguards that reject routes received from providers and peers on external export policies
- Add automatic routing policy evaluation to the CI/CD pipelines that looks for empty or erroneous policy terms
- Detect problems in network configurations and the effects of automated changes earlier
- Validate routing equipment vendors' implementations of RFC9234 before rolling out BGP roles and the Only-to-Customer Attribute
- Encourage long-term adoption of RPKI Autonomous System Provider Authorization (ASPA)
Lessons
Removing only the prefix lists from export policy terms left terms that were too permissive, because a match on route-type internal accepts IBGP routes as well. Cloudflare states that route leaks are easy to cause by accident and that RFC9234 is the only way, independent of routing policy, to prevent route leaks caused at the local AS.
Sources
Related cases
IL-0225An IGP purge command made every Microsoft WAN router recompute routes, causing global latency, timeouts and packet lossMicrosoft Azure · 2023same root cause · shared Backbone network (WAN), BGP routing
IL-0228IPv4 routing across OVHcloud's global network broke after a router change put the full internet routing table into OSPFOVHcloud · 2021same root cause · shared Backbone network (WAN), BGP routing
IL-0400Public sector sites across Wales lost connectivity for over 8 hours as a PE router wrongly advertised a default routePSBA · 2026same root cause · shared BGP routing
IL-0266Eqiad routers looped traffic between each other after a BGP MED change and could not reach other sites' public IPsWikimedia Foundation · 2021same root cause · shared BGP routing
IL-0539KT's internet service failed nationwide for about 89 minutes as wrong routes spread from one router in BusanKT · 2021same root cause · shared BGP routing
Check your cluster for this
For AI tools and scripts
- This case as JSON: https://incidentlib-api.benchlet.com/cases/IL-0394
- The cases most like it: https://incidentlib-api.benchlet.com/cases/IL-0394/similar
- MCP server (Streamable HTTP, no key):
https://incidentlib-mcp.benchlet.com/mcp
The API needs no key. The MCP server offers search_incidents, get_incident, list_incidents, get_topic and vocab; both are explained on API and MCP.