AWS Outage One Year Later: Lessons From Every Cloud Failure

TL;DR

Every major cloud provider has had a serious outage since mid-2025, each from a different cause, so any single provider is a single point of failure.

On October 20, 2025, a DNS race condition in DynamoDB’s automation took the DynamoDB endpoint in AWS us-east-1 offline. The event ran about 15 hours and spread to EC2 instance launches and Network Load Balancers.
Google Cloud (June 2025, global), Azure Front Door (October 2025, global), Oracle Cloud US East (Ashburn) (March 2026), and an automated Google Cloud action that suspended a customer’s production account (May 2026) each failed in a different layer.
Failures hit three blast radii: a zone, a region, or a whole provider. Multi-region on one provider covers the first two. It doesn’t cover a provider’s global control plane, edge network, or account layer.
Put a second, independent, already-running path under every layer you can’t lose, keep failover off the failing provider, and drill the switch.
Control Plane combines any cloud, neocloud, and bare metal into one virtual cloud and runs workloads active-active across providers, with a 99.999% availability SLA for workloads running at least two replicas in at least two locations. During the October 20 outage, no Control Plane customer experienced downtime, according to Control Plane internal incident telemetry.

What happened in the AWS us-east-1 outage on October 20, 2025?

A latent race condition in DynamoDB’s DNS management system left the DynamoDB regional endpoint in us-east-1 with an empty DNS record, and everything that resolves that name failed. According to the AWS post-event summary, the event ran from 11:48 PM PDT on October 19 to 2:20 PM PDT on October 20 (06:48 to 21:20 UTC on October 20), about 15 hours. The disruption affected dozens of AWS services in us-east-1, and AWS’s Health Dashboard event history lists the full set.

AWS’s summary splits the impact into three periods:

DynamoDB became unreachable (11:48 PM to 2:40 AM PDT). Two automation components, a DNS Planner and DNS Enactors, manage DynamoDB’s DNS. One unusually delayed Enactor applied an outdated plan, and another Enactor’s cleanup step then deleted that plan, leaving an empty record for dynamodb.us-east-1.amazonaws.com. Customer traffic and internal AWS services that depend on DynamoDB couldn’t connect.
New EC2 instances failed to launch (2:25 AM to 10:36 AM PDT). After DNS was restored, internal subsystems that depend on DynamoDB stayed impaired, and AWS throttled some operations while it recovered. Some newly launched instances had connectivity problems until 1:50 PM PDT.
Network Load Balancers saw connection errors (5:30 AM to 2:09 PM PDT). Health checks began failing because new instances came into service before their network state had propagated, and automatic health check failover then removed capacity from service. AWS engineers disabled automatic NLB health check failover at 9:36 AM PDT.

AWS said it disabled the DynamoDB DNS automation worldwide until it fixed the race condition and added protections. Recovery that needed new instances was blocked for hours: AWS says Amazon Redshift’s host-replacement workflows stalled while EC2 launches were impaired, leaving some clusters unavailable. AWS also notes that customers with DynamoDB global tables could still reach replicas in other regions, though replication to and from the us-east-1 replica lagged. Regional redundancy helped where it was real and tested.

What other major cloud outages happened in 2025 and 2026?

Four more major incidents hit a different provider or a different failure layer. The table lists major incidents from June 2025 to May 2026; it is a selection, and smaller incidents in the same period are not included.

Date	Provider	What failed	Blast radius	Duration	Source
June 12, 2025	Google Cloud	An invalid quota policy with blank fields replicated globally and crashed Service Control, the API management layer	Global, all regions	About 3 hours (most regions began recovering within 40 minutes; us-central1 took up to 2 hours 40 minutes)	Google Cloud incident report
October 19 to 20, 2025	AWS	A DynamoDB DNS race condition in us-east-1, cascading to EC2 launches and Network Load Balancers	One region, many dependent services	About 15 hours	AWS post-event summary
October 29 to 30, 2025	Microsoft Azure	Incompatible customer configuration metadata crashed the Azure Front Door data plane at edge sites	Global edge network	About 8.5 hours (15:41 to 00:05 UTC)	Azure Front Door post-incident review
March 3 to 4, 2026	Oracle Cloud Infrastructure	A network infrastructure issue affecting multiple OCI services in US East (Ashburn); Oracle has not published a root cause	One region	About 20 hours (13:24 UTC March 3 to 09:18 UTC March 4)	Oracle incident record
May 19 to 20, 2026	Google Cloud (account level)	An automated Google Cloud action that Railway reports incorrectly suspended its production account	One customer’s whole production account, cascading to its workloads on AWS and its own hardware	About 8 hours	Railway incident report

Google Cloud (June 2025). Google’s report says the policy change replicated globally within seconds, and the failing code path was never exercised during its region-by-region rollout. Google’s Cloud Service Health status page was down too, delaying its first incident post by about an hour, and some customers’ monitoring running on Google Cloud also failed.
Azure Front Door (October 2025). Microsoft’s review says this incident was not directly related to an earlier Front Door incident on October 9, although both were broadly related to configuration propagation risk.
Oracle US East (March 2026). Oracle describes a network infrastructure issue that caused connection timeouts, errors, and increased latency across multiple OCI services. The Register reported that it disrupted parts of TikTok’s U.S. service, which TikTok’s U.S. joint venture acknowledged (The Register). Oracle logged a second Ashburn networking incident that began 52 minutes after the first was resolved and ran until 04:40 UTC on March 5.
Account suspension (May 2026). Railway’s report says the automated action extended to many Google Cloud accounts. Its edge proxies cached routing tables from a control plane hosted in Google Cloud, so when the cache expired, workloads on AWS and on its own hardware became unreachable even though they were still running. Railway takes responsibility for that architecture. Redundant workloads can still share one dependency that decides whether anyone can reach them.

Large systems have shared layers, and shared layers fail, whichever provider runs them.

What is a blast radius in a cloud outage?

A blast radius is the set of workloads that fail together when one thing breaks. Cloud failures fall into three scales, and each needs a different kind of redundancy.

Blast radius	Typical cause	What it takes out	Redundancy that covers it	Recent example
Zone	A facility, power, cooling, or network fault in one availability zone	Workloads placed only in that zone	Replicas spread across zones in a region	None among the five above; multi-zone designs usually absorb zone faults
Region	A regional service, a regional dependency, or regional networking	Everything in that region, plus anything that depends on its endpoints	Active capacity in more than one region, with data replicated	AWS us-east-1 (October 2025), Oracle US East (March 2026)
Provider	A global control plane, global edge network, global configuration push, or account-level action	Every region of that provider, or your whole account	A second, independent provider, including the control path	Google Cloud Service Control (June 2025), Azure Front Door (October 2025), the account suspension (May 2026)

Why does multi-region fail during cloud outages?

Multi-region covers regional failures. It doesn’t cover failures in layers every region shares. Three of the five incidents above weren’t regional. Google’s Service Control policy data replicated to every region within seconds. Azure Front Door’s crash spread to all edge sites worldwide. The account suspension took down an entire account, which spans every region, and then spread to other clouds through a routing dependency hosted in Google Cloud.

Multi-region plans also fail for quieter reasons:

The failover step needs a control plane call (launch an instance, change a route, update a record) in the region that is failing. On October 20, new EC2 launches in us-east-1 failed from 2:25 AM to 10:36 AM PDT.
The failover target shares a dependency with the primary, such as a global service, an identity system, or a DNS zone.
The monitoring that tells you to fail over runs on the provider that is down. Google’s June 2025 report says this happened to some customers.
The plan was written but never exercised under load.

AI workloads add their own dependencies: model APIs, GPU capacity from neoclouds, and agents that fail a customer if they die mid-task. Each is another layer that needs a second path. For a regional failure like October 20, tested multi-region capacity is a real defense, but it covers only the regional layer.

How do you keep services up during a cloud outage?

Post-incident reports don’t say who stayed up, but they show which preparations would have mattered. Build them before the incident:

Keep capacity already running. When instance launches fail, a plan to scale up on demand depends on a control plane that may be struggling. Capacity already serving in another location doesn’t.
Keep the failover path off the failing layer. Traffic shifting, health checks, and routing decisions should run somewhere independent of the provider or region in trouble.
Monitor from outside the provider. Google’s report shows what happens when the monitoring stack and the workload share a failure domain.
Map hidden dependencies. The account-suspension report shows workloads in three environments and a routing control plane in only one of them.
Drill the switch. Teams that rehearse find the stale credential, the missing quota, or the unreplicated dataset on a Tuesday afternoon instead of during an outage.
Match redundancy to the cost of downtime. Rank services by business impact and put the strongest design under the ones that cost the most when they stop.

Control Plane builds the first two into the platform: capacity running in every location, and routing that sits above any single provider. For a deeper look at targets and trade-offs, see our guides to high availability, RTO and RPO targets, and what single-region, multi-region, and multi-cloud designs can deliver.

How does Control Plane keep you up when a provider fails?

Control Plane combines any cloud, neocloud, and bare metal into one virtual cloud, so you stay up when others are down. You choose the locations for a Global Virtual Cloud (GVC): AWS, Google Cloud, and Azure regions, Kubernetes clusters you already run, and your own servers on premises. Every workload runs active-active in all of them, with replicas already serving in each location. A built-in global endpoint sends each request to the nearest healthy location. When a zone, region, or provider fails, traffic moves to locations that are already running, and autoscaling adds replicas there as load arrives. Nothing has to boot, nobody runs a runbook, and failover doesn’t depend on the failed provider’s control plane.

The rest of the platform serves the same outcome. Universal Cloud Identity gives workloads short-lived, least-privilege access to 600+ AWS, Google Cloud, and Azure services, with no long-lived cloud keys in your code, so nothing needs re-keying when traffic moves. Each location keeps serving the last healthy version until a new release passes its health checks, so a broken update can’t take the workload down (docs). Capacity AI right-sizes CPU and memory to actual usage in every location.

On October 20, 2025, 20% of Control Plane customers had us-east-1 in their Global Virtual Clouds. Their workloads were active and running in healthy locations within 10 seconds, with no customer downtime, according to Control Plane internal incident telemetry (Beyond Backups). For the Google Cloud, Azure, Oracle, and account-level failures above, a GVC that spans more than one provider is designed to route around a failure confined to one.

Control Plane offers a 99.999% availability SLA for workloads running at least two replicas in at least two locations, and those locations can be on different providers. No migration is required: existing Kubernetes clusters and your own servers join as locations, so you add failure domains from what you already run.

Next step: watch us recreate the us-east-1 outage live on October 20 at 11 AM PT. Register for the webinar, or talk to the Control Plane team.

Cloud outage resilience checklist

Use this to find the layers you can’t recover without. Answer each row for every service that matters to revenue.

Check	Question to answer
Single points of failure	Have we listed every provider, region, account, DNS zone, identity system, and control plane each service depends on?
Zone spread	Are replicas spread across at least two zones?
Region spread	Can the service run in a second region without a human launching new capacity?
Provider spread	Can the service run on a second provider if a global layer or account fails?
Capacity already running	Is the failover location already running enough replicas to carry real load?
Data	Is data replicated to the failover location, with an RPO we have actually measured?
Failover path	Does traffic shifting work if the primary provider’s console, API, and DNS are unavailable?
Monitoring	Does alerting run outside the provider we are monitoring?
Account risk	Do we have a documented path if the provider suspends or restricts our account?
Credentials	Can the failover location authenticate to everything it needs without a manual step?
Drills	Have we run a failover in the last quarter and written down what broke?
Communication	Does the team know who decides to fail over, and how customers are told?
SLA reality	Do we know that provider SLA credits are service credits, not compensation for lost revenue?

Frequently asked questions

What caused the AWS us-east-1 outage in October 2025?

A latent race condition in the DNS management system for Amazon DynamoDB left the regional endpoint in us-east-1 with an empty DNS record, so services that resolve that endpoint couldn’t connect. Recovery then continued through EC2 instance launch failures and Network Load Balancer connection errors.

How long did the October 2025 AWS outage last?

AWS’s summary gives the event window as 11:48 PM PDT on October 19 to 2:20 PM PDT on October 20, about 15 hours, with three distinct periods of impact.

Which major cloud outages happened in 2025 and 2026?

This post covers Google Cloud’s global Service Control outage (June 12, 2025), the AWS us-east-1 outage (October 19 to 20, 2025), the Azure Front Door global outage (October 29 to 30, 2025), the Oracle US East outage (March 3 to 4, 2026), and an automated Google Cloud suspension of a customer’s production account (May 19 to 20, 2026). Other incidents happened in the period, so treat it as a selection.

How do you survive a cloud provider outage?

Identify what each service depends on, then give each dependency a second independent path: spread across zones, run active capacity in more than one region, and for critical services run on a second provider. Keep failover and monitoring off the failing provider, keep capacity already running, and drill the switch.

Is multi-region enough to survive an outage?

Multi-region protects against regional failures, such as the AWS us-east-1 and Oracle US East incidents, if failover works without the failing region. It doesn’t protect against a failure in a provider’s global control plane, edge network, or your account, as in the Google Cloud, Azure Front Door, and account-suspension incidents.

What is the difference between multi-cloud and multi-region?

Multi-region means running in more than one region of the same provider. Multi-cloud means running across more than one provider. Multi-region covers regional failures, and multi-cloud also covers provider-level failures, at the price of more work in networking, identity, and data replication unless a platform such as Control Plane handles them for you.

Is multi-cloud worth it?

It depends on what an hour of downtime costs you. Provider SLA credits are service credits, not compensation for lost revenue. For revenue-critical services, a second provider removes a failure class that regions can’t. Control Plane makes a second provider another location in the same GVC, with capacity scaled to demand in each location. Rank services by business impact and give the most critical ones cross-provider redundancy first.

What happened to Control Plane customers during the AWS outage?

During the October 2025 AWS outage, 20% of Control Plane customers had us-east-1 in their Global Virtual Clouds. Their workloads were active and running in healthy locations within 10 seconds, with no customer downtime, according to Control Plane internal incident telemetry. Control Plane offers a 99.999% availability SLA for workloads running at least two replicas in at least two locations.