Many SaaS production environments run in a single AWS region, often us-east-1. When that region has a bad day, the app has a bad day. This guide covers what fails in a regional outage, which architectures survive it, what they cost relative to each other, and what to do first.
Disclosure: Control Plane publishes this guide.
TL;DR: How do I keep my SaaS app online when an entire AWS region goes down?
Run the application in more than one region (or more than one provider) and serve traffic from all of them, remove the hidden single-region dependencies, and make failover automatic. A standby that never serves traffic is untested until the outage.
What to do, in order:
- Map your single-region dependencies. List everything that must be in one region to serve a request: identity, DNS, secrets, CI/CD, registries, observability, third-party APIs, and data stores. Use the checklist below.
- Get multi-AZ right. Spread each tier across at least two Availability Zones, so one zone failure does not take you down.
- Deploy a second region that serves real traffic. Run the same stateless services in two regions behind health-checked routing. Do not park it as an idle standby.
- Replicate the data layer on purpose. Choose a replication design, set your RPO and RTO targets, and measure replication lag.
- Run a game day. Block one region on purpose, watch what breaks, and fix the dependencies you missed. Then add a second provider for the workloads that cannot go down.
The deciding question is which customer-facing requests would still succeed if your primary region disappeared for 14 hours and no one logged in to the AWS console.
What actually fails when a region goes down?
Regions fail for ordinary reasons: a bad change, a software bug, an overloaded dependency, or a power and cooling fault. The Control Plane 2026 Cloud Outage Report and the Cloud Outage Tracker document 62 major AWS, Azure, Google Cloud, and Cloudflare outages since January 2020. The findings:
- Most outages start with a change. Of the 62 outages, 34 (55%) started with a configuration or deployment change, a software bug, or automation acting on bad input.
- us-east-1 shows up more than any other region. AWS us-east-1 (Northern Virginia) appears in 9 of the 62 incidents, 3 of them limited to one Availability Zone. No other region appears more than twice.
- Many outages cross regions. 31 of the 62 outages hit more than one region or a provider’s global services. Among AWS, Azure, and Google Cloud alone, 18 of 46 did.
- They last hours. The median major outage lasted 6 hours 8 minutes. Power and cooling failures ran a median of 21 hours 48 minutes.
- The cost is large. The October 19-20, 2025 AWS us-east-1 outage lasted 14 hours 32 minutes per AWS’s post-event summary. Parametrix estimated that US companies lost $500 million to $650 million.
Two lessons follow. A second Availability Zone does not help when the failure spans the region. And a failure that starts as a bad change can spread past the region where it began.
How do companies stay online during AWS us-east-1 outages?
They do not depend on us-east-1 to serve a request. Their application runs in at least one other location, traffic reaches that location through health-checked routing, and the request path does not call a service whose control plane lives in us-east-1. Companies that only keep backups in another region stay offline until they finish restoring.
Why can a us-east-1 problem affect customers who never deployed there?
Some AWS global services have control planes in us-east-1. AWS’s Fault Isolation Boundaries whitepaper says global services separate a control plane from a data plane. For most of them, the control plane is hosted in a single AWS Region while the data plane is globally distributed. In the aws partition, the whitepaper lists the control planes of AWS IAM, AWS Organizations, AWS Account Management, and Route 53 Private DNS in us-east-1, and says Route 53 operates its control plane in us-east-1 with a data plane spread across points of presence.
The whitepaper’s guidance is to avoid relying on those control planes in your recovery path and to rely on data plane operations instead. If your failover plan says “update the DNS record” or “create a new IAM role,” check whether that step needs a control plane that sits in the failing region.
What are the four architecture options for regional resilience?
AWS’s Disaster Recovery of Workloads on AWS whitepaper groups strategies into four approaches, from low cost and low complexity (backups) to more complex designs that use multiple active Regions. The table summarizes them. Recovery ranges follow the whitepaper’s strategy summary; the cost column is relative, not priced.
| Option | What runs where | Typical recovery time and data loss (per AWS) | Cost profile | Protects against |
|---|---|---|---|---|
| Backup and restore | Production in one region. Backups copied to another region. Infrastructure redeployed from code after a disaster. | Hours | Lowest. You pay for storage and for the restore when it happens. | Data loss and corruption. A zone failure. A region failure, after a full rebuild. |
| Pilot light | Core data replicated to a second region. Databases and storage are always on. Application servers are off or minimal until needed. | Tens of minutes | Low to moderate. Always-on data tier plus small standing footprint. | Zone and region failures, after you start the application tier. |
| Warm standby | A scaled-down but fully functional copy of production runs in a second region and does not serve production traffic until failover. | Minutes | Moderate to high. A second copy runs all the time at reduced scale. | Zone and region failures, once you scale up and switch traffic. |
| Multi-site active-active | The workload runs in multiple regions at once, and all regions serve traffic. | Near zero for most disasters. Recovery from data corruption still relies on backups. | Highest build and operating complexity. Capacity is spread across regions. | Zone and region failures. With a second provider, provider-wide failures too. |
AWS notes that active/passive strategies use an active site to serve traffic while the passive site does not serve traffic until a failover event is triggered. It also notes that multi-site active/active serves traffic from all regions where it is deployed, and hot standby serves traffic from a single region only.
Two cautions from the same whitepaper apply to every row. First, test your strategy regularly, because an untested plan is a guess. Second, use data plane operations in the failover path, because control planes typically have lower availability design goals.
What is the best cloud architecture model for the highest resilience and availability?
Multi-site active-active across regions is the model that delivers the highest resilience and availability. Every location serves live traffic, so failover only changes routing and no servers have to start. Because every location runs continuously, you know each one works. For failures that take out a whole provider, or a provider’s global services, extend the same pattern across providers.
Active-active is not free of trade-offs. Stateless services are straightforward. State is harder, because every write has to land somewhere and replicate. Plan the data layer before you plan the routing.
Which hidden single-region dependencies should I check first?
Most failed failovers trace back to something nobody listed. Review each item against the question “does this need us-east-1, or any single region, to work?”
- Identity and SSO: Does login depend on one region’s identity provider or on IAM control plane calls? Can users and services authenticate when that region is down? Older AWS SDK and CLI versions default to the global STS endpoint, which is served from us-east-1 for requests from opt-in Regions; AWS recommends Regional STS endpoints and IAM trust policies that accept SAML sign-in from more than one Region.
- DNS: Does failover require editing records in a control plane in the failing region? Are health checks and routing policies in place before the outage? Are TTLs short enough to matter?
- Secrets: Is your secrets manager reachable from the surviving region, and is every secret replicated there?
- CI/CD: Can you deploy a fix while the primary region is down? Pipelines, runners, and artifact stores often live in the same region as production.
- Container registries: Do nodes in the second region pull images from a registry in the first? A new pod in a healthy region cannot start if its image is unreachable.
- Observability: Do logs, metrics, traces, and alerting run in the region you are monitoring? A dashboard that fails with the region cannot tell you the region failed.
- Third-party APIs: Do payment, email, auth, and feature-flag vendors run in the same region? Check their status pages and architecture notes.
- Data stores: Where is the primary for each database, queue, cache, and object bucket? Which have cross-region replicas, and how far behind do they run?
- Configuration and service discovery: Does the app read config from a single-region service at startup?
- Egress and allowlists: Do partners allowlist only your primary region’s IP addresses?
Is multi-region active-active worth the cost for a growing SaaS company?
For most SaaS companies with paying customers and uptime commitments, yes, once the cost of downtime is compared with the cost of the extra footprint. The cost math is less severe than the “double the bill” intuition suggests.
The cost side: a hypothetical in relative units
The figures below are illustrative. Suppose one region needs N servers at peak load.
| Design | Capacity you pay for, in units of N | Does the second location serve traffic? |
|---|---|---|
| Single region | N | Not applicable |
| Warm standby | N plus a scaled-down second copy (call it S, where S is less than N) | No. The copy sits idle until failover, then must scale up. |
| Active-active, 2 regions, each able to carry full load | 2N | Yes, both |
| Active-active, 3 regions, sized to survive the loss of any one | 1.5N | Yes, all three |
| Active-active, 4 regions, sized to survive the loss of any one | About 1.33N | Yes, all four |
To survive the loss of one of R regions at full load, total capacity is N times R divided by (R minus 1). More regions means each region carries a smaller share of peak, so the headroom you pay for shrinks as a fraction of the total. Capacity that serves traffic every day also earns its keep, because it lowers latency for users nearby.
Two levers cut the bill further. If you accept degraded service in a failure, for example by shedding low-priority work, you can size for less than full load. If your platform scales replicas with demand, surviving locations add replicas when traffic shifts to them, so you do not hold all headroom in advance.
Active-active also adds engineering cost: data replication design, deploy tooling across locations, and testing. That cost is real, and it is the main reason to stage the work, as described below.
The downtime side
Compare that against what an outage costs. In the Uptime Institute’s 2026 annual outage analysis, 57% of respondents said their most recent major outage cost more than $100,000, and for the second consecutive year, 1 in 5 reported costs above $1 million. The Cloud Outage Tracker lists published downtime-cost estimates alongside the outage timeline, so you can set your own revenue per hour against the median outage of 6 hours 8 minutes.
As a simple test, multiply your revenue and recovery costs per hour of downtime by the median outage length, then compare the result with the yearly cost of the second location. If one outage costs more than a year of the extra footprint, active-active pays for itself on the first event.
What are the most cost effective high availability options for a 30 person startup?
Stage the work. Each stage protects against a larger failure and builds on the one before.
| Stage | What you do | Protects against | Main cost |
|---|---|---|---|
| 1. Multi-AZ | Run each tier across at least two Availability Zones. Use managed multi-AZ databases. Add health checks and autoscaling. | Instance, rack, and single-zone failures | Low. Mostly configuration and some duplicate capacity. |
| 2. Second region serving real traffic | Deploy stateless services in a second region, route traffic with health checks, and replicate data. Fix the hidden dependencies. | Regional outages, including us-east-1 events | Moderate. Added capacity plus data replication and deploy work. |
| 3. Second provider for critical workloads | Run the workloads that cannot go down in a second provider or on your own servers, behind the same routing. | Provider-wide failures and outages in a provider’s global services | Higher. Different tooling, unless a platform abstracts it. |
For a 30 person startup, stage 2 is the right stopping point for most services. Reserve stage 3 for the few workloads where an outage ends a customer relationship, such as authentication, payments, and the core API. Stage 1 alone leaves you exposed to the failures that hit multiple zones, which the tracker shows are common.
Use infrastructure as code for all three stages. Terraform and Pulumi work for this, and a platform that deploys one workload definition to many locations reduces how much you maintain.
How do I survive a full cloud provider outage without a disaster recovery runbook?
Design so there is nothing to run. If every location serves live traffic and routing is health-checked, a failed location drops out of rotation and the others take its requests. No one has to find a runbook at 3 a.m., and no failover step depends on the failing provider’s control plane.
Some things still need a plan. Keep this short list current:
- The data layer. Stateless compute fails over by routing. Stateful failover depends on replication lag and backup frequency. Set an RPO and RTO per data store, know the replication lag, and keep clean restore points in case corruption replicates. See cloud disaster recovery RTO and RPO targets and best practices.
- DNS TTLs. Short TTLs on records you control. Avoid failover steps that depend on cached records expiring.
- Dependencies. Anything on the checklist above that you cannot make multi-region needs a fallback or an owner.
- Capacity. Know whether surviving locations can scale to the extra load, and how fast.
- Communication. Decide who updates the status page, so engineers can focus on the fix.
How do I test it with a game day?
- Pick a window and tell stakeholders.
- Define success: which requests must succeed, and the latency and error budgets.
- Block one region at the network layer, or remove it from routing. Do not shut it down gracefully, because real outages are rarely graceful.
- Watch requests, logins, deploys, and alerts. Record what fails.
- Restore the region and verify data consistency.
- Fix the findings, then repeat with a different failure, such as a global service control plane being unavailable.
Run a game day at least twice a year, and after any major architecture change.
Is a cloud-based disaster recovery solution better than on premises for resilience and failover?
Neither is better on its own. The deciding factors are whether the second location serves traffic and whether it fails independently of the first. A cloud region that sits idle and shares a global dependency with the primary can fail with it. An on-premises site that serves live traffic and shares nothing with the cloud region can carry the load.
Cloud DR gives you fast provisioning, many locations, and pay-as-you-go standby. On-premises DR gives you control over hardware, data location, and independence from any single provider’s control plane. Many teams need both, for example to meet data residency rules. One Control Plane Global Virtual Cloud can include public cloud regions and your own servers, so both can serve traffic behind one endpoint.
How does Control Plane approach regional resilience?
Control Plane is the platform for running unbreakable apps and agents. It combines any cloud, neocloud, and bare metal into one Global Virtual Cloud (GVC). A workload deployed to a GVC runs active-active in every location of that GVC, and a global endpoint routes each request to the nearest healthy location, so you get failover without paying for idle standby capacity. Adding a region or cloud means adding a location to the GVC. There is no second cluster to build.
During the October 2025 AWS us-east-1 outage, 20% of Control Plane customers had us-east-1 in their Global Virtual Clouds. Their workloads were active and running in healthy locations within 10 seconds, with no customer downtime, according to Control Plane internal incident telemetry.
What this covers:
- Where it runs. Serverless, standard, stateful, cron, and VM workloads run on AWS, GCP, and Azure, on clusters you already run through Bring Your Own Kubernetes (BYOK), on your own machines (bare metal or VMs) turned into managed locations with Managed Kubernetes (MK8s), and on premises. On Control Plane-provided compute, there is no Kubernetes cluster you have to operate. Many teams run it in one cloud, across several regions.
- Availability. The 99.999% availability SLA applies to workloads running at least two replicas in at least two locations. Those locations can be two regions of one cloud or locations on different providers.
- Credentials. Universal Cloud Identity gives workloads short-lived, least-privilege access to 600+ AWS, GCP, and Azure services, with no long-lived cloud keys in your code. That removes one class of hidden dependency from the checklist.
- State. Platform-level routing moves stateless traffic. Stateful recovery still depends on your replication design, backup frequency, and restore points, and you own the RPO and RTO targets and the failover drills.
- Cost. You pay for the CPU and memory reserved for each workload, by the millicore and the megabyte, with no contracts or minimums. Capacity AI right-sizes CPU and memory for each workload based on observed usage, within a minimum and maximum you set. Customers typically cut compute costs 30 to 50 percent compared with running directly on AWS, GCP, or Azure.
- Operations and compliance. Teams operate Control Plane through one API, with the same capabilities in the CLI, console, Terraform, Pulumi, and the Control Plane MCP server. Control Plane complements Terraform and Pulumi; it does not replace them. Control Plane is PCI DSS Level 1, SOC 2 Type II, HIPAA, and GDPR compliant.
For the build steps, read how to build active-active high availability across regions and clouds. To compare approaches, see best multi-cloud failover solutions in 2026.
FAQ
How do I keep my SaaS app online when an entire AWS region goes down?
Run the application in more than one region, or more than one provider, and serve traffic from all of them. Remove hidden single-region dependencies, and make failover automatic with health-checked routing.
How do companies stay online during AWS us-east-1 outages?
They keep the request path independent of us-east-1. The application runs in other locations, routing moves traffic away from failing ones, and the failover does not rely on control planes hosted in us-east-1.
Why can a us-east-1 outage affect apps that are not deployed in us-east-1?
Some AWS global services have control planes in us-east-1, per AWS’s Fault Isolation Boundaries whitepaper. If your login, DNS changes, or recovery steps need those control planes, a us-east-1 problem can reach you.
Is multi-region active-active worth the cost for a growing SaaS company?
Usually, when the cost of downtime exceeds the extra footprint. Sized to survive the loss of one of three regions, total capacity is about 1.5 times single-region peak capacity, and the added capacity serves traffic every day.
What are the most cost effective high availability infrastructure options for a 30 person startup?
Multi-AZ first, then a second region serving real traffic, then a second provider for the workloads that cannot go down. Most services can stop at stage two.
How do I survive a full cloud provider outage without a disaster recovery runbook?
Design so there is nothing to run, with active-active deployment and health-checked routing. Keep a short plan for the data layer, DNS TTLs, and third-party dependencies, and test it with a game day.
What is the best cloud architecture model for the highest resilience and availability?
Multi-site active-active across regions. Extend it across providers to cover provider-wide failures.
Is a cloud-based disaster recovery solution better than on premises for resilience and failover?
Neither wins by location. What matters is whether the second location serves traffic and fails independently. A Control Plane GVC can include public cloud regions and your own servers.
What is the difference between warm standby and active-active?
In warm standby, a scaled-down copy waits in a second region and does not serve production traffic until failover. In active-active, every region serves traffic all the time, so failover is a routing change.
How often should I test regional failover?
At least twice a year with a game day, and after any major change to your architecture or dependencies.
Next steps
Start with the dependency checklist, then move one stateless service into a second location that serves real traffic.
