SaaS Platform
Case Study: Multi-Tenant SaaS Platform
A B2B SaaS product: a browser application, a JSON API used by both the browser and customer integrations, and a few hundred tenant organizations whose usage is wildly uneven — three customers generate more load than the other four hundred combined. Traffic follows European office hours almost perfectly. This design starts as
cs-api-database with a CDN in front of it, and every component after that was added by a specific, dated problem.Requirements
- Serve a single-page application and an authenticated multi-tenant API with predictable latency during business hours.
- Store relational tenant data transactionally, plus customer-uploaded files.
- Absorb a 6x daily traffic swing and a Monday-morning login spike without over-provisioning for the peak all night.
- Keep slow work — exports, imports, emails, webhooks — off the request path.
- Survive the loss of a single availability zone without a customer-visible outage.
- Keep one tenant's workload from degrading everyone else's.
Deliberately not requirements
Half of a design is what it refuses to do. These are the refusals.
Out of scope, on purpose
- No multi-region active-active: the product is sold in one geography and a regional failure is a documented, insured risk.
- No per-tenant infrastructure isolation: tenancy is enforced in the data model and the authorization layer, not by giving each customer their own stack.
- No Kubernetes yet — one service and two workers do not need a control plane (§35).
How the design got here
In order. Each stage leads with the problem that forced it.
Stage 1
CDN, load balancer, application, database
Forced by
The product. A single-page application needs its bundle served fast and globally; the API needs the same shape as cs-api-database. The CDN is here from day one for exactly one reason: the JavaScript bundle is the largest thing every user downloads, it is identical for everyone, and it is fingerprinted — the textbook case for an edge cache.
The starting point: static assets at the edge, dynamic requests to the origin.PROVIDER-NEUTRAL
Browser / integrationpublic
CDNpublic— caches the SPA bundle and static assets; passes /api/* through uncached
Regioninternal
VPCinternal
Public subnetspublic
Load balancerpublic
Private subnetsprivate
Application instancesprivate— stateless; a fixed count at this stage
Managed PostgreSQLprivate— tenant_id on every row; every query is tenant-scoped
Browser / integration→CDN· HTTPScrosses boundary
CDN→Load balancer· uncached /api/*
Load balancer→Application instances· HTTP
Application instances→Managed PostgreSQL· SQL
| Decision | Reason | Alternative | Trade-off |
|---|---|---|---|
| One CDN distribution in front of both static assets and the API. | It gives one hostname, one certificate and one place to apply edge rules, while the API path stays uncached. It also puts a layer between the internet and the origin that can absorb a volumetric flood. | A CDN for assets on a separate hostname and the API pointed straight at the load balancer, which is simpler to reason about and costs you a second certificate plus CORS complexity. | Every API request now traverses one more hop, adding a small amount of latency, and a cache misconfiguration on an API path is a data-leak-shaped bug — a cached tenant-specific response served to another tenant. The rule "never cache anything with an Authorization header" has to be enforced, not assumed. |
| Multi-tenancy in the data model, not in the infrastructure. | One database and one application serving all tenants is dramatically cheaper and simpler to operate, and it is what lets a small team ship features to everyone at once. | A database or a stack per tenant, which gives real isolation and is the right answer under regulatory pressure or for a handful of very large customers. | A missing tenant filter in a single query is a cross-tenant data breach, so tenant scoping must be enforced structurally — a repository layer or row-level security — rather than by developer discipline. And one tenant's expensive query is everyone's slow afternoon. |
Stage 2
A cache for the hot read path
Forced by
The tenant settings and permission lookup ran on every single request. At around 300 requests per second it accounted for the majority of database CPU, and p95 API latency doubled between 09:00 and 11:00 every weekday. The queries were already indexed; the problem was volume, not shape.
A cache added for a measured reason, with an explicit invalidation story.PROVIDER-NEUTRAL
Regioninternal
Private subnetsprivate
Application instancesprivate
Managed Redisprivate— tenant settings, permissions, session data; single node at this stage
Managed PostgreSQLprivate
Application instances→Managed Redis· read-through on settings and permissions
Application instances→Managed PostgreSQL· cache miss, and all writes
Application instances→Managed Redis· invalidate on write
| Decision | Reason | Alternative | Trade-off |
|---|---|---|---|
| A managed in-memory cache rather than a larger database instance. | The workload was a small, hot, rarely-changing dataset read constantly. That is precisely what a cache is for, and it is a fraction of the cost of the database capacity it saves. | Scale the database up, or add a read replica — both work, both cost more, and neither removes the round trip. In-process caching in the application is cheaper still and gives every instance a different, independently stale copy. | A new stateful component to run, monitor, secure and pay for, and a new class of bug: stale reads after a write. Cache invalidation is now part of your data model, and it is the part that produces "the customer changed the setting and it did not take effect" tickets. |
| The cache is an optimization, and the application must work without it. | If a cache outage is an application outage, you have not added a cache; you have added a second database with no durability. Falling back to the database on a cache error keeps the failure a performance event. | Treat the cache as required, which is simpler code and turns every cache incident into a full outage. | The database must be able to absorb the full uncached load, at least briefly — which means the capacity the cache "saved" cannot be fully removed. That is the honest cost of the fallback. |
Stage 3
Object storage for customer files
Forced by
Customers started attaching documents. Files were written to the application instance's local disk, so a file uploaded through instance 2 returned 404 when the next request landed on instance 1. Worse, the instances could no longer be replaced — an autoscaling event or a deploy destroyed whatever had been uploaded since the last one.
Uploads bypass the application entirely; the API only signs and records.PROVIDER-NEUTRAL
Browserpublic
Regioninternal
Private subnetsprivate
Application instancesprivate— issues a signed, expiring, size-limited upload URL — never touches the bytes
PostgreSQLprivate— holds the metadata row: owner, tenant, key, content type, scan status
Tenant file bucketprivate— objects keyed by tenant prefix; no public access
CDNpublic— serves downloads via signed URLs so bytes never traverse the application
Browser→Application instances· request upload URLcrosses boundary
Application instances→PostgreSQL· record metadata
Browser→Tenant file bucket· PUT directly with signed URLcrosses boundary
Browser→CDN· GET download with signed URLcrosses boundary
CDN→Tenant file bucket· origin fetch
| Decision | Reason | Alternative | Trade-off |
|---|---|---|---|
| Direct-to-storage uploads with short-lived signed URLs. | A large upload streaming through the application ties up a worker for the duration of a mobile connection and puts the file on a disk that will not survive the next deploy. Signing moves the bytes off the request path and the storage problem off your instances. | Proxy uploads through the API, which is simpler, keeps validation and virus scanning inline, and does not survive contact with a 500 MB file on hotel wifi. | You lose the ability to inspect content synchronously, so validation and scanning become asynchronous steps with a state machine — the metadata row needs a status, and the UI needs to represent "uploaded but not yet scanned". Signed URLs also need tight expiry and a size limit, or they become a free file-hosting service. |
| Tenant isolation by key prefix plus authorization at the signing step. | The application decides who may read or write which prefix at the moment it signs, which keeps a single, auditable authorization point instead of scattering bucket policies. | A bucket per tenant, which gives hard isolation and hits per-account bucket limits and a management problem at a few hundred tenants. | An authorization bug in the signing code is a cross-tenant read. This code path deserves tests that assert failure, not just success. |
Stage 4
Queue and workers for slow work
Forced by
Report generation and the welcome-email sequence ran inside the request. A large tenant's CSV export took ninety seconds, held a web worker for the whole time, and then timed out at the load balancer's sixty-second idle limit — so it failed *and* consumed the capacity. Three concurrent exports were enough to make the whole API unresponsive for everyone.
Two compute pools with different shapes: latency-sensitive and throughput-sensitive.PROVIDER-NEUTRAL
Regioninternal
Private subnetsprivate
API instancesprivate— accepts the job, returns 202 with a job id, never does the work
Worker poolprivate— separate deployment, separate identity, separate scaling policy
PostgreSQLprivate
Job queueprivate— at-least-once delivery, visibility timeout, dead-letter queue
Object storageprivate— finished exports land here; the API hands out a signed download URL
NAT gatewayspublic— workers call the email provider and customer webhooks outbound
API instances→Job queue· enqueue job
Job queue→Worker pool· receive
Worker pool→PostgreSQL· read source data
Worker pool→Object storage· write result
Worker pool→NAT gateways· email + webhookscrosses boundary
API instances→PostgreSQL· job status
| Decision | Reason | Alternative | Trade-off |
|---|---|---|---|
| Workers are a separate deployment with their own identity and their own scaling policy. | The two workloads have opposite shapes: the API is latency-sensitive and bursty, the workers are throughput-sensitive and can queue. Sharing one pool means a batch of exports competes with interactive requests for CPU. | Run background threads inside the API process, which needs no new infrastructure and reintroduces exactly the contention that caused this stage. | A second deployable, a second scaling configuration, a second on-call surface, and the question "which pool is this code running in?" becomes something engineers must hold in their heads. |
| Jobs are idempotent and the queue is at-least-once. | Every practical queue redelivers, and a worker killed at 95% completion will see its message again. Designing for exactly-once delivery is designing against the medium. | Assume single delivery and add locks, which works until the first visibility timeout expires mid-job and produces two sets of side effects. | Every side effect needs a natural key or a dedup record — the export must overwrite deterministically, the email must be keyed so it is not sent twice. This is application work the queue cannot do for you. |
| A dead-letter queue with an alert on its depth. | One malformed job that fails on every attempt will otherwise be redelivered forever, consuming a worker permanently and never surfacing anywhere. | Unlimited retries, which is how a single poison message pins a worker and silently halves your throughput. | The dead-letter queue is a place where work goes to be forgotten unless somebody owns it. An unread DLQ is worse than no DLQ, because it looks like a control. |
Stage 5
Autoscaling both pools
Forced by
Traffic between 08:00 and 18:00 CET is roughly six times the overnight level, and the Monday 09:05 login spike saturated a fixed fleet twice in one quarter. Fixed capacity meant paying for the peak all night and still being short at the moment that mattered most.
Two scaling policies driven by two different signals.PROVIDER-NEUTRAL
Load balancerpublic— requests-per-target is the API scaling signal
API pool (auto)private— scales on requests per target and CPU; a floor of two, never zero
Job queueprivate— oldest-message age is the worker scaling signal — depth alone lies about latency
Worker pool (auto)private— scales on queue age; may scale to a very low floor overnight
PostgreSQLprivate— connection limit is the ceiling that autoscaling runs into first
Load balancer→API pool (auto)· requests per target
Job queue→Worker pool (auto)· oldest message age
API pool (auto)→PostgreSQL· pooled connections
Worker pool (auto)→PostgreSQL· pooled connections
| Decision | Reason | Alternative | Trade-off |
|---|---|---|---|
| Scale the API on requests-per-target, not on CPU alone. | CPU is a lagging, indirect proxy for the thing users feel. Requests per target moves at the same moment demand does, and a request-based policy reacts before latency degrades. | CPU-based scaling, which is universally available and reacts late for an I/O-bound service that saturates on connections rather than on cycles. | Requires a meaningful per-target request budget, which you only learn by load testing. Wrong, it either flaps or never triggers. |
| Scale the workers on oldest-message age, not queue depth. | Depth answers "how much work is waiting", which is not the user-facing question. Age answers "how long has the oldest job been waiting", which is the SLA. Ten thousand fast jobs are fine; one job waiting twenty minutes is not. | Depth-based scaling, which is simpler and scales aggressively for a flood of trivial jobs while missing a slow backlog entirely. | Age is a slightly noisier signal and needs a floor of workers to keep it meaningful — with zero workers, age climbs and nothing is wrong yet. |
| A hard floor on both pools and a cap on the database connection count. | Scaling to zero adds a cold start to the first user of the day, and scaling up without bound exhausts the database's connection limit — at which point new instances make the outage worse, not better. Autoscaling always runs into a fixed dependency eventually. | Uncapped scaling, which is fine until the day the ceiling is discovered during an incident caused by discovering it. | You pay for the floor overnight, and the cap means there is a demand level at which the system degrades rather than scaling. Both numbers must be written down and revisited, or they become folklore. |
Stage 6
Every tier spread across zones
Forced by
A zone incident took out the single Redis node and the database primary at once. The API tier survived perfectly and served errors for forty minutes, while the load balancer reported two of three targets healthy the entire time. Being multi-zone in the tier that was already redundant turned out to be worth nothing.
The final shape: no tier has a member in only one zone.PROVIDER-NEUTRAL
CDNpublic
Regioninternal
Zone Ainternal
API + workersprivate
NAT Apublic
PostgreSQL primary + standbyprivate— synchronous standby in zone B, automatic failover
Zone Binternal
API + workersprivate
NAT Bpublic
Redis with replicaprivate— replica in zone C; the application tolerates a cold cache after failover
Zone Cinternal
API + workersprivate
NAT Cpublic
Load balancer (regional)public
CDN→Load balancer (regional)· HTTPScrosses boundary
Load balancer (regional)→API + workers
Load balancer (regional)→API + workers
Load balancer (regional)→API + workers
API + workers→PostgreSQL primary + standby
API + workers→Redis with replica
API + workers→NAT Ccrosses boundary
| Decision | Reason | Alternative | Trade-off |
|---|---|---|---|
| Every stateful component gets a cross-zone replica; every stateless pool gets members in three zones. | A zone is a failure domain, so redundancy only counts when it crosses one. The audit that matters is component by component: for each one, name the zone it would die with. | Multi-zone only for the tiers where it is cheap, which is what this design already was — and which produced a forty-minute outage. | Roughly double the cost of every stateful component, cross-zone data transfer charges between tiers, a write latency penalty from synchronous replication, and per-zone NAT charges. This stage is where the bill stops looking like a small system's. |
| Readiness checks verify the database and the cache; liveness does not. | The forty-minute outage happened because healthy instances kept accepting traffic they could not serve. A dependency-aware readiness check would have pulled them out. | Keep shallow checks everywhere, which never causes a self-inflicted mass ejection and never detects a broken dependency either. | A dependency check makes health correlated across all instances, so a database blip can eject the entire fleet at once. It needs a minimum-healthy floor, and the cache check must be advisory rather than fatal — the application is designed to survive without the cache. |
| Rehearse the zone failure rather than reasoning about it. | The singleton that caused this outage was visible on the architecture diagram for a year and nobody saw it. Removing a zone deliberately is what makes those invisible. | An architecture review, which is cheaper and finds the components you remember to look at. | An announced window of degradation and the organizational nerve to break production on purpose — the scarcest resource in this entire case study. |
What would break this
Every design has a load, a failure or an organization size at which it stops being the right one.
Breaking points
- Write throughput past a single primary. Read replicas and the cache defer this; nothing in this design solves it. The next step is functional partitioning or sharding by tenant, which is an application change measured in quarters.
- One enormous tenant. Shared everything means a customer with 500x the data can dominate the database's cache, the connection pool and the queue. The answer is per-tenant limits and eventually a dedicated stack for that tenant — the exception that proves multi-tenancy is a business decision, not a technical one.
- Customers in another geography. Latency across an ocean is physics, and the CDN only helps the static half. Serving them properly means a second region and the data question that comes with it.
- A regulatory requirement for data residency or per-tenant encryption keys. Both break "one database for everyone" in ways that no amount of infrastructure hides.
- More services than a team can deploy in one pipeline. This is the point where a container platform starts paying for itself — and the point, not before it, where the Kubernetes conversation is legitimate (§35).
- Queue latency becoming a product feature. When "the export must be ready in ten seconds" is a promise, worker autoscaling with cold starts stops being adequate and you are buying warm capacity.
Cost shape
Drivers and relative weights. Never a price.
A SaaS platform bill: a large fixed floor you hold all night, plus a usage layer that follows office hours.ILLUSTRATIVE
Database primary + standby fixed
driven by instance size × hours, storage, IOPS · The floor cannot follow demand: the database is sized for the peak and paid for at 03:00.
API instances usage
driven by instance-hours, following the autoscaling curve · Autoscaling turned this from fixed to usage-shaped, which is most of the saving in the whole design.
Worker instances spiky
driven by instance-hours driven by queue age
Cache (primary + replica) fixed
driven by node size × hours · Memory-sized and always on, so it is pure fixed cost — and doubling it for the replica is the price of the zone the last stage bought.
Object storage · the surpriseusage
driven by gigabytes stored and growing monotonically · Customer files are never deleted unless you build deletion. This line only goes up, forever, and a lifecycle policy is the only thing that changes that.
Load balancer + CDN usage
driven by hours plus processed capacity and delivered gigabytes
NAT gateways (3 zones) · the surprisefixed
driven by hours × 3 plus per-GB processed
Cross-zone data transfer · the surpriseusage
driven by gigabytes between tiers in different zones · The invoice line created by the multi-zone stage. Nobody predicts it, and it is the direct price of the redundancy you chose.
Logs, metrics, traces usage
driven by gigabytes ingested × retention
Bars are relative weights, not currency. Real rates depend on provider, region, commitment and volume.