Scale Thought Experiments
Take the store from one server and one database to a size where something breaks, and watch which component is actually the bottleneck at each level — then see that a cache fixes reads and not writes, a queue fixes checkout wait and nothing else, and at low traffic every added component is unjustified. The lab runs the numbers; the lesson is how to read them.
The situation, the reflex, and why it stalls
Every lesson starts where being stuck starts: someone has a problem, and the first move that comes to mind feels like progress.
You have imagined the system at scale and named the assumptions. How do you tell which one actually bends first, and which component — if any — would move it?
You ran the thought experiment and have three candidate bottlenecks. The team has three opinions, one per candidate, and each comes with a component: index, cache, replica. Someone suggests doing all three "to be safe". You suspect at most one of them matters at the size you are actually going to see, and you have no way to say which.
Do all three. Each is a known pattern, each is cheap in isolation, and the combined diagram looks like a system that will hold. Nobody can be blamed for a cache.
Three components are added and none is attributable. When load arrives and checkout is still slow, there is no way to say which of the three was supposed to help, because none was tied to a measured bottleneck.
- Three components are added and none is attributable. When load arrives and checkout is still slow, there is no way to say which of the three was supposed to help, because none was tied to a measured bottleneck.
- The read cache is added to a system whose first bottleneck is writes. Every product-page read is served from memory; every checkout still contends on the same stock row, and the graph does not move.
- The queue makes checkout asynchronous. Customers now see "your order is being processed" instead of a confirmation, which is a product change made to solve a latency problem that did not exist at launch size.
- "To be safe" has no stopping rule. The next scaling article names a CDN and a read replica, and the system grows by the reading list rather than by the readings.
The move
Precisely enough to apply it to a problem you have never seen — not a slogan.
- Turn the thought experiment into a model with numbers, however rough: for each component — web tier, database CPU, database connections, disk, the external provider — how much of it does one core-workflow request use, and how many requests arrive per second at each size? Multiply. The component whose use crosses its capacity first is the bottleneck *at that size*, and it is usually not the one the loudest opinion named.
- Raise the size in steps and watch the bottleneck move. At the launch size, nothing is a bottleneck and every added component is unjustified — that is the honest half of the exercise. As the size rises, one component crosses first; note it. Add the one component that addresses *that* reading, re-run, and see what crosses next.
- For each candidate component, know what it moves and what it does not before adding it. A cache reduces database reads; it does nothing for writes, and checkout is writes. A queue removes work from the request path; it does not reduce work. A replica adds read capacity and introduces replication lag. Each one also carries a ledger entry — what it solves, what it introduces, what happens when it fails — and a component without those answers is not a candidate (The Complexity Ledger).
- Stop at the size the constraint named. Everything beyond it is the answer to a question nobody asked; keep it as a note — "at this size the next bottleneck is X, and the component that would move it is Y" — so that when the constraint changes, the next step is already known.
What crosses first, at each size
The matrix is the model's output for the store, with sizes as columns and the state of each component as the cells. Reading it left to right is the growth path; reading it top to bottom at one size is the answer to "what would I add now?". The launch column is the honest half: nothing crosses, and every component is unjustified.
| Component | Launch (hundreds/day) | Ten times launch | A hundred times | A thousand times |
|---|---|---|---|---|
| Web tier | idle | fine | fine | crosses — add instances behind a balancer |
| Database CPU (reads) | idle | fine | crosses — index the product query, then cache | cache holds; replica for the long tail |
| Database (checkout writes) | idle | fine | fine | crosses — stock-row contention; reservation redesign, not a cache |
| Connections | plentiful | fine | fine | crosses — pool; queue the email out of the request |
| Payment provider | the slowest step, and not ours | same | same | same — a queue shortens our request, not their tail |
| Justified additions | none | none | index; cache if needed | queue for email; pool; reservation change |
The growth path as an order of work
Read as a sequence, the matrix is an implementation order — with the essential property that each step is triggered by a reading, not by a date. The alternative order is real: a store whose workload is write-heavy from the start walks the same components in a different sequence, and the device says so.
- 1Correctness first: conditional stock decrement, captured prices
because These bend at a hundred users and are cheap now; no reading is needed to justify them.
- 2Index the product query the plan flagged
because The first read crossing in the model; an index moves it an order of magnitude with no new component.
- 3Cache product reads — only when the index is not enough
because The second read lever; it introduces staleness, so it waits for the index to prove insufficient.
- 4Queue the email and non-critical provider calls out of checkout
because Connection and latency crossing; a queue shortens the request path without changing what the customer is told.
- 5Redesign stock reservation
because Write contention on the stock row is the one crossing no cache or queue touches; it is a design change and is done last because it is the most expensive.
What each component actually touches
The diagram is the store's request path with the three candidate components drawn where they sit. The point is geometric: the cache is on the read path and the checkout path never passes through it; the queue is after the order is written and cannot make the write faster. A component that is not on the path of the bottleneck cannot move it.
How to do it
Most important first.
- Write the request budget for the core workflow: rows read, rows written, external calls, bytes returned, per request. It does not need to be precise; it needs to distinguish a page that reads ten rows from one that reads ten thousand.
- Pick capacities you can defend for each component — from the documentation, from a single load test, or from a stated rough guess labelled as such. Rough and labelled beats precise and invented.
- Multiply at each size and find the first component to cross. Write the size and the component down; that pair is the finding.
- Before adding a component, say in one sentence what reading it moves and what reading it leaves alone. If you cannot, you do not know what it is for (Add Complexity Only When Required).
- Add one component, re-run, find the next crossing. The order in which bottlenecks appear is the architecture's growth path, and it is different for a read-heavy catalog and a write-heavy checkout.
- Use the growth lab to check your model against one that has been worked through — not because its numbers are yours, but because the shape (nothing at low traffic; database CPU first on reads; a cache that does not touch checkout) is the shape to recognise (Architecture From Requirements).
Worked on a concrete problem
The move has to produce something. This is what it produced.
- The store at launch size, a few hundred users a day: the model says the database CPU is idle, connections are plentiful, and the payment provider is the slowest thing in checkout. Adding an index, a cache or a replica moves no reading. The team's three opinions are all correct about *something* and all unjustified *now* — and that is written down, with the sizes at which each would become justified.
- Raise the size until something crosses. Product-page reads cross database CPU first, because every view reads the product plus its images plus its stock, and views outnumber checkouts by a large ratio. The bottleneck is reads; the first justified component is an index on the query the plan flagged, and it moves the crossing an order of magnitude out. The cache is second, and only if the index is not enough.
- Raise it again. Now checkout contends on the stock row and on connection count, and neither the index nor the cache touches it — the model says so before anyone builds anything. The candidate is a queue for the email and the provider call, which shortens the request but does not reduce the row contention; that needs a different decision about how stock is reserved. Two different bottlenecks, two different components, and the cache that "everyone knew we needed" never addressed the second.
- The constraint is the launch size, so nothing is built except the index, which was a correctness-adjacent fix anyway. The rest is a note: "reads bend first, around size X, index then cache; checkout bends around size Y, queue for email and a reservation redesign for stock". The next scaling conversation starts from that note.
How you know it worked
What now exists that did not before, and what question you can now ask.
- For each size, you can name the component that crosses first and show the multiplication that says so.
- Every proposed component has a sentence saying what reading it moves — and at least one proposed component has been rejected because it moves nothing at the current size.
- The growth path is written down in order: what bends, at what size, and what would move it; nothing on the list is built until its size is real.
- The word "safe" has left the conversation, replaced by readings.
The questions you can now ask
The field this whole domain exists for. After this lesson, these are the questions to put to an unfamiliar problem.
- ?How much of each component does one request of the core workflow use, and at what request rate does the first one cross its capacity?
- ?Which reading would this component move, and which would it leave alone?
- ?At the size the constraint actually named, is anything crossing — and if not, what am I adding this for?
- ?What is the order in which bottlenecks appear as the size rises, and where does the constraint sit on that path?
What can go wrong
- The model is trusted like a measurement. Rough capacities and imagined request rates produce a bottleneck that is a hypothesis, and the component built on it may be the wrong one. The model says where to load-test; the load test says what is true.
- The size is raised past the constraint and the components for that size are built, because now there is a number that looks like a reason. A number from a model is a reason to write a note, not a reason to build.
- Only one axis is raised. Users go up; catalogue size, order history and image sizes stay flat, and the bottleneck that arrives first in reality — a full-table scan over a year of orders — never appeared in the model.
- The model is built once and never revisited. The launch was fine, the retailer arrives, and the year-old note is read as if the workload had not changed shape.
- A capacity model is work — a request budget, defensible capacities, a spreadsheet — and its precision is low; the return is direction, not numbers.
- Rejecting a component "everyone knows we need" costs social capital, and if the model is wrong the rejection is remembered.
- Writing the growth path down commits you to a picture of the future workload that may be wrong in shape, not just in size.
- "The model tells me what to build." It tells you what to measure first and what would be unjustified now. The measurement tells you what to build.
- "A cache never helps checkout." A cache helps checkout when checkout's bottleneck is a *read* — a product lookup or a price fetch on the request path. The lesson is that you have to know which reading you are moving, not that caches are for catalogs.
- "At launch size, do nothing." At launch size, do the correctness work, fix the query that the plan flagged, and write the note. "Nothing" is the architecture; the thinking still happens.
Where this applies
Problem-solving advice is stated as universal far more often than it is. These labels say what each method is specific to — and where CONTESTED appears, the note gives the strongest form of the opposing view.
- GENERALRequest budget × rate against component capacity, raised in steps, is how any system's growth path is found — a store, a chat app's fan-out, a file-upload service's storage bandwidth; the components differ, the multiplication does not.
- ILLUSTRATIVEThe launch size, the "few hundred a day", the ratio of views to checkouts and the order in which the store's bottlenecks appear are invented for the shape of the argument; the growth lab's numbers are a worked model, not a benchmark of any real store.
- SCALE-SPECIFICBelow the size where anything crosses, the whole exercise reduces to "write the note"; a team already operating at the size where two components cross at once needs a measured profile rather than a model, and the model's job there is only to explain what the profile shows.
Where the depth lives
This domain asks the question and hands the answer off by name.