Problem Solving and System Design
Requirements → Scale → Bottlenecks → Architecture. Architecture is the last step, and the reflex starts there. The requirements say what must hold; the scale says how much load; the bottleneck is what actually breaks first under that load; the architecture is the smallest change that relieves it.
The situation, the reflex, and why it stalls
Every lesson starts where being stuck starts: someone has a problem, and the first move that comes to mind feels like progress.
The store has to "scale". How does the loop get from that word to an architecture decision with evidence behind it, and what does it hand to the architecture and distributed-systems domains?
The founder says the store will be on television and asks whether it will hold up. You have one server and one database. You can picture the diagram people draw for this — load balancer, replicas, cache, queue, maybe services — and you could draw it now. You cannot say which box the store actually needs, or what would break first without it.
Draw the scalable architecture. Every box in it solves a real scaling problem, the diagram is what "ready for scale" looks like, and drawing it produces a plan someone can approve. The alternative — saying "I do not know what will break" — sounds like not having a plan.
Every box solves a scaling problem the store may not have. A cache fixes repeated reads; a queue fixes slow synchronous work; replicas fix read load on the database. Without knowing which of those the television spike produces, the diagram is a list of solutions and the problem is still one word (The Why Ladder).
- Every box solves a scaling problem the store may not have. A cache fixes repeated reads; a queue fixes slow synchronous work; replicas fix read load on the database. Without knowing which of those the television spike produces, the diagram is a list of solutions and the problem is still one word (The Why Ladder).
- Each box added is a component to operate, a failure mode to handle and a consistency question to answer. The cache serves a stale price; the queue delays the confirmation email; the replica lags and the order page shows nothing. None of those was in the requirements, and all of them now are (The Complexity Ledger).
- The diagram cannot be tested. "Will it hold up?" has no number in it, so nothing can be measured against it, so the diagram is approved on the strength of looking like the diagrams that scale.
- The actual bottleneck — which in a small store under a burst is very often the database's write path at checkout, or a single slow external call — is not addressed by any of the read-scaling boxes, and the television night finds it.
The move
Precisely enough to apply it to a problem you have never seen — not a slogan.
- Run the chain: Requirements → Scale → Bottlenecks → Architecture. Requirements are what must hold under load — orders are never lost, prices are correct, the page loads. Scale is the honest number: how many users, how many checkouts, over what period, with what evidence. Bottlenecks are what breaks first at that scale — found by reasoning about each component's limit, and confirmed by a load test or a thought experiment with numbers. Architecture is the smallest change that relieves the bottleneck found, and only that one.
- Get the scale number from evidence, not from fear. "On television" becomes "how many viewers, what fraction click, over how many minutes" — a rough estimate with a range, written down with its assumptions (Scale Thought Experiments).
- Find the bottleneck before choosing the box. Walk the request path — page, API, database reads, checkout writes, provider call — and for each ask what it can sustain and what the estimated load is. The component whose limit is closest to the load is the bottleneck, and it is usually one, not six (The Bottleneck Moves After Every Fix in the performance domain says what happens after you fix it).
- Add one component, for the bottleneck found, with its ledger entry: what it solves, what it introduces, what happens when it fails. Then measure again, because the bottleneck moves. This is the loop the growth lab at /thinking/grow runs; the architecture and distributed domains teach each component in depth (Architecture From Requirements).
The chain, one component per pass
Four steps that repeat. The architecture and distributed domains own the last step in depth — what a cache, a queue, a replica or a service boundary is and how it fails. This domain owns the three steps that say which one, and whether any.
- 1Requirements
What must hold under load, as sentences that can fail: no lost order, correct prices, page within budget.
fails by "It should scale."
- 2Scale
A number with a range, a period and written assumptions — from evidence or an honest estimate.
fails by A number chosen to justify the diagram.
- 3Bottlenecks
The request path walked; each component's limit against its load; the smallest ratio named and confirmed.
fails by Assumed to be reads, because that is what the diagrams fix.
- 4Architecture
One component for the confirmed bottleneck, with a ledger entry; then measure again.
fails by Six components, drawn once, never measured.
The growth lab at /thinking/grow runs exactly this loop: raise the load, read the bottleneck, add one thing, read again. Its honest half is that at low load nothing is justified, and the chain says so too.
"We need microservices", walked down the ladder
The claim that most often arrives with the scaling diagram is the one below. The ladder runs it to the requirement underneath, names the simpler thing that meets it, and says when the claim was right — because sometimes it is, and the device exists to find out which time this is.
“We need to split the store into microservices before the television spot.”
- ↓Why microservices? So the parts can scale independently — checkout under load should not take the catalog down.
- ↓Why would checkout take the catalog down? Because they share request handlers, and if checkout handlers are all waiting on the payment provider, catalog requests queue behind them.
- ↓Why are checkout handlers waiting on the provider? Because the charge call is synchronous in the request, and the provider is slow under load.
the claim was right when When the parts genuinely have different scaling shapes and different teams — a catalog read at enormous volume by many clients, a checkout owned by a payments team with its own release cadence and compliance boundary — and the cost of operating separate services is already being paid for organisational reasons. Then the split is right, and the ladder found the same requirement with a different answer (Microservices Are a Distribution Decision, Not a Scaling Technique in the distributed domain says what the split costs).
The components, scored honestly
The matrix scores the candidate responses to the television burst on the axes the decision moves. The scores are relative and rough; the caveat says what they cannot say, which is the part the diagram never admits.
| Option | Simplicity | Reliability | Cost | Time | Maintainability | Note |
|---|---|---|---|---|---|---|
| Nothing yet; watch the walk's smallest ratio | Right when the walk finds margin at the estimated load. Reliability depends on the estimate being honest. | |||||
| Provider call moved to a background job | Right when the bottleneck is handlers waiting on the provider. Introduces a pending state and a worker to operate. | |||||
| Cache in front of the listing | Right when the bottleneck is repeated reads. Introduces stale prices and an invalidation path that must be built and tested. | |||||
| Read replica | Right when the database read path is the bottleneck and the cache cannot serve it. Introduces replication lag on the order page. | |||||
| Microservices | Right when the parts have different scaling shapes and teams. Introduces the network between every call (The Network Changes Everything). |
caveat The numbers rank options for one store under one estimated burst and say nothing about a different store. They cannot say which bottleneck you have — only the walk can — and a low-scoring option is the right one exactly when the walk names its bottleneck. The matrix is for comparing responses to a known bottleneck, not for choosing without one.
How to do it
Most important first.
- Write the requirements that must survive the load as sentences that can fail: "no order is lost", "no price shown is more than a minute stale", "the product page responds within the budget".
- Estimate the scale with a range and its assumptions: viewers, click-through, minutes, checkout fraction. Mark it as a guess; it is the input to everything after.
- Walk the request path and write each component's rough limit next to the estimated load on it. The smallest ratio is the bottleneck candidate.
- Confirm it: a load test against a copy if you can, a thought experiment with the numbers if you cannot. Predict what breaks first, then look (Prediction Before Execution).
- Add the one component that relieves the confirmed bottleneck, with its ledger entry, and repeat from the walk. Stop when the requirements hold at the estimated scale with margin.
Worked on a concrete problem
The move has to produce something. This is what it produced.
- The store on television, run through the chain. Requirements: no order lost; prices correct; product pages load. Scale, estimated: a burst of viewers over a few minutes, a small fraction clicking through, a smaller fraction checking out — written with a range. Walk: the product page is a read of a few rows, cheap; the API is stateless and can be duplicated; the database read path is well within limits at the estimated page load; the checkout write path is a transaction per order, and at the high end of the checkout estimate it is the first thing near its limit; the provider call is synchronous and its latency, not its throughput, is the problem — a slow provider ties up request handlers. Bottleneck: checkout handlers held open by the provider. Architecture: move the provider call after the response — order pending, charge in a background job, confirmation when it completes. One component, a job queue, with its ledger. No cache, no replica, no services: the bottleneck was not reads.
- The same chain, six months later, with real numbers. The store has grown steadily and the product listing is the slowest page. Walk: the listing query reads every product on every page view, and the database's read path is now the smallest ratio. Bottleneck confirmed by a query plan and a load test. Architecture: a cache in front of the listing, with a ledger entry that names the stale-price failure mode and the invalidation on admin edit. The box the television diagram drew first was the right box on the second pass, for a reason that existed by then.
- The URL shortener, where the chain says "distribute". Requirements: redirects never fail, even if the admin interface is down. Scale: redirects vastly outnumber everything else. Walk: the redirect path is a key lookup and can be served from a replica or a cache; the admin path is rare. Bottleneck at the estimated scale: none yet — but the requirement that redirects survive an admin outage is an availability requirement, not a load one, and it does justify separating the redirect path. The chain sends you to the distributed domain's "why distribute" and "when not to distribute" lessons with the reason in hand.
How you know it worked
What now exists that did not before, and what question you can now ask.
- The scale is a number with a range and written assumptions, not the word "scale".
- The request path has been walked and each component has a rough limit next to its estimated load.
- One bottleneck is named, and it was predicted before it was confirmed.
- The architecture change is one component, with a ledger entry naming what it introduces and how it fails.
The questions you can now ask
The field this whole domain exists for. After this lesson, these are the questions to put to an unfamiliar problem.
- ?What must still hold under the load — and can I write each requirement as something that can fail?
- ?How much load, over what period, with what evidence and what assumptions?
- ?Walking the request path, which component's limit is closest to the estimated load on it?
- ?What is the smallest change that relieves that one bottleneck, what does it introduce, and where does the bottleneck move next?
What can go wrong
- The walk becomes a capacity model of every component to three significant figures. The walk needs ratios, not precision; the smallest ratio is the answer, and a factor of two does not change it.
- The bottleneck is found and then the whole diagram is built anyway "while we are at it". The chain adds one component per confirmed bottleneck; the rest are ledger entries with no reason.
- The scale estimate is refused because it is uncertain. A wide range with assumptions is more useful than no number; the walk works with ranges.
- The chain is re-run for every feature. It is for load and availability questions; adding a coupon field does not need a bottleneck walk.
- The chain delays the diagram, and a diagram is what the founder wanted to see when they asked whether it would hold up. "One queue, and here is why" is a harder sell than six boxes.
- Finding the bottleneck by measurement costs a load-test environment or a careful thought experiment; the reflex costs a drawing.
- Adding one component at a time means the architecture is revised more often, and each revision is visible work. The reflex's architecture is revised too, but invisibly, as its unneeded boxes are debugged.
- "So never add a cache before it is needed." Add it when the walk names reads as the bottleneck, which may be on day one for a read-heavy product. The chain is about evidence, not about delay for its own sake.
- "Bottleneck means performance." It means the first requirement to fail under load, which can be availability — the shortener's redirects — or correctness — oversell under concurrent checkout. The walk checks each requirement.
- "This is architecture." It is the step before: finding the reason. The patterns — caching, queues, replication, service boundaries — are the architecture domain's and are linked, not restated.
Where this applies
Problem-solving advice is stated as universal far more often than it is. These labels say what each method is specific to — and where CONTESTED appears, the note gives the strongest form of the opposing view.
- GENERALRequirements → Scale → Bottlenecks → Architecture applies to any system asked to carry load; what varies is which component is usually the bottleneck — a database write path for a store, a fan-out for a chat app, a single external call for many.
- SCALE-SPECIFICAt small scale the walk finds no bottleneck and the chain's answer is "nothing yet, and here is what to watch". At large scale several components approach their limits together and the chain runs repeatedly, one component per pass — the growth lab shows both ends.
- CONTESTEDA serious opposing view: some architecture decisions are effectively irreversible — the choice of a single database, synchronous coupling to a provider, a monolith with shared state — and waiting for a measured bottleneck means paying the migration cost under load, at the worst moment. Proponents argue that the scalable shape should be chosen early where it is cheap to choose and expensive to change later, and that "add one component per bottleneck" is only safe when every component is cheap to add. The view is strongest for systems with a known, imminent load and for decisions like data partitioning that cannot be retrofitted without downtime.
- ILLUSTRATIVEThe television burst, the six-month listing bottleneck and the shortener are invented to show the chain choosing one component at a time; the load figures are ranges for the shape of the argument, not measurements.
Where the depth lives
This domain asks the question and hands the answer off by name.
- — The architecture domain's "from simple to scaled" lesson is the chain run many times over a system's life; arrive with the walk and the named bottleneck, and each pass has a reason.