Iterative Development
Change → run → observe → fix, in loops small enough that each one teaches something. A system is grown from one that already works; scale is added after a working system and a measured bottleneck, not before either exists.
The situation, the reflex, and why it stalls
Every lesson starts where being stuck starts: someone has a problem, and the first move that comes to mind feels like progress.
You have a plan for the whole store and it will take weeks before anything runs end to end. How do you turn that plan into loops that each produce evidence, and when — precisely — does scaling belong in one of them?
The store plan is in a document: schema first, then all the endpoints, then the frontend, then payment, then "make it scale". Nothing will be runnable for a fortnight. You have started on the schema and already changed the Order table three times, each time for a reason you found only by imagining checkout.
Finish the design and then build it. It seems wasteful to run a half-built system — what would you even look at? — and it seems responsible to get the data model right before code depends on it. A week of design up front feels like it saves a month of rework.
The design is being tested by imagination, and imagination is a poor test harness. The Order table was changed three times because each change was found by picturing checkout; running a checkout would have found all three in an afternoon and found the fourth you have not pictured yet.
- The design is being tested by imagination, and imagination is a poor test harness. The Order table was changed three times because each change was found by picturing checkout; running a checkout would have found all three in an afternoon and found the fourth you have not pictured yet.
- Nothing produces evidence for a fortnight, so for a fortnight every decision is made on the same information as the first day. The plan looks like progress because it grows; the store has not yet done a single thing a customer could see.
- The "make it scale" step at the end has no input. No load has been observed, so the scaling work is chosen by what scaling articles describe — a cache, a queue, replicas — rather than by which part of this store is actually slow, if any.
- When something is finally run and does not work, the bug is somewhere in a fortnight of untested code. The first observation of the system is also the largest debugging job it will ever present.
The move
Precisely enough to apply it to a problem you have never seen — not a slogan.
- Treat the system as something that is always running and always changed in small steps: change one thing, run it, observe what happened against what you expected, fix the difference. The loop is the unit of work; a plan is a list of loops in a defensible order, not a list of parts to assemble before the first run (The Problem-Solving Loop).
- Make the first loop the smallest thing that runs end to end — one product, one endpoint, one page — because every later loop needs a running system to change. A walking skeleton is not a milestone; it is the precondition for iterating at all (The Walking Skeleton).
- Size each loop by what you can observe the same day. "Add checkout" is several loops; "create an order from a cart, no payment" is one. If the result of a change cannot be seen until next week, the change is too large to attribute anything to (Short Feedback Loops).
- Make "scale" a loop like any other, with the same precondition: a running system, a measurement that names the bottleneck, and a change aimed at that bottleneck. The slogan "premature optimisation is the root of all evil" is precise enough to test here: optimisation is premature exactly when it targets a bottleneck no measurement has shown. It is not premature once one has.
The loop, and how each step is skipped
The loop is short enough to seem trivial, which is why it is skipped so easily. Each step below has a product and a characteristic way of being faked: a change that is really several, a run that is really a compile, an observation that is really the absence of a stack trace. The third column is the one to check yourself against.
- 1Change
One alteration to a running system, small enough that whatever happens next can be attributed to it.
fails by Several changes at once — a new table, a new endpoint and a new page — so the result belongs to none of them.
- 2Run
The system executes the path the change affects, with real inputs.
fails by "It compiles." The store starts; no order is created.
- 3Observe
What happened, written against what was expected: the order row, the total, the message shown.
fails by No errors, therefore it works.
- 4Fix
The difference between expectation and observation is closed, or the expectation is revised and the reason written down.
fails by Moving to the next feature with the difference unexplained.
The loop returns to Change with a system that does slightly more and a plan that is slightly better informed. That second part is the point; a loop that changed the system and not the plan taught nothing.
When a scaling change is allowed into the loop
Scaling is not forbidden; it is gated. The decision below is the gate: the same change — say, adding a cache in front of the product query — is the right loop in one row and motion in the others. What moves it between rows is not the size of the store or the ambition of the founder but whether a measurement of this system exists and what it named.
A component is proposed — a cache, a queue, a replica. Under which conditions does it enter the loop?
when The store handles a synthetic load; the product query is the slowest thing and the reason; the proposed change targets it.
cost The measurement had to be built — a load generator, a way to read timings — which is work that produces no feature.
when Someone is confident the database will be slow. The confidence may be right.
cost Build the measurement first; it is cheaper than the component and it may show the bottleneck is elsewhere. The component waits one loop, not forever.
when Scaling is on the plan because the plan copied an architecture that had it.
cost The change cannot be observed, so it cannot be a loop. It is a design decision made without evidence — allowed only where a constraint forces it, such as a known launch load that cannot be tested for.
"Make it scale" on the why ladder
The plan's last item is a claim, and claims go on the ladder. Asked why enough times, "make it scale" turns into a requirement that is either already met, met by something small, or genuinely unmet — and in the last case the scaling work is justified, which is what the ladder is for. It is not a device for refusing scale; it is a device for finding out whether scale is the requirement (The Why Ladder).
“Before launch we need to make the store scale — cache, queue, read replica.”
- ↓Why does the store need to scale before launch? Because if it is slow on launch day customers will leave.
- ↓Why would it be slow on launch day? Because there will be a lot of traffic.
- ↓Why do we believe that, and how much is a lot? The founder expects a newsletter to a few thousand subscribers; the traffic is a burst of product-page reads over an afternoon.
- ↓Why would a burst of product-page reads be slow? We do not know that it would be. Nobody has run the store under a burst of reads.
the claim was right when The measured burst saturates the database on reads even with the query fixed, and the reads are of a small set of products that change rarely — then a cache is the loop the measurement asks for, and building it before launch is iteration, not guessing.
How to do it
Most important first.
- Before any new work, ask what currently runs. If the answer is "nothing yet", the first loop is the skeleton and every other item on the plan waits.
- Write each planned item as a loop: what changes, how you will run it, what you expect to observe, what "fixed" would mean. An item you cannot phrase this way is not ready to be built (Prediction Before Execution).
- Observe deliberately, not by the absence of errors. The store "worked" after adding checkout means: an order row exists with the cart's items and prices, and the confirmation page shows the same total.
- Keep a note of what each loop taught, and which requirement or assumption it changed; that record is what makes the next loop better informed than the last (The Engineering Notebook).
- Treat any scaling change as blocked until it can cite a measurement of this system under load that named the thing it fixes (Measure Before You Optimize in Performance says how to measure; this lesson says only that the measurement comes first).
Worked on a concrete problem
The move has to produce something. This is what it produced.
- The store, re-planned as loops. Loop one: a product row, an endpoint that returns it, a page that shows it — runnable in an afternoon. Loop two: admin creates a product and it appears on the page. Loop three: add to cart, see the cart. Loop four: create an order from the cart with no payment. Each loop leaves a store that does strictly more than the one before, and each was observed before the next began.
- The Order table, found by running instead of imagining. Loop four revealed that the order needs the price at the time of purchase, not a reference to the product's current price — the third of the imagined changes. It also revealed that the cart can be empty at checkout and something must be said to the customer — a case no amount of picturing had produced.
- Scaling as a loop with a precondition. After the store ran end to end under a synthetic load of repeated product-page reads, the database was the slowest component and the product query was the reason. That measurement made "add an index on the product lookup" a loop: change, run the load again, observe the query time, done. A cache was on the plan too; the measurement did not name it, so it stayed on the plan.
- The chat app version. The plan said "build message delivery with realtime from the start". As loops: send a message and see it on refresh; then see it without refresh; then see it on a second device. The realtime loop came second, once there was a message to deliver, and it was observed against a real message rather than a diagram.
How you know it worked
What now exists that did not before, and what question you can now ask.
- At any moment there is a version of the system that runs, and the current work is a change to it rather than a part awaiting assembly.
- Every change has an expected observation written before it is run, and the observation was actually made.
- Design decisions are being found by running the system — a price snapshot, an empty cart — rather than by imagining it.
- Every scaling change on the plan can cite the measurement that put it there, and the ones that cannot are still waiting.
The questions you can now ask
The field this whole domain exists for. After this lesson, these are the questions to put to an unfamiliar problem.
- ?What currently runs, and what is the smallest change to it that I could observe today?
- ?What do I expect to see when I run this change — and what would I conclude if I saw something else?
- ?Which decisions am I making by imagination that a running system would make for me?
- ?For each scaling item on the plan, what measurement of this system put it there?
What can go wrong
- Loops so small that nothing is ever decided. Each loop changes one line and observes one thing; the data model is never thought about; the cart is rewritten four times because no loop was large enough to see the shape of checkout. The loop size is a judgment, and "small enough to observe today" is a ceiling, not a target.
- Observe is skipped because the loop was small. The whole value of small loops is the observation; a loop whose result was assumed has cost the time of iteration and bought none of the evidence.
- The skeleton becomes the product. Iterating on a working system is so comfortable that the hard loop — payment, concurrency — is postponed indefinitely in favour of loops that polish. The order of loops still has to be chosen by risk and value (Implementation Order).
- Iteration as an excuse for no design. The data model is not rediscovered from scratch each loop; a loop that reveals the price-snapshot requirement should change the model once, deliberately, with the reason written down.
- Iterating costs rework. The Order table was changed after code depended on it; a perfect up-front design would not have needed that. The bet is that the up-front design would not have been perfect, and the bet is usually right but not always.
- Small loops mean many runs, and if running the system is slow — a long build, a manual deploy — the loop tax is real and the first investment should be in making runs cheap.
- Refusing scaling work until a bottleneck is measured means the first real load may find the bottleneck for you, in production. Where that is unacceptable, the measurement has to be made synthetically before launch, and that is itself a loop.
- "Iterative means no plan." The plan exists; it is a list of loops in an order chosen by dependency, risk and value. What changes is that the plan is revised by evidence after each loop instead of executed blind.
- "Never optimise until production is slow." The rule is measurement first, not production first. A synthetic load on a staging store is a measurement; a scaling article is not.
- "Each loop must deliver a feature." Some loops deliver an observation — a load test, a spike — and nothing a customer can see. Those are loops too, and often the most valuable ones.
Where this applies
Problem-solving advice is stated as universal far more often than it is. These labels say what each method is specific to — and where CONTESTED appears, the note gives the strongest form of the opposing view.
- GENERALChange, run, observe, fix applies at every grain from a product to a function; what varies is how expensive a run is, and that expense sets the loop size.
- STAGE-SPECIFICOn a greenfield store the first loop is the skeleton; in an existing system the skeleton exists and the first loop is the smallest change that touches every layer the new work needs. In a prototype, Observe can be a glance; in production it is a log line, a metric or a test.
- DOMAIN-SPECIFICWhere a run is expensive or dangerous — firmware on hardware, a schema migration on a large table, a payment charge — the loop still holds but the "run" is a simulation, a copy or a test mode, and the cost of building that stand-in is part of the first loop.
- ILLUSTRATIVEThe fortnight, the three schema changes, the synthetic load and the product-query index are invented to show the shape of the loop; no real project is described.
Where the depth lives
This domain asks the question and hands the answer off by name.
- — The Architecture Evolution lab at /thinking/grow is this loop run on the store at rising traffic: every scaling change is either justified by a reading or flagged as unjustified.