Short Feedback Loops
Change, run, observe, fix — minutes per cycle — beats design for three weeks, build everything, and discover on the last day that one assumption was wrong. The reason is not that thinking is bad; it is that the cost of a wrong assumption grows with everything built on top of it, and a short loop finds it before anything is.
The situation, the reflex, and why it stalls
Every lesson starts where being stuck starts: someone has a problem, and the first move that comes to mind feels like progress.
How long is it between making a change and knowing whether it was right — and what does that number cost you?
The store's checkout redesign has been in progress for three weeks. There is a design document, a new data model, a payment step, an inventory step and a notification step, all coded, none run end to end. Today the first end-to-end run shows that the payment provider needs the order to exist before the charge — and the design created the order after. Everything downstream of that assumption is affected.
Design it properly first. Think the whole checkout through, write it down, get it reviewed, then build it in one pass so there is no rework. It feels like the mature approach — the opposite of hacking — and rework is a real cost that up-front design is meant to avoid.
The design was built on an assumption that a single run would have refuted in the first hour. Three weeks of work sits on top of "create the order after the charge", and every piece that touches it has to move.
- The design was built on an assumption that a single run would have refuted in the first hour. Three weeks of work sits on top of "create the order after the charge", and every piece that touches it has to move.
- Nothing was observed for three weeks, so nothing was learned for three weeks. The design document got longer; the understanding of the provider did not change, because understanding comes from contact.
- The review reviewed the document, not the system. Reviewers could check the design against their beliefs; nobody could check it against the provider, because the provider had not been asked.
- Rework is now maximal — the outcome the approach was chosen to prevent. Up-front design does not avoid rework; it postpones discovering the need for it until the most expensive moment.
The move
Precisely enough to apply it to a problem you have never seen — not a slogan.
- Measure your loop: the time from a change to an observation that tells you whether it was right. Then shrink it, deliberately, as its own task. A loop of minutes means an assumption is tested the hour it is made; a loop of weeks means it is tested after everything built on it, and the cost of being wrong is that everything.
- Order the work so that the first thing run end to end is the thinnest path through every layer — the walking skeleton — and every subsequent change is run against it. The skeleton is the loop's instrument: once it exists, "does the provider need the order first?" is a ten-minute question instead of a three-week one (The Walking Skeleton).
- Design *inside* the loop, not instead of it. Think, write the intent down, then run the smallest thing that could refute it, then think again with what you saw. The document still exists; it is updated per cycle rather than defended per quarter. What changes is not the amount of thinking but when the thinking meets reality.
- Treat a slow loop as a bug in the process with a cause: a test suite that takes an hour, a deploy that needs a ticket, an environment only one person can run, a provider with no test mode. Each cause has a fix, and the fix is usually worth more than the feature it is delaying (The Smallest Executable Thing).
Two ways to spend three weeks
The pair below is the same checkout, the same wrong assumptions, two orders of work. The engineering reason the second wins is not speed — it is *where* the wrong assumptions were discovered relative to the work that depended on them.
Week one: design document, data model, review. Weeks two and three: order creation, payment step, inventory step, notification step, each coded against the document. Day fifteen: first end-to-end run. The provider needs the order first; the callback can race the response. Both assumptions are in the document and in every step built on it.
Hour one: cart → order → test charge → log line, running. First assumption refuted. Hour two: order first, then charge; callback observed racing the response; second assumption refuted. Afternoon: inventory, duplicates, declines, timeouts, each a small change against the running skeleton. The design page is updated four times, each time by evidence.
Both approaches hit the same two wrong assumptions. In the loop each was found before anything depended on it and cost an hour; in the up-front version each was found after everything depended on it and cost the rework of all of it. Up-front design did not prevent the rework — it maximised it by postponing the run.
What the loop length actually trades
The matrix scores the two approaches, plus the hybrid the lesson recommends, on the axes an engineering decision moves. The caveat is doing real work here: these are directions, not measurements, and the "design first" column wins on an axis the matrix cannot show.
| Option | Simplicity | Reliability | Time | Maintainability | Note |
|---|---|---|---|---|---|
| Design fully, then build, then run | Coherent on paper; every untested assumption is load-bearing by the time it is tested. | ||||
| Loop only, no written intent | Fast contact with reality; the result has no shape beyond the cases tried. | ||||
| Skeleton first; design per cycle; irreversible decisions designed up front | Assumptions tested in the hour they are made; a one-page design that stays current; the schema and the public contract still get the slow treatment. |
caveat The numbers are directions, not measurements, and they cannot show the axis that matters most to the design-first camp: for a decision whose first run is production — a shared schema, a published API — the loop scores zero on reliability because there is no cheap run, and design-first is the only column that applies. Which decisions those are is a judgement the matrix cannot make.
An order that keeps the loop short
The sequence below is one way to build checkout so that every step is run within minutes of being written and the riskiest assumptions meet reality first. The alternative is real and is the honest answer to the contested scope: when the schema is the risk, design it first.
- 1Make checkout runnable end to end locally against the provider's test mode
because This is the loop's instrument; without it every later step is observed in production or not at all.
- 2Skeleton: one item, one order row, one test charge, one log line
because The thinnest path through every layer; the first run refutes or confirms the provider's ordering assumption in an hour.
- 3Riskiest assumption next: callback timing and the order reference
because If wrong it moves the data model, and the model has nothing built on it yet.
- 4Inventory decrement, duplicates, declines, timeouts — one per cycle
because Each is a small change observed in minutes; the design page is updated as each one teaches something.
- 5Only now, the parts a run cannot test: the order table's final shape, the API other teams will call
because These are the irreversible decisions, and they are designed with everything the loop has taught, not before it.
How to do it
Most important first.
- Time the loop today. From editing a line in checkout to seeing whether an order is created correctly: how many minutes? If you cannot run checkout locally end to end against a test-mode provider, that is the first task.
- Build the skeleton before the design is complete: cart → order → charge in test mode → confirmation, each step as thin as it can be. Run it. Now the design has something to be wrong against.
- Make each change small enough to run. If a change cannot be observed in one cycle, split it; the split is usually along the same seams the design would have found (Vertical Slices).
- Put the riskiest assumption into the loop first — the one whose failure would move the most. "Does the provider need the order first?" is worth a cycle before the notification step exists (Risk-First Development).
- Keep the design document as a record of intent that each cycle can update. One page, current, with the assumptions that the last cycle tested marked as tested.
- When the loop gets slow, stop and fix the loop. A cached test database, a provider mock for the fast path with the real test mode for the slow one, a script that resets state — these are the tools of the job, not overhead.
Worked on a concrete problem
The move has to produce something. This is what it produced.
- Checkout, done as a loop. Hour one: a skeleton — a cart with one item, an order row, a test-mode charge, a log line. Run. The provider rejects the charge because no order reference was supplied. The assumption "order after charge" is refuted in the first cycle, before a data model exists to depend on it. Cost: one hour and one line of design.
- Cycle two: create the order first, pass its reference, charge. Run. Success — and the callback arrives before the charge request returns, which the design document had not considered. Second assumption refuted, second cycle. The callback handler now has to find an order by reference, and the data model gets that column before anything is built on top.
- Cycles three to ten, an afternoon: inventory decrement inside the order creation, the duplicate-click case, a declined card, a timeout. Each is a small change against a running skeleton, each observed within minutes, each recorded. The design document is one page and has been updated four times; all four updates were forced by something the system did.
- Compared to the three-week version: the same assumptions were wrong in both. In the loop they cost an hour each and moved nothing; in the up-front version they cost three weeks and moved everything. The design in the loop is not less designed; it is designed against evidence gathered as it was needed.
How you know it worked
What now exists that did not before, and what question you can now ask.
- You can say, in minutes, how long your loop is — and it is short enough that a wrong assumption is found the day it is made.
- The skeleton ran end to end before the design was finished, and the design changed because of what it showed.
- The riskiest assumption was tested first, in the loop, rather than documented first and tested last.
- When the loop got slow, fixing it was treated as work, not as a distraction from work.
The questions you can now ask
The field this whole domain exists for. After this lesson, these are the questions to put to an unfamiliar problem.
- ?How many minutes pass between changing this and knowing whether it was right — and what makes it that long?
- ?What is the thinnest path through every layer that would let me run the next assumption?
- ?Which assumption in the design would move the most if wrong, and can it go into the loop first?
- ?Is this a decision the loop can test cheaply, or one where reversal is expensive enough that design and review must carry the weight?
What can go wrong
- The loop becomes the only thinking. Change, run, observe, fix — with no intent written down — produces a checkout that works for the cases tried and has no shape. The loop is where design meets reality; it is not a substitute for having a design to test.
- The loop is short but observes the wrong thing. A test suite that runs in seconds and checks nothing about the provider gives fast feedback about nothing; the observation has to be the one that could refute the assumption (Experiment Design).
- Loop-shrinking becomes its own project. A perfect local environment is built for a month before the first assumption is tested; the loop should be shrunk enough to run the next cycle, then used.
- The approach is applied where it does not fit: a schema migration on a live database, a protocol that other teams will build against, a hardware interface. Some assumptions cannot be cheaply tested, and there the loop is replaced by more design and more review, deliberately.
- A short loop produces working code before the design is settled, and to a reviewer who expected a document first it looks like hacking; the discipline is invisible from outside.
- Shrinking the loop is real work — environments, test modes, reset scripts — that delays the first feature and pays back only over many cycles.
- Designing per cycle means the design document is never finished, and stakeholders who wanted a finished plan to sign off get a page that keeps changing.
- "So up-front design is wrong." Up-front design is wrong *about the things a run would have told you*; it is right about the things a run cannot — an irreversible schema, a public interface, a consistency guarantee. The slogan "design before you build" is precise only as "design the things you cannot cheaply test; run the things you can, as early as you can".
- "Short loops mean small thinking." The thinking is the same size; it happens more often and with better information. The engineer in the loop thought about the callback race within an hour of encountering it; the engineer with the document thought about it in week four.
- "The loop replaces the skeleton, the slice and the experiment." It is what they are for. The skeleton is the instrument, the slice is the unit of change, the experiment is the observation — and the loop is the cadence at which they run.
Where this applies
Problem-solving advice is stated as universal far more often than it is. These labels say what each method is specific to — and where CONTESTED appears, the note gives the strongest form of the opposing view.
- GENERALThe cost of a wrong assumption grows with what is built on it, so shortening the time to test it pays in any domain where a test is possible — code, queries, infrastructure, prompts; the loop's instrument differs, the argument does not.
- ILLUSTRATIVEThe three weeks, the hour-long first cycle, the afternoon of cycles and the four document updates are invented to show the cost difference in shape; no real project's timeline is being quoted.
- CONTESTEDThe up-front-design position, at its strongest: some decisions are cheap to make and ruinous to reverse — a schema other services depend on, a public API, a consistency model — and for those the loop cannot help, because the first "run" is production; more thinking, more review and more modelling before any code is the only defence. Practitioners in payments, infrastructure and protocol work hold that a team trained on short loops under-designs exactly these decisions and pays for it in migrations. This lesson agrees that the loop is for reversible assumptions and says so; the disagreement is about how many decisions in a typical system are truly irreversible — the loop camp says few, the design camp says more than you think.
Where the depth lives
This domain asks the question and hands the answer off by name.