Reproduce It First
A bug you cannot reproduce cannot be shown fixed. Before hypotheses, get the failure to happen on demand — the exact input, state and environment — and write down the recipe, because the recipe is the test the fix has to pass.
The situation, the reflex, and why it stalls
Every lesson starts where being stuck starts: someone has a problem, and the first move that comes to mind feels like progress.
A customer says checkout failed, you try it and it works. What do you do before forming any theory about the cause?
Support forwards a message: "I tried to pay and it said Payment failed, twice." You open the store, add the same product, pay with a test card, and it works. There is no error in the logs from the last hour. You could close the ticket as "cannot reproduce" or start guessing.
Guess at the cause from the description and patch the guess. It feels responsive — a customer is waiting — and a plausible story ("probably their card was declined") is easy to produce.
The patch cannot be verified. If the failure never happened on your machine, the fix's absence of failure on your machine proves nothing, and the ticket comes back with "still broken".
- The patch cannot be verified. If the failure never happened on your machine, the fix's absence of failure on your machine proves nothing, and the ticket comes back with "still broken".
- The guessed story becomes the accepted story. "Card declined" is written on the ticket, the customer is told to try another card, and the real cause — a race between two clicks — stays in the system for every other customer.
- Without a reproduction, every hypothesis is tested by deploying it and waiting for the customer, a feedback loop measured in days when it could have been seconds (Short Feedback Loops).
- The intermittent bug is filed as flaky and ignored. Intermittent means the reproduction has a condition you have not named yet; it does not mean the bug is random.
The move
Precisely enough to apply it to a problem you have never seen — not a slogan.
- Make the failure happen on demand before trying to explain it. The reproduction is the concrete instance of the abstract complaint, and the domain's move for anything abstract is to construct a concrete case (Example-Driven Thinking).
- Close the gap between your attempt and theirs, one variable at a time. Same product, same quantity, same account state, same browser, same time pressure, same number of clicks. Each variable you match either reproduces the failure or is ruled out; a reproduction that appears when you match a variable also tells you what the cause depends on.
- When it reproduces, write the recipe down: the exact steps, the state before them, the observation that counts as failure. The recipe is not documentation; it is the experiment you will run after every candidate fix.
- When it does not reproduce after matching everything you can, treat the remaining variables as the hypothesis space — timing, concurrency, data specific to that customer, provider state — and instrument for them rather than guessing among them.
Reproduce or instrument
The first decision on any report is whether to chase a reproduction now or to instrument and wait. The criteria are how much the report already tells you and how expensive matching the reporter's conditions is; neither option is always right.
Can I make this fail on demand, and is it worth trying now?
when The report names input and steps; the environment can be matched locally; the bug is deterministic or nearly so.
cost Time matching variables; the risk of reproducing a similar but different bug in an unlike environment.
when A stack trace or log already names the cause unambiguously.
cost If the evidence was misread, the fix ships without ever having seen the failure.
when Every matchable variable is matched and it still does not fail; the missing variable is timing, load, or state you cannot see.
cost Days until the next occurrence; the instrumentation must be designed to capture the right thing the first time.
when The cause is suspected to depend on data volume, concurrency or the real provider.
cost Environment setup; staging drifts from production and can itself mislead.
Closing the gap one variable at a time
A reproduction attempt is a sequence of experiments, each matching one more of the reporter's conditions. The pipeline is worth following in order because a cheap match that reproduces saves the expensive ones, and because the variable that flips the result is information about the cause.
- 1Exact report
Message, time, steps, frequency, account — from the reporter, verbatim.
fails by Working from the support summary instead of the customer's words.
- 2Same path
The same UI route, the same product, the same quantity.
fails by Using a convenient test product with none of the customer's properties.
- 3Same state
Their account, cart and order history, or a copy of it.
fails by A clean account that has never had a failed payment or a stale cart.
- 4Same timing
Two clicks, a slow network, a tab left open — the things a customer does that an engineer does not.
fails by Assuming the customer used the page the way it was designed.
- 5Same environment
Their browser, their locale, the provider's state at the time.
fails by Ignoring the provider status page because "that's external".
- 6Recipe
Preconditions, steps, expected, actual — written down, ideally as a failing test.
fails by Fixing the bug and losing the recipe.
The recipe as an artefact
A recipe is short and specific. It reads like a test because it is one, whether or not it is ever automated. The example below is the double-click case; note that "actual" records what was observed in the database, not just on the screen, because the screen is where the symptom is and the database is where the damage is.
1Preconditions2 - any cart with one product, stock >= 13 - test-mode card that succeeds4 5Steps6 1. open checkout for the cart7 2. click Pay twice within about a second (or send two POST /checkout/pay concurrently)8 9Expected10 - exactly one order, status paid; page shows confirmation11 12Actual13 - first request: 200, order A paid14 - second request: 500, "Payment failed" shown15 - orders table: order A paid, order B created and unpaid for the same cart16 17Depends on18 - two requests for the same cart overlapping; does not reproduce with sequential clicksThe "depends on" line is the part a fix will be judged against: any change that makes overlapping requests safe passes, and any change that merely disables the button does not, because the API is still reachable.
How to do it
Most important first.
- Get the exact observation from the reporter: the message, the time, what they did just before, whether it happened once or every time. "Twice" is data — it means the second attempt failed too, or that they clicked twice.
- Reproduce in the most similar environment you have: their account or a copy of its state, the same product, the same path through the UI. Test data that differs from theirs is a variable, not a shortcut (Edge Cases From Examples).
- Vary one thing at a time towards their conditions until it fails. Note which variable made the difference — that is a fact about the cause.
- Write the recipe: preconditions, steps, expected, actual. If it can be a test, make it one now, before the fix, and watch it fail (Invariants as Tests).
- If it will not reproduce, decide what to log for the next occurrence: the request id, the order state, the provider response, the timing. Then wait with instrumentation instead of guessing (Logs Are Evidence, Not Thinking).
Worked on a concrete problem
The move has to produce something. This is what it produced.
- The ticket: "Payment failed, twice." Matching variables: same product — works; same quantity — works; a cart with a product whose stock is exactly the quantity ordered — works; clicking Pay twice quickly — the second request gets "Payment failed" and, in the orders table, the first order is paid and a second order is created and unpaid. Reproduced: the recipe is "double-click Pay on a cart with any product". The cause is now a specific question about idempotency (Duplicate Requests).
- The recipe as a test, before any fix: create a cart, send two pay requests for it concurrently, assert exactly one order exists and it is paid. It fails. Every candidate fix — disabling the button, an idempotency key, a unique constraint on cart id — is judged by whether this test passes, not by whether the customer stops writing in.
- A case that does not reproduce: "Payment failed" for one customer, once, at a time when the provider's status page shows an incident. Matching every variable fails to reproduce, because the variable was the provider's state. The right output is not a fix but a log line that records the provider's response code and a decision about what the customer should see when the provider is down (External Systems Fail).
How you know it worked
What now exists that did not before, and what question you can now ask.
- You can make the failure happen whenever you want, and you can say which condition it depends on.
- A written recipe or a failing test exists, and it was written before the fix.
- Fixes are judged by the recipe, not by the absence of new complaints.
- For the bug that will not reproduce, there is a specific list of what to record next time, and it is in place.
The questions you can now ask
The field this whole domain exists for. After this lesson, these are the questions to put to an unfamiliar problem.
- ?Can I make this failure happen on demand, and if not, which variable have I not yet matched to the reporter's conditions?
- ?What is the exact recipe — preconditions, steps, expected, actual — and can it be a test?
- ?Which condition, when I matched it, made the failure appear — and what does that say about the cause?
- ?If it will not reproduce, what would I need to have recorded at the time to know what happened?
What can go wrong
- Days spent reproducing a bug whose cause is already obvious from the stack trace. When the evidence names the cause, reproduce the fix rather than the bug — write the test, apply the fix, watch it pass.
- Reproducing in an environment so different from the customer's that the reproduction is a different bug. The test database has one product and no concurrency; theirs has neither property.
- The recipe is found and not recorded. The bug is fixed, the recipe is forgotten, and the regression next quarter starts from zero.
- Instrumentation for the unreproducible case is added and never read. A log line nobody looks at is a promise, not an observation.
- Reproduction can take longer than the fix. For a one-line typo visible in the stack, insisting on a reproduction first is ceremony.
- Reproducing production conditions — real data volumes, real concurrency, the real provider — is sometimes impossible locally, and the honest reproduction is then a staging run or an instrumented wait.
- A reproduction recipe is a test, and a test that reproduces a race is itself hard to make deterministic; the cost of a reliable reproduction for concurrency bugs is real (Invariants Under Concurrency).
- "Cannot reproduce means not a bug." It means the reproduction has a condition you have not identified. The customer observed something; the observation is real even when its cause is not yet visible.
- "It works on my machine" is a statement about my machine, not about the bug. Made precise, it says: the failure depends on a variable that differs between my machine and theirs — which is the start of the reproduction, not the end of the ticket.
- "A reproduction is a test case." Often, but a reproduction can be a manual recipe when a test would be disproportionate; what matters is that it is written down and repeatable.
Where this applies
Problem-solving advice is stated as universal far more often than it is. These labels say what each method is specific to — and where CONTESTED appears, the note gives the strongest form of the opposing view.
- GENERALAny bug in any system; the variables that need matching change — browser and account for a frontend bug, data and timing for a backend one, hardware for an embedded one — and the move does not.
- DOMAIN-SPECIFICFor a payments bug, reproducing against the real provider is often impossible and unsafe; the reproduction uses test mode and a recorded provider response, and the gap between that and production is itself a variable to name.
- ILLUSTRATIVEThe double-click, the duplicated unpaid order and the provider incident are invented to show how a reproduction narrows the cause; no real system is described.
Where the depth lives
This domain asks the question and hands the answer off by name.
- — Testing & Reliability has no domain of its own yet; the practice of writing the failing test before the fix is the reproduction recipe automated.