Failure-First Questions
Five questions to ask of any design, at any boundary: what if it fails, what if it is slow, what if it repeats, what if it arrives out of order, what if it is unavailable. They are dull, they are the same every time, and that is why they work.
The situation, the reflex, and why it stalls
Every lesson starts where being stuck starts: someone has a problem, and the first move that comes to mind feels like progress.
You are looking at a design you have never seen — a chat app, an upload service, a dashboard. What do you ask to find its failure modes without knowing the domain?
I am reviewing a design for a file-upload service someone else wrote. I know nothing about uploads. I can see the boxes and arrows and I have no idea what to ask, so I am about to say "looks good" and hope someone else catches the problems.
Ask about the technology. Which storage, which queue, which framework — those are questions that can be asked without understanding the problem, and they produce a conversation that sounds like a review.
The technology answers say nothing about failure. "Object storage" is correct and reveals nothing about what happens when the upload finishes but the record is never written, which is the question that would have found the design's gap.
- The technology answers say nothing about failure. "Object storage" is correct and reveals nothing about what happens when the upload finishes but the record is never written, which is the question that would have found the design's gap.
- The review is only as good as the reviewer's domain knowledge, so an unfamiliar design gets a shallow review, which is backwards: the unfamiliar design is the one that most needed a method that does not depend on knowing the domain.
- The design's author has already thought about the technology. They may not have thought about the arrows failing, and a question that repeats their own thinking back to them changes nothing.
The move
Precisely enough to apply it to a problem you have never seen — not a slogan.
- Point at each arrow on the design — every place where one thing talks to another — and ask the same five questions: what if this fails? What if it is slow? What if it happens twice? What if it arrives out of order relative to that other arrow? What if the thing on the other end is not there? The questions require no knowledge of the domain, only of where the boundaries are (Where Does My System End?).
- Listen for whether the answer names a state. "It retries" is a mechanism; "the upload is marked incomplete and a job resumes it" is a state and a closer. A design that answers with mechanisms has not been through the failure model; a design that answers with states has.
- Let the domain tell you which questions matter most. In a store, "twice" is the expensive one; in a chat app, "out of order" is; in an upload service, "slow" and "fails half-way" are; in an analytics dashboard, almost nothing is expensive and the questions are asked quickly and the answers are generic. The questions are constant; the ranking is domain-specific (Failure Modeling).
- Write the answers into the design as states and closers, not as a list of concerns. A concern is a feeling about the design; a state is a change to it (States That Must Be Unrepresentable).
From "is this robust?" to a question with an answer
The vague form is the one reviewers ask when they do not know what to ask. The ladder sharpens it to the point where the author has to answer with a state, which is where the review starts being useful.
why The best form names the arrow, the moment, and the three things a failure answer must contain — the state, the observer and the closer — so that "it retries" is no longer an acceptable answer and "we have not decided" becomes a visible gap rather than a vague unease.
The five questions across the case studies
The matrix shows the same five questions applied to one arrow in each of the case-study systems, with the answer that matters most for that domain. Reading across a row shows how constant the question is; reading down a column shows how much the domain changes the weight.
| Arrow | Fails | Slow | Twice | Out of order | Unavailable |
|---|---|---|---|---|---|
| Store: backend → provider "charge" | Declined: failed state, cart kept | Pending state, customer told | Expensive: attempt key before the call | Notification before response: handler must not depend on response | Cannot charge: refuse or queue, a decision |
| Chat: client → server "send" | Local "failed", retry offered | Local "sending" state | Expensive: client message id | Expensive: sender sequence number | Queue locally, send on reconnect with the id |
| Upload: API → storage "write bytes" | Expensive: partial object, record in-progress, resume or clean up | Expensive: progress state visible to the user | Upload id prevents a second record | Completion before record: write the record first | Refuse the upload early; do not accept bytes you cannot store |
| Dashboard: UI → API "query" | Show stale data with a timestamp | Expensive: the whole product is latency | Harmless: reads repeat freely | Harmless for reads | Show the last result and say when it was computed |
What the questions find, and what they miss
The table is the questions turned on themselves: the failure modes of the method. Each is a way the five questions produce a wrong or empty review, with the correction. The last row is the honest one — the method's limit, which is that it finds boundary failures and nothing else.
| Trigger | Symptom | Cause | Response |
|---|---|---|---|
| Asked with equal weight on every arrow | Long review, nothing prioritised, author overwhelmed | The ranking step was skipped | Start from the arrow with the lasting effect; let the domain say which of the five is expensive. |
| Answers accepted as mechanisms | "It retries" closes every row | No follow-up to a state | Ask "and what state is it in then, and who moves it on?" until a state or a "not decided" appears. |
| Design has no arrows | Nothing to point at; the review becomes about technology | Boundaries were never drawn | Ask for the boundaries first; a design without them cannot be reviewed for failure (Where Does My System End?). |
| Domain-specific failure not on the list | The five pass; the system fails on refunds, malware, read state | The five are the floor | Add the domain's own questions from the domain's own lessons; the five get you to the boundary, not through the domain. |
How to do it
Most important first.
- Draw or find the boundaries. If the design has no arrows, ask for them; a design without boundaries cannot be reviewed for failure.
- Ask the five questions of the arrow with the most lasting effect first — the one that charges, sends, writes to something external.
- For each answer, ask "and what state is it in then?" until you get a state or an honest "we have not decided".
- Rank the rows by what the domain makes expensive, and let the low rows have generic answers.
- Ask the author which of the rows they could cause on purpose today. A row nobody can cause is a row nobody has tested (Failure Injection).
Worked on a concrete problem
The move has to produce something. This is what it produced.
- The upload service, arrow by arrow. Browser → API "start upload": twice → two upload records for one file, unless the client sends an upload id. API → storage "write bytes": slow → what does the user see for a large file; fails half-way → partial object in storage, record says in-progress, needs a resume or a cleanup. Storage → API "write completed": out of order → completion arrives before the record exists if the record is written after the storage call; unavailable → completion lost, record stuck in-progress, needs a job that asks storage.
- The chat app, same questions on "send message": twice → the same message appears twice unless the client stamps it with an id; out of order → two messages from one sender display reversed if ordering is by arrival rather than by a sender sequence; unavailable → the sender needs a local "sending" state and a retry that is safe because of the id. Three questions produced the message id, the sender sequence and the pending state — most of the chat app's hard design, with no chat expertise.
- The store, for contrast: the same five questions on "create charge" reproduce the whole failure module — declined, timeout, duplicate, notification before response, provider down — which is the point. The questions are a generator; this module's other lessons are what they generate for one arrow.
How you know it worked
What now exists that did not before, and what question you can now ask.
- You can review a design in a domain you do not know and leave the author with states they had not named.
- The design gained at least one state or one id because of the questions — a message id, an upload id, a pending state.
- The questions have become automatic: you see an arrow and the five questions arrive before you have decided to ask them.
The questions you can now ask
The field this whole domain exists for. After this lesson, these are the questions to put to an unfamiliar problem.
- ?Where are the arrows — every place one thing talks to another — in this design?
- ?For this arrow: what if it fails, is slow, repeats, arrives out of order, or has nothing on the other end?
- ?What state is the system in after each of those, and who moves it out of that state?
- ?Which of these questions does this domain make expensive, and which can have a generic answer?
What can go wrong
- All five questions are asked of every arrow with equal weight, including the arrow from the dashboard to its cache, and the review takes a day and buries the two rows that mattered. The domain ranks the rows; use it.
- The questions are asked and the answers accepted as mechanisms — "it retries", "it is idempotent" — without the follow-up that turns them into states. The questions open the conversation; the follow-up is the review.
- The reviewer supplies the answers. The questions are for finding what the author has not decided; a reviewer who answers them has redesigned the system without the author's knowledge of it.
- The questions are cheap to ask and the answers are expensive to build; a review that finds nine gaps has handed the author nine pieces of work, and the ranking is what keeps that honest.
- A method that needs no domain knowledge also finds nothing that needs domain knowledge; the five questions will not tell you that an upload service should scan for malware.
- "These are the only failure questions." They are the ones that apply at every boundary. Domains add their own: a payments system asks about currency and refunds; a chat app asks about read state; the five are the floor, not the ceiling.
- "Failure-first means design the failures before the happy path." It means ask the failure questions of the happy path once it exists on paper. The arrows have to be there to point at; the contested part is how soon after drawing them you point (Happy Path First, Failure Path Second).
- "A design that answers all five is safe." It is a design whose author has thought about failure; whether the answers work is what injection is for.
Where this applies
Problem-solving advice is stated as universal far more often than it is. These labels say what each method is specific to — and where CONTESTED appears, the note gives the strongest form of the opposing view.
- GENERALThe five questions apply to any boundary in any system; what the domain changes is which answers are expensive, and the ranking step exists to let it.
- CONTESTEDThe strongest opposing view: asking failure questions of every arrow at design time produces defensive designs full of states for failures that never occur, and a team moves faster by shipping the happy path, watching what actually fails, and designing responses to observed failures rather than imagined ones. That view is right for low-stakes systems and for teams with good observability and cheap deploys; it is wrong wherever the first observed failure is a double charge or lost data, because by then the data model has no room for the state. The lesson's answer is to ask all five and let the domain decide which get designed responses now.
- ILLUSTRATIVEThe upload service review and the chat app walkthrough are invented to show the questions producing design changes; neither is a complete design for either system.
Where the depth lives
This domain asks the question and hands the answer off by name.