Question Quality
The same need asked three ways: "how do payments work?" cannot be answered; "how does a backend confirm a payment?" is answered by a diagram; "which system should be authoritative for whether an order has been paid?" is answered by a decision. The ladder is how you tell where a question stands.
The situation, the reflex, and why it stalls
Every lesson starts where being stuck starts: someone has a problem, and the first move that comes to mind feels like progress.
You have a question about something unfamiliar. How do you tell whether it is good enough to answer yet — and what would make it better?
You have asked a senior engineer "how do payments work with the provider?" and got a patient fifteen-minute overview that you could not use. You asked an AI assistant the same thing and got a longer overview that you also could not use. Both answered the question you asked.
Ask again with more words. Describe the store, the framework, the deadline, the provider; hope that the volume of context makes the answer more specific. It feels like effort, and effort feels like it should be rewarded.
Context added to a vague question produces a longer vague answer. The overview now mentions your framework and still does not say which system decides that the order is paid.
- Context added to a vague question produces a longer vague answer. The overview now mentions your framework and still does not say which system decides that the order is paid.
- The person answering cannot tell what you would do with the answer, so they cover everything, and the useful sentence is buried in twelve others.
- You cannot evaluate the answer, because a question with no answer-shape has no wrong answers. The AI's overview might contain a mistake about retries; nothing in the question would let you notice.
- The real question — where is "paid" decided? — is never asked, because it was never separated from the topic, so the design decision it hides gets made by default, by whichever tutorial the code was pasted from.
The move
Precisely enough to apply it to a problem you have never seen — not a slogan.
- Place the question on a ladder with three rungs. Vague: names a topic ("how do payments work?"). Researchable: names a mechanism and can be answered by a document ("how does a backend initiate and confirm a payment?"). Decisive: names a choice in your system and is answered by a decision you then build ("which system should be authoritative for whether an order has been paid — my database, or the provider?").
- Notice what each rung makes answerable. A vague question is answerable only by an overview. A researchable one is answerable by reading — and produces knowledge but no decision. A decisive one is answerable by a decision, and the decision is the thing the feature was waiting for.
- Climb by asking what you would do with the answer. If the answer to "how does a backend confirm a payment?" is "via a webhook", the next thing you would do is decide whether to trust the webhook or the redirect or your own poll — and that is the decisive question. The rung above is always the question you would ask next, asked now.
- Match the rung to the audience. A senior engineer's minute is best spent on a decisive question; documentation is best at researchable ones; a vague question is for yourself, as the first line on the board, and should not leave the page in that form (Asking People).
The payments ladder
The ladder verbatim from §15. Read the three questions aloud and imagine answering each: the first invites an overview, the second a diagram, the third a decision. The why says what the third makes answerable that the others cannot.
why The best form contains a choice — my database, the provider, or some combination — and answering it produces a design decision that a line of code then implements. The better form produces a diagram of the mechanism, which is necessary for making the choice but is not the choice. The vague form produces an overview, which cannot be wrong and cannot be built.
What each rung can be answered by
The rungs differ in who or what can answer them, in what the answer is made of, and in whether a wrong answer can be detected. The matrix is the assessor's heuristics written out.
| Rung | Example | Answered by | Answer is | Can it be wrong? |
|---|---|---|---|---|
| Vague | How do payments work? | Anyone, at length | An overview | No — there is nothing to be wrong about |
| Researchable | How does a backend initiate and confirm a payment? | Documentation, an experiment | A mechanism | Yes, and the experiment shows it |
| Decisive | Which system is authoritative for "paid"? | The owner of the area; you, after research | A decision with a trade-off | Yes — and it is the decision the feature was waiting for |
How the rungs go wrong
Each rung has a characteristic misuse, and the symptom is always the same: an answer that could not be used. The table is for diagnosing which misuse produced it.
| Trigger | Symptom | Cause | Response |
|---|---|---|---|
| A vague question sent to an expert | A patient overview; nothing to build | The question named a topic, so the expert covered the topic | Climb: what would you do with the answer? Ask that. |
| A decisive question asked before the research | A confident answer you cannot evaluate | The candidates were named without knowing what each guarantees | Drop a rung; answer the mechanism question first, then return. |
| Only researchable questions, ever | Deep knowledge, no design; decisions made by default | Each answer generated another mechanism question and never a choice | Write the decision the feature is waiting on and ask that. |
| Detail mistaken for decisiveness | Exact JSON known; still no idea where "paid" is decided | Specificity about data, not about a choice | Ask what the detail is for; the choice is one level up. |
How to do it
Most important first.
- Write the question and label its rung honestly. "How does X work?" is vague regardless of how much context follows it.
- Ask "what would I do with the answer?" and write the next question that follows. That is usually one rung up.
- For a decisive question, name the candidates in the question itself — "database, provider, or both?" — so the answer must choose.
- Before asking a person, check the rung. Researchable questions go to documentation first; the person gets the decisive one, with what you found.
- Paste the question into the assessor at /thinking/unknowns and read why it scored as it did; the heuristics are the ones above, made mechanical.
Worked on a concrete problem
The move has to produce something. This is what it produced.
- The payments ladder. Vague: "How do payments work?" Researchable: "How does a backend initiate and confirm a payment with a provider?" Decisive: "Which system should be authoritative for whether an order has been paid?" The first got an overview. The second was answered by two documentation pages and the experiment in Unknown, Question, Experiment. The third was answered in one sentence by the senior engineer — "your database, updated only from the verified webhook" — and that sentence became the design of the checkout handler.
- The inventory ladder. Vague: "How should I handle inventory?" Researchable: "How do databases prevent two transactions from both decrementing the last unit?" Decisive: "Should stock be reserved when the order is created or when payment succeeds, given that a reservation at order time can be abandoned?" The decisive form exposed that the answer depends on a product decision — how long an unpaid order holds stock — which was then asked of the founder rather than researched.
- The AI assistant, asked the decisive question with the candidates named, produced a short answer with the trade-off between the two options — and one claim about the provider that the experiment had already shown to be wrong. The question's shape was what made the error visible (AI as Reviewer).
How you know it worked
What now exists that did not before, and what question you can now ask.
- You can say which rung your question is on and what an answer at that rung would let you do next.
- People answer in sentences instead of overviews.
- Answers can be wrong — you can name what a wrong answer to your question would look like.
- The decisive question has produced a decision that is now in the code, and you can point at the line.
The questions you can now ask
The field this whole domain exists for. After this lesson, these are the questions to put to an unfamiliar problem.
- ?Which rung is this question on — topic, mechanism, or decision?
- ?What would I do with the answer, and what question does that make me ask next?
- ?What are the candidates my decisive question has to choose between?
- ?Who or what is the right thing to ask at this rung?
What can go wrong
- Climbing too early. A decisive question asked before the researchable one is answered — "should I use the webhook or the redirect?" — when you do not yet know that both exist or what each guarantees — gets an answer you cannot evaluate. Rungs are climbed in order.
- Mistaking specificity for quality. "What is the exact JSON of the payment-succeeded event?" is extremely specific and still researchable — it has no decision in it. The decisive rung is about a choice, not about detail.
- Never leaving the researchable rung. Some engineers read very well and decide nothing; every answer produces another researchable question, and the design is made by default.
- Treating the ladder as a rule for talking to people. Sometimes the vague question is the right one to ask a colleague — "I have no idea about payments, where would you start?" — when the aim is orientation rather than an answer. The ladder is for questions that need answers.
- A decisive question commits you to having done the research. That is work the vague question let you skip — at the cost of never getting a usable answer.
- Naming the candidates in the question can exclude the right answer. "Database or provider?" hides "both, reconciled nightly". Leave room for "neither" when you ask.
- The senior engineer's one-sentence answer is only as good as the question that shaped it; a decisive question with a wrong premise gets a confident wrong decision.
- "A good question is a short one." The decisive form is often the longest of the three; it is good because it contains a choice, not because of its length.
- "Vague questions are wrong." They are the first line on the board, and the right thing to ask a person for orientation. They are wrong only as the final form.
- "AI answers vague questions fine." It answers them fluently, which is worse: the overview reads as an answer and contains no decision and possibly an error you cannot see (Good Tool Use).
Where this applies
Problem-solving advice is stated as universal far more often than it is. These labels say what each method is specific to — and where CONTESTED appears, the note gives the strongest form of the opposing view.
- GENERALTopic, mechanism, decision: the three rungs apply to any question in any domain, including questions asked of yourself in a notebook.
- TEAM-SPECIFICA solo learner climbs all three rungs alone; on a team the researchable rung is usually answered by documentation and the decisive rung by the person who owns the area, and the skill becomes knowing which rung to bring to whom.
- ILLUSTRATIVEThe fifteen-minute overview, the senior engineer's one-sentence answer and the AI's claim about the provider are invented to show the shape of the ladder.
Where the depth lives
This domain asks the question and hands the answer off by name.
- — The manifesto's /manifesto/review page — reviewing an LLM's answer — starts from a question of known rung; a decisive question is the one whose answer can be reviewed.