The Next Question
The domain's promise in one move: from "I have no idea" to "I know the next question I need to answer". Progress on an unfamiliar problem is measured in questions sharpened, not features shipped.
The situation, the reflex, and why it stalls
Every lesson starts where being stuck starts: someone has a problem, and the first move that comes to mind feels like progress.
You do not know how to solve this and cannot see a path to knowing. What is the one thing you can always produce from that state, and how do you make sure it is the right thing?
The task is an AI assistant that answers questions about company documentation. You have never built one. You know the words — retrieval, embeddings, prompts — and not what any of them means for this problem. Every article you open assumes you already know what you are building.
Learn the field first. Read about embeddings, vector databases, chunking strategies, evaluation, until you understand enough to design the thing. It is the responsible-sounding move: do not build what you do not understand.
The field is bigger than the problem. A week of reading later you know six ways to chunk a document and still cannot say what a good answer to an employee's question looks like — the one thing the reading could not tell you, because it is about your documentation, not about the field.
- The field is bigger than the problem. A week of reading later you know six ways to chunk a document and still cannot say what a good answer to an employee's question looks like — the one thing the reading could not tell you, because it is about your documentation, not about the field.
- Reading produces no next action. Every article ends with more articles, and there is no natural point at which you have "understood enough", so the reading continues until an external deadline ends it.
- The questions you could have answered on day one are still unanswered: how many documents, what format, who asks, what they ask, what happens today when they cannot find the answer. None of those needs a paper.
- When you finally start, you start from the article's architecture, not from your problem's. The pipeline has a reranker because the article had one; nobody has checked whether the questions people actually ask need it.
The move
Precisely enough to apply it to a problem you have never seen — not a slogan.
- Accept that on an unfamiliar problem you cannot produce a solution, and notice that you can always produce a question. The move is to make the question as good as it can be — specific, answerable by an experiment or a person, and about *your* problem — and to treat that as the day's output. "I know the next question" is a real state, and it is the state the loop moves you through.
- Find the next question by asking which unknown, if answered, would change what you build. Not the most interesting unknown and not the most fundamental one — the one on the path. For the assistant, "how do embeddings work?" is interesting; "what do employees actually ask, and what does a good answer look like?" changes the design.
- Sharpen the question until it has a verb and an experiment. "What do people ask?" becomes "if I collect the last fifty questions sent to the support channel, what fraction are answerable from one document, and which documents?" — and that is an afternoon's work with no vector database in sight (Question Quality).
- Answer it, then ask what the answer makes askable. Each answered question moves the frontier of what you know about this problem and exposes the next one. That chain — question, experiment, answer, next question — is what progress on an unfamiliar problem looks like from the inside.
From "I have no idea" to the next question
The ladder below is the whole move in one example. The vague form is honest and useless; the better form is researchable and off the path; the best form is researchable, on the path, and answerable this afternoon by reading a channel history. The why column says what only the best form lets you decide.
why The best form decides whether retrieval is needed at all (a handful of documents may fit in a prompt), what "good" means for the evaluation, and which documents to start with. The better form assumes an architecture before any of that is known, and the vague form cannot be started on.
The board, with the path marked
An unknowns board for the assistant after the first afternoon. The point of this one is the ordering: the unknowns are listed in the order they would change the design, and the first is the only one being worked on. The others are written so they are not lost, and left alone so they do not become the reading stall in a new form.
- ✓Users: employees, asking in a support channel today; a human answers within hours.
- ✓Documents: a few dozen policy and process pages, in a wiki, mostly text.
- ✓Goal: fewer questions waiting on a human; a wrong answer is worse than no answer.
- ~The wiki is the source of truth and is reasonably current. To be checked — stale pages would make confident wrong answers.
? What people ask.
becomes Of last month's questions, what fraction are answerable from one document, and which ones?
experiment Read the channel history; tag each question with the document that answers it, or "none", or "depends on the case".
? What a good answer is.
becomes Does a good answer need to cite its document, and what should it say when two documents disagree or none applies?
experiment Take five answered questions from the channel and write the answer the assistant should have given; compare with the human's.
? Retrieval.
becomes Given the number and size of the relevant documents, is retrieval needed, or does the relevant set fit in a prompt?
experiment Measure the total size of the documents tagged in the first experiment; try the largest realistic question with all of them in context.
? Embeddings and vector databases.
becomes Deferred: only becomes a question if the retrieval experiment says retrieval is needed.
experiment None yet — its experiment depends on the previous answer, which is why it is last.
The last unknown is the one the reflex would have started with. It is on the board, sharpened, and deliberately not being worked on.
A chain of questions, as an order of work
The chain, written as an order. Each step is a question and the reason it comes before the next. The alternative order is a real one — an evaluation-first approach that some teams prefer — and the device names it because the sequence below is one path, not the path.
- 1What do people ask, and where do the answers live?
because Decides whether the problem is retrieval, synthesis, or neither; nothing else can be chosen before this (Problem Formulation).
- 2What does a good answer look like, and what should the assistant do when unsure?
because Becomes the evaluation and the failure requirement — "say you do not know" — before any pipeline exists.
- 3Does the relevant set of documents fit in a prompt?
because If yes, there is no retrieval problem in V1 and the design is a prompt and a guardrail.
- 4Only if not: how should documents be split and found?
because The retrieval question, asked with real documents and real questions, is now specific enough to run experiments on.
- 5What does the first slice — one question, one document, one answer with a citation — teach?
because The smallest end-to-end thing exposes the unknowns the questions missed (The Smallest Executable Thing).
How to do it
Most important first.
- When stuck, write "the next question is:" and finish the sentence. If you cannot, write down the vague word that is blocking you and rewrite it until it contains a verb (Unknown to Specific Question).
- Rank candidate questions by how much the answer would change what you build, not by how much you would like to know the answer. Put the design-changing one first (Questions That Reduce Uncertainty).
- Attach an experiment or a person to the question. If neither can answer it, it is not yet a question; split it.
- Time-box the answer. A question worth asking is worth answering in an afternoon or a day; one that needs a week is several questions.
- Write the answer and the question it opened. The notebook page for an unfamiliar problem is a chain of questions, and re-reading it is how you see that you moved (The Engineering Notebook).
Worked on a concrete problem
The move has to produce something. This is what it produced.
- The assistant, from "I have no idea". Vague: "I don't know how to build a documentation assistant." Next question: "what do employees ask, and where do the answers live today?" Experiment: read the last month of the support channel and, for each question, find the document that answers it. Answer: most questions were about a handful of documents, and the hard ones were about which policy applied to a specific case. Next question, opened by the answer: "does a good answer need to cite the document, and what happens when two documents disagree?"
- The same problem, with the wrong next question. "Which vector database should we use?" It is specific, it has an experiment, and its answer changes nothing until the questions people ask are known, because a small number of documents may not need retrieval at all. Specific is necessary; on-the-path is the other half.
- The store, on inventory, one question at a time. "I don't understand inventory" → "when two customers buy the last unit at once, what should happen?" → answered by asking the founder (reject the second, say why) → "where is that decided so it cannot be wrong under concurrency?" → answered by an experiment on one row → "what does the losing customer see?" Each question was about a day's work and each one changed the design.
How you know it worked
What now exists that did not before, and what question you can now ask.
- You can finish the sentence "the next question is:" with something that has a verb, an experiment or a person, and a reason it is on the path.
- The notebook shows a chain: a question, what answered it, and the question it opened.
- Reading has become targeted — you open an article to answer a question you already have, and close it when the question is answered.
- You can say what the current answer changed about the design.
The questions you can now ask
The field this whole domain exists for. After this lesson, these are the questions to put to an unfamiliar problem.
- ?What is the next question — and does it have a verb, an experiment or a person, and a reason it is on the path?
- ?Which unknown, if answered, would change what I build — and am I working on that one?
- ?What did the last answer make askable that I could not ask before?
- ?Am I reading to answer a question I have, or reading in the hope a question appears?
What can go wrong
- The next question is chosen for interest rather than for leverage, and the chain wanders through the field instead of through the problem. "How do embeddings work?" is a fine question for an evening and a poor one for the project.
- Questions are generated and not answered. A list of twelve excellent next questions is a research plan; if none has been run by the end of the week, the list is the new form of the reading stall.
- The question is sharpened past the point of usefulness. "What fraction of questions are answerable from one document?" needs a rough answer, not a rigorous study; rigour here is a way of not moving on.
- The move is applied when a solution is already known. Someone who has built three documentation assistants has a design, and their next question is about this documentation, not about the shape of the system.
- A chain of questions is slower to show than a chain of features, and to anyone measuring output it looks like the project has not started. The questions are the project; the features come faster afterwards, but "afterwards" is a promise.
- Choosing the design-changing question over the interesting one means not learning the field in the order the field is taught. You will know your problem well and the field patchily, and that trade is deliberate.
- Time-boxing answers means accepting rough ones. Sometimes the rough answer was wrong and a later question has to undo it.
- "So do not read." Read to answer a question. The stall is reading in the hope that a question will appear; reading that starts from a question is the fastest way to answer it (A Reading Strategy for an Unfamiliar Library).
- "The next question is always about requirements." Early on it usually is, because that is where the leverage is; later the next question is about a failure mode, a measurement, or a bug. The move is the same and the content moves with the loop.
- "A good question needs a rigorous answer." It needs an answer good enough to choose the next question. A rough count of last month's support questions is enough to decide whether retrieval is needed; a study is not required.
Where this applies
Problem-solving advice is stated as universal far more often than it is. These labels say what each method is specific to — and where CONTESTED appears, the note gives the strongest form of the opposing view.
- GENERALAny unfamiliar problem yields a next question; what changes across domains is who can answer it — a founder, an experiment, a document, a colleague — and how long a reasonable answer takes.
- TEAM-SPECIFICA solo learner has to answer every question themselves and should bias towards experiments; an engineer on a team has colleagues who have already answered many of them, and the fastest next move is often to ask (Asking People). The move is unchanged; the answering mechanism is not.
- ILLUSTRATIVEThe documentation assistant, the support channel and the month of questions are invented to show the chain; no real system or count is described.
Where the depth lives
This domain asks the question and hands the answer off by name.
- — The Unknowns Board at /thinking/unknowns includes the question assessor — paste the next question and see whether it scores vague, researchable or decisive, and why.