Experiments & Prototypes

A prototype answers a question. Spikes, experiment design, predicting before running, and the feedback loop that beats three weeks of design.

A Prototype Answers a Question

"Can the browser upload a five-gigabyte file straight to object storage without going through our server?" is a question. A prototype that does only that — and nothing around it — answers it in an afternoon. A prototype built to "see how the upload feature would work" answers nothing and takes a week.

Q · You are about to build a prototype. What question is it answering, and what is the smallest thing that answers it?
Prototype vs Production

A prototype is allowed to ignore polish, scaling, error handling and extensibility — that is what makes it fast. The condition is that it says, in writing, what it ignored and therefore what it does not prove. A prototype that does not say so becomes production by accident, and the omissions become bugs nobody chose.

Q · What is a prototype allowed to skip, what is it never allowed to skip, and how do you stop it quietly becoming the product?
Spikes

A spike is a time-boxed investigation whose only output is reduced uncertainty: a question, a box of hours, an answer, and then a decision to discard or integrate. The time-box is what distinguishes it from research that never ends, and the written answer is what distinguishes it from a branch someone will find later and wonder about.

Q · There is something you need to understand before you can design the next piece. How do you investigate it without the investigation becoming the project?
Experiment Design

Question, hypothesis, test, observation, conclusion — the same structure whether the experiment is a load test, a spike or a debugging probe. The hypothesis is written before the test so the result can disagree with you; the observation is recorded before the conclusion so the conclusion cannot rewrite it.

Q · You are about to try something to find out whether it is true. How do you set it up so that the result can actually change your mind?
Prediction Before Execution
▶ lab

Before you run the code, say what you expect to see. If the output matches, you understood the system; if it does not, you have just found the exact place your model of it is wrong — and that is information you can only get by having committed to a prediction first. It is the cheapest experiment in engineering and the most skipped.

Q · You are about to run something — a test, a query, a request, a debugging change. What do you expect to happen, and what will you learn if it does not?
Short Feedback Loops

Change, run, observe, fix — minutes per cycle — beats design for three weeks, build everything, and discover on the last day that one assumption was wrong. The reason is not that thinking is bad; it is that the cost of a wrong assumption grows with everything built on top of it, and a short loop finds it before anything is.

Q · How long is it between making a change and knowing whether it was right — and what does that number cost you?