Problem Before Technology
A technology is an answer. Before accepting one, find the question: what requirement it serves, what simpler thing meets it, what it costs to run, and what would have to be true for it to be the right call. The Why Ladder, applied to "we need Redis".
The situation, the reflex, and why it stalls
Every lesson starts where being stuck starts: someone has a problem, and the first move that comes to mind feels like progress.
A technology has been proposed — by you, a colleague, or the last article you read. How do you find the problem it is supposed to solve, decide whether that problem exists, and choose between it and the simpler thing?
A colleague says the store needs Redis for the cart, and they are more experienced than you. You cannot say they are wrong, you cannot say why they are right, and "because caching is fast" is the whole of the argument you have heard.
Defer. Someone more senior proposed it; Redis is popular; every store you have read about has a cache somewhere. Agreeing costs nothing today and the alternative is an argument you are not equipped to win.
The component enters the design with no requirement attached, so nothing can ever justify removing it. It will still be there in a year, with a runbook, an on-call page and a failure mode, serving a cart that gets a handful of reads a minute.
- The component enters the design with no requirement attached, so nothing can ever justify removing it. It will still be there in a year, with a runbook, an on-call page and a failure mode, serving a cart that gets a handful of reads a minute.
- The requirement that would have justified it — or not — is never written down. "The cart must be fast" is not a requirement; "the cart page must render within a budget under this load" is, and only the second one can be checked against the simpler option.
- Every later decision about the cart is made inside the constraint that it lives in Redis. When the cart needs to survive a restart, the answer becomes "Redis persistence configuration" rather than "why is durable state in an ephemeral store?"
- You learn nothing. Deferring on this one means deferring on Kafka next week, because the skill being avoided — turning a technology back into a requirement — is the same skill both times.
The move
Precisely enough to apply it to a problem you have never seen — not a slogan.
- Treat a technology as a compressed claim about a requirement, and decompress it by asking why until a requirement appears. "We need Redis" → why → "the cart must be fast" → why → "reads of the cart happen on every page" → why is that slow → "it is not, yet; nobody has measured". The ladder ends when it reaches something you could test, or something nobody can defend.
- At the bottom of the ladder, name the real requirement in checkable terms, then name the simplest thing that meets it. Usually the simplest thing is what you already have, plus at most an index or a query change. Only if the simple thing fails the requirement does the component have a case.
- Also name the case where the proposal was right — the load, the data shape, the constraint under which the simpler thing genuinely fails. The device is not "big tools are always wrong"; it is "know what would make this one right, so you recognise it when it arrives".
- Turn the disagreement into an experiment where you can. "Would a cart table be too slow?" is answerable in an afternoon with a table, an index and a load script, and the answer settles the argument in a way that seniority cannot (The Smallest Experiment With a New Technology).
The Why Ladder, on the cart
The ladder is the device this lesson exists for. Read the rungs as a conversation with the proposer, not an interrogation: each answer is a real thing they believe, and the ladder ends where a belief turns into a requirement or into "nobody knows". Both endings are useful.
“We need Redis for the cart.”
- ↓Why Redis? Cart reads happen on every page, so the cart has to be fast.
- ↓Why would a database table be too slow? Under heavy read load a table would be a bottleneck.
- ↓What load do we have, and has a table been measured under it? There is no traffic yet, and nothing has been measured.
- ↓What else does the cart have to do? Survive a restart, and survive the customer coming back tomorrow.
the claim was right when Cart reads are measured as a bottleneck the database cannot serve after indexing — or the requirement changes so that carts really are ephemeral, in which case an in-memory store is the honest model of the data.
Which question to ask of a proposal
Not every proposal needs the full ladder. The decision below sorts proposals by what kind of requirement they compress, and says which question settles each kind. The costs column is the price of asking, which is why the cheap-to-reverse row exists.
What is the requirement behind the proposal, and how should it be settled?
when The argument is about latency or throughput under load.
cost Measure the simple thing under the load you actually have; an afternoon with a load script. Asserting either way is worthless.
when The argument is about an invariant.
cost Write the invariant and check which option upholds it — a table in a transaction usually beats an ephemeral store here, and the ladder often reverses the proposal.
when The simple thing may genuinely lack a feature.
cost Try the simple thing against the requirement in a spike; if it fails, the component enters with its reason attached (Spikes).
when The team operates the component elsewhere.
cost Count it as a real requirement and weigh it honestly — then still ask what the component fails at, because familiarity does not change its failure modes.
when The component sits behind an interface and can be removed in a day.
cost Decide fast, record the revisit condition, move on. The ladder costs more than the mistake (Reversible vs Irreversible Decisions).
Two options, scored honestly
A matrix without a caveat is fake precision. The one below scores the cart's two options on the axes that actually move, and its caveat says what the numbers cannot: that the whole comparison is conditional on load, and that the scores would swap on a store with a measured read bottleneck.
| Option | Simplicity | Reliability | Cost | Maintainability | Performance | Note |
|---|---|---|---|---|---|---|
| Cart table in the existing database | One more table, durable in the same transaction as everything else. Fast enough until measured otherwise. | |||||
| Redis, with persistence configured | A second system to run, secure and back up; durability is a configuration choice, not a default; a cache-invalidation surface appears. |
caveat These scores hold at low, unmeasured load. With a measured read bottleneck on carts the performance column decides, and the reliability score for the table falls as it saturates. The numbers say which option to try first, not which is right forever.
How to do it
Most important first.
- Write the proposal as a claim — "we need X" — and climb the ladder in writing, one why per line, until you reach a requirement someone could check or an answer nobody can give.
- Under the ladder write three things: the real requirement, the simpler thing that meets it, and the condition under which the proposal would be right after all (The Why Ladder).
- Ask the proposer for the requirement, not for a defence of the technology. "What would break if we used a table?" is a question they can answer and it is not a challenge to their competence.
- If the real requirement is a performance one, measure the simple thing before replacing it (Measure Before You Optimize). If it is a durability or consistency one, write the invariant and ask which option upholds it.
- Record the decision with its evidence and its revisit condition, so the component can be added later without an argument and removed later without one too (The Decision Journal).
Worked on a concrete problem
The move has to produce something. This is what it produced.
- "We need Redis for the cart." Why? "Cart reads are on every page and must be fast." Why would a table be slow? "Lots of reads." How many? "We have no traffic yet." Real requirement: the cart page must render quickly at the load we actually have, and the cart must survive a restart. Simpler thing: a cart table with an index on session id — it is durable, which Redis is not by default, and at this load it is fast. Justified when: measured read latency on carts becomes a bottleneck the database cannot serve, or the cart genuinely becomes ephemeral by requirement.
- "We need Kafka for order events." Why? "So that email, analytics and the warehouse get told about orders." How many consumers exist today? "Email." Does checkout have to wait for it? "No." Real requirement: after an order is paid, send one email, and do not make the customer wait for it. Simpler thing: a job row written in the same transaction as the order, and a worker that sends the email (Background Jobs and Workers). Justified when: there are several independent consumers, each needing its own replayable history, at a volume where a job table becomes the bottleneck.
- The one where the ladder said yes. "We need a search engine for product search." Why? "Name search must tolerate typos and rank by relevance." Would a database text search do? Tried it: prefix matching worked, typo tolerance and ranking did not, and the requirement was not negotiable for this catalogue. Real requirement: fuzzy, ranked search over product names. Simpler thing: tried, and it failed the requirement. The component enters the design with its reason attached.
How you know it worked
What now exists that did not before, and what question you can now ask.
- The proposal has become a requirement sentence that could be checked, and the technology has become one of at least two options for meeting it.
- You can say what would have to be true for the proposal to be right, and whether it is true now.
- The disagreement has an experiment attached, or has been settled by one.
- A decision record exists that a future engineer could read to add the component later — or to understand why it was added now.
The questions you can now ask
The field this whole domain exists for. After this lesson, these are the questions to put to an unfamiliar problem.
- ?What requirement is this technology a compressed answer to, and can I write that requirement so it could be checked?
- ?What is the simplest thing that meets the requirement, and has it been tried or measured?
- ?What would have to be true for this proposal to be the right call — and is it true now?
- ?How reversible is this choice, and does the cost of deciding carefully exceed the cost of deciding wrongly?
What can go wrong
- The ladder is used as a weapon: every proposal from a colleague is interrogated until it dies, and the lesson learned is that proposing things is dangerous. The device must climb to "justified when" as seriously as it climbs to "simpler".
- The ladder stops one rung early, at a slogan. "The cart must be fast" feels like a requirement and is not; a ladder that stops there proves nothing either way.
- The simpler thing is named and never tested. "A table would be fine" is as much an assertion as "we need Redis"; when the requirement is performance, the simple thing has to be measured.
- The move is applied to a decision that was already made and is cheap to reverse. A cache behind an interface can be removed in a day; spending a week deciding whether to add it costs more than adding it wrongly would.
- Climbing the ladder on a senior colleague's proposal costs social capital, and the return is uncertain: sometimes they were right and knew it, and the ladder just confirms that slowly.
- The simpler thing is often a table and a query — and it may need replacing later, which is a migration the component would have avoided. The bet is that the migration is cheaper than running the component for the months it was not needed.
- A disagreement turned into an experiment costs an afternoon; deferring costs nothing today and something unknown later.
- "The simpler thing always wins." The search-engine ladder ended in yes. The device finds the requirement; the requirement decides.
- "Senior engineers propose technologies for no reason." Usually they compressed a real requirement from a previous system, and the ladder's job is to check whether this system has the same requirement. Often it does not, and the engineer is the first to say so once the question is asked properly.
- "This is about performance components only." The same ladder applies to a framework, a microservice split, an ORM, a workflow engine. Anything proposed as a solution can be decompressed into the problem it assumes.
Where this applies
Problem-solving advice is stated as universal far more often than it is. These labels say what each method is specific to — and where CONTESTED appears, the note gives the strongest form of the opposing view.
- GENERALAny technology proposal — cache, queue, framework, service split, search engine — can be decompressed into the requirement it assumes; the ladder is the same, and what changes is how far down the requirement lives.
- SCALE-SPECIFICOn a store with no traffic the simpler thing wins nearly every ladder; as measured load grows, the same ladders start ending in "justified now", and a team that recorded the revisit conditions knows exactly when. The device does not change with scale; its answers do.
- CONTESTEDA serious opposing view holds that the ladder underweights convention: a team that already runs Redis and Kafka for other systems pays almost nothing to add one more consumer, and the "simpler thing" — a bespoke job table — is simpler only on paper, since it is one more custom mechanism nobody else on the team knows. On that view, organisational familiarity is a requirement in its own right and the ladder should count it as one.
- ILLUSTRATIVEThe colleague, the cart, the search catalogue and the afternoon load test are invented to show the shape of the ladder; no real measurement is reported.
Where the depth lives
This domain asks the question and hands the answer off by name.
- — The Why Ladder lab at /thinking/why runs the Kafka, Redis, microservices and Kubernetes ladders interactively; the decision tool at /thinking/decide asks which problem you are solving before letting you pick.