The Trade-off Matrix
Six axes that real design decisions move, scored side by side — useful because it forces every option to be described on the axes it is worst at, and dangerous because the digits look like evidence.
The requirement, the obvious build, and why it breaks
Every lesson starts where the work starts: someone asked for something, and the first implementation that comes to mind survives until the requirement changes.
Three designs, three advocates, and the argument keeps moving. How do I make the comparison concrete without pretending I measured something?
Scheduled reports need somewhere to keep job state: which schedules exist, which run is in flight, what failed and what to retry. Three proposals are on the table and the meeting is going in circles.
Argue it out. Everyone is experienced, the strongest argument will win, and writing a table is process for its own sake.
Unstructured argument selects for whoever cares most, not for the option with the best profile. The person who has run Temporal before will win a discussion about Temporal.
- Unstructured argument selects for whoever cares most, not for the option with the best profile. The person who has run Temporal before will win a discussion about Temporal.
- Each advocate describes their option on the axes it is good at. Without a fixed set of axes, nobody has to say what their option is worst at, and that is precisely the information the decision needs.
- The argument also drifts: it starts about operational burden, moves to performance, and lands on familiarity, with no record that the first two were ever answered.
- And the outcome is unrecorded, so in a year nobody can distinguish "we considered Redis and rejected it for at-most-once semantics" from "nobody thought of Redis" (Decision Records).
What limits the solution, and what must never stop being true
This domain leads with these two. A design that ignores its constraints is not a design, and an invariant nobody named is one nothing is protecting.
- Three engineers own the service part-time; whatever is chosen is operated by whoever is on call for everything else.
- Postgres is already run by the team and already backed up. Redis exists for caching. Nothing else is in the stack today.
- Volume is roughly two thousand scheduled runs a day, and there is no reason to expect an order of magnitude more this year.
- A scheduled run executes at most once, even if a worker dies mid-run and another picks it up.
- No schedule silently stops running: a failure is either retried or visible.
Who owns what, and where the seams fall
Responsibilities decide boundaries; boundaries decide what an interface has to say.
- The person proposing the comparison owns choosing the axes *before* the options are scored, because axes chosen afterwards are chosen to produce a winner.
- Each advocate owns writing the note for their own option's worst axis. The notes are the artefact; the digits are scaffolding.
- Whoever decides owns the caveat: what this comparison could not express, and what would change the answer (Revisit Triggers).
- The matrix compares designs, not implementations. Once the discussion is about a specific library's API it has left the level where a matrix helps.
- Six axes is the practical limit. Beyond that, the exercise becomes a spreadsheet and people start optimising the total rather than reading the rows.
The six axes, on a real decision
The point of a fixed axis list is that every option has to be described on the dimension it is worst at. Left to prose, each advocate writes about what their option is good at and the comparison never happens.
Read the notes and ignore the digits. The digits exist to make the shape visible at a glance; every piece of information that could change the decision is in the text.
| Option | Simplicity | Flexibility | Performance | Testability | Operational | Migration cost | Note |
|---|---|---|---|---|---|---|---|
| Postgres table plus a polling worker | One store the team already backs up, already monitors and already knows how to restore. At-most-once comes from a transactional claim, which is a well-understood pattern. Polling wastes queries at low volume and stops being sensible somewhere above a few thousand runs a minute — nowhere near this workload. | ||||||
| Redis with a sorted set and a worker pool | Fast and idiomatic for scheduling, and the team already runs Redis — but for caching, where losing the data is fine. Using it as the system of record makes persistence configuration a correctness concern, and at-most-once now depends on getting a lock and its expiry right, which is the classic place this goes wrong. | ||||||
| A managed workflow service | Retries, backoff, history and visibility arrive for free, and complex multi-step schedules become easy. It is a new vendor in the critical path, local testing needs an emulator or a live account, and the workflow definitions are the least portable artefact you can create (What a Framework Charges). |
caveat These are judgements about sketches, on one team's constraints, and nothing here was measured. Higher is better on every axis, so the migration column says the third option is expensive both to adopt and to leave. What the numbers cannot express: that the first option's operational score depends entirely on the team already running Postgres well — score it for a team without that and it drops to a 2; that the third option's flexibility is only valuable if multi-step workflows actually arrive, which is a product question nobody in the meeting can answer; and that a 3 on simplicity for Redis hides a specific, sharp risk — the failure mode is a duplicated financial report, not a slow one. Do not total the rows.
Running it in twenty minutes
The exercise is worth doing quickly and badly rather than slowly and well. Its value comes from the order of the steps — axes before options, notes before numbers — and almost none of it comes from precision.
The step people skip is the last one. A matrix with no caveat and no trigger is an artefact that will be quoted in two years by someone who was not in the room and cannot tell what it did not consider.
- 1State the constraints
Team size, existing stack, on-call reality, volume, deadline. Three lines.
fails by Being left implicit, after which two people score the same option differently and cannot work out why (Constraints Are Part of the Design).
- 2Fix the axes
Pick from the six before looking at the options; drop any that genuinely does not apply and say so.
fails by Choosing axes after the options are on the table, which selects the axes on which the preferred option wins.
- 3List real options
Two to four, each of which the team could actually operate.
fails by Including a straw option to flatter the favourite. Everyone notices, and the exercise loses its credibility (Tone, Disagreement and Receiving Review).
- 4Write the notes
One or two sentences per option, and each must name the axis it is worst at.
fails by Writing notes that only describe strengths, which reproduces the unstructured argument inside a table.
- 5Score last
Assign the digits after the notes exist, in one direction, without totalling.
fails by Scoring first and then writing notes to justify the scores.
- 6Write the caveat and the trigger
What the comparison could not see, and what evidence would change the answer.
fails by Being skipped because the decision is already made — which is exactly when the artefact starts misleading its future readers (Revisit Triggers).
If the decision is cheap to reverse, stop after step three and just pick one. The full exercise is for decisions that will be expensive to undo (Reversible and Irreversible Decisions).
The ways a matrix lies
Every one of these has happened in real teams, and none of them is caused by dishonesty. They are caused by the format: a table looks like a result, and people treat results differently from opinions.
The mitigation is not more rigour in the scoring. It is keeping the notes attached, refusing totals, and being explicit that the exercise ranks nothing.
| Trigger | Symptom | Cause | Response |
|---|---|---|---|
| The table is pasted into a slide without the notes | A rough judgement is quoted a year later as though the team had measured something | The digits are portable and the reasoning is not | Put the caveat in the same block as the numbers, and keep the whole thing next to the code rather than in a deck (Docs Close to Code). |
| Someone totals the columns | The option with the most 4s wins, including on axes nobody cared about | Unequal weights treated as equal by the arithmetic | Name the one or two axes that actually decide before scoring, and record that they were the deciders. |
| Constraints change — volume grows tenfold | The old comparison is cited to block a migration | The scores were always relative to constraints that were not restated | Every matrix carries its constraints and a revisit trigger, so a constraint change reopens it automatically (Revisit Triggers). |
| An option nobody understands is scored anyway | A 2 on testability that reflects unfamiliarity, not the design | Scoring what the team knows rather than what is true | Spike it for an afternoon or leave it off the table. An unscored option is more honest than a guessed one (Simple Is Not Easy). |
How to build it
Most important first.
- Fix the axes first: simplicity, flexibility, performance, testability, operational complexity and migration cost. Six is enough for almost every codebase-level decision, and fixing them in advance is what stops the comparison being shaped around a preferred answer.
- Use one direction consistently — higher is better on every axis, so a 2 on operational complexity means "this will cost you operationally" and a 2 on migration means "getting here, or leaving, is expensive". Mixed directions produce arguments about the table rather than the decision.
- Write the note before the number. A row whose note says "fine" on every axis has not been thought about, and the scoring will inherit that.
- Never sum the columns. Adding an operational score to a testability score produces a number with no meaning, and the total will then be defended as if it were a result (What Makes Software Hard to Change).
- Attach the caveat and the revisit trigger to the matrix itself, so that whoever finds it in eighteen months knows both what it could not see and what would make it wrong (Decision Records).
What the next change costs
The field this whole domain exists for. A structure is only better if it makes the change after this one cheaper — and it is worth saying which changes it does not help.
- The matrix itself is cheap: twenty minutes, and it is thrown away or attached to a record.
- What it buys is a cheaper *next* decision in the same area: the axes and constraints are already written, so revisiting when volume grows is an edit rather than a re-argument.
- What it costs is a bias towards options that score legibly. An unusual design that is excellent on an axis nobody listed will lose to a mediocre one that fits the table, and that failure is invisible from inside the exercise.
- Scoring makes an unmeasured judgement look measured, and that misreading survives longer than the meeting does. The caveat is the only mitigation and it is the first thing dropped when the table is copied.
- The exercise favours options that are easy to describe, and penalises the unfamiliar one that would have to be explained before it could be scored.
- It costs meeting time for decisions that did not need it. Applied to every choice, it becomes the process overhead its critics accuse it of being (Reversible and Irreversible Decisions).
What can go wrong
- The digits get quoted later without the notes, and a rough judgement becomes "we scored it".
- Axes are added mid-exercise once someone sees their option losing, which is unfalsifiable by construction.
- A dominated option is included to make the preferred one look moderate, and everyone can see it, which discredits the whole exercise.
- The matrix is produced, the decision goes the other way for a reason nobody writes down, and the artefact now actively misleads (Documentation Decay).
- The comparison depends on the constraints being stated honestly — team size, existing stack, on-call reality. Change the constraints and every row moves (Constraints Are Part of the Design).
- It depends on the options being genuinely available. Scoring an option the team cannot operate is theatre that makes the chosen option look better by comparison.
- Downstream, the decision creates dependencies: whatever is chosen becomes a thing every later feature in this area assumes (Volatile Dependencies).
- "The highest total wins." The axes have no common unit and the weights are not equal, so a total is a category error. The matrix ranks nothing; it displays (The Complexity Budget).
- "A matrix makes the decision objective." It makes it explicit. The subjectivity moves into the axis choice and the notes, which is where it can at least be argued with.
- "Every decision deserves one." Most do not. Sort by cost to undo first, and reserve this for the ones that are expensive to reverse (Reversible and Irreversible Decisions).
- "We can reuse last year's matrix." Only if the constraints are unchanged. The scores were always relative to a team size, a volume and a stack (Revisit Triggers).
Testing it, and how it ages
- Test the invariant, not the choice: whichever store is picked, a test that two workers racing for the same scheduled run produce exactly one execution (Idempotency by Design).
- For an option scored well on testability, write one test against a sketch of it before committing. Testability scores are the ones most often assigned optimistically (Testing as Design Feedback).
- A matrix expires when its constraints do. Ten times the volume, or a second team, and the operational column is the first to invert.
- Kept next to the code, it becomes the thing that makes the eventual migration a decision rather than a surprise (Docs Close to Code).
- Teams that run this a few times stop needing the table for small decisions, which is the intended end state — the axes become a habit of asking "and what is it worst at?" (The Design Loop).
Where this applies
This domain's advice is contested more than most. These labels say what each claim is specific to — and where CONTESTED appears, the note gives the strongest form of the opposing view rather than a caricature.
- GENERALThe six axes describe pressures that exist in any codebase — though "operational" means almost nothing for a library with no runtime of its own, and for that case the axis should be replaced by adoption cost rather than left in and scored uniformly.
- SIMULATEDEvery score in this domain, including the ones below, is a judgement produced by reasoning about a sketch rather than by measuring a system. They are useful for comparing shapes and worthless as absolute values, and Engineer Atlas labels them rather than presenting them as findings.
- CONTESTEDThe strongest objection is that scoring is worse than prose: it launders opinion into apparent data, invites totalling, and the notes — which carry all the information — get dropped the moment the table is pasted into a document. Experienced engineers who have watched a matrix be used to justify a predetermined answer are right that this happens often. The defence is narrow: the value is in being forced to score your own option on its worst axis, and a comparison written as prose almost never does that.
Where the depth lives
This domain teaches the codebase-level structure and hands the rest off.
- — System Design — the same six axes reappear when choosing between queues, streams and schedulers, with availability and partition behaviour added as axes that a single codebase does not need.