Designing for Cost
Do not optimise blindly — and notice that the structural choices deciding allocations, copies, round trips and contention are all made before there is anything to profile.
The requirement, the obvious build, and why it breaks
Every lesson starts where the work starts: someone asked for something, and the first implementation that comes to mind survives until the requirement changes.
Which performance decisions are structural, made before any measurement is possible, and which genuinely should wait for a profiler?
"Show each customer a dashboard of their last 90 days of activity." Nobody mentions performance, because at design time there are eleven customers and the largest has forty rows.
Build it the simple way and optimise later when we can measure. Premature optimisation is the root of all evil, we have no data, and guessing about performance now is exactly the mistake everyone warns about.
The advice is right about *tuning* and silent about *shape*. Later measurement will faithfully report that the dashboard is slow; it will not report that the interface it is built on cannot be made fast without changing every caller.
- The advice is right about *tuning* and silent about *shape*. Later measurement will faithfully report that the dashboard is slow; it will not report that the interface it is built on cannot be made fast without changing every caller.
- The first version loads every activity row and aggregates in memory. That is invisible at forty rows, linear in customer age, and by the time it is measurable the aggregation logic has three features attached to it.
- Some of these decisions are not reversible by tuning at all. An interface that returns a list has no place to put a page cursor; adding one changes every call site (Cost-Aware Interfaces).
- The team then does what the advice suggests: they profile, find the slow query, add an index, and the shape — one round trip per panel per request — is untouched because a profiler shows you time, not structure (Observability & Performance owns that discipline, and it is right about what it covers).
What limits the solution, and what must never stop being true
This domain leads with these two. A design that ignores its constraints is not a design, and an invariant nobody named is one nothing is protecting.
- There is no production data yet, so measuring is not merely premature — it is impossible.
- The dashboard will be the most-loaded page in the product, and every future feature will add a panel to it.
- The team is small, and any structure proposed now has to be justified without a number to point at.
- The work a request performs must be bounded by something the design controls, not by how much data a customer happens to have accumulated.
- Whatever the shape of the query, the numbers shown must be internally consistent — a total and its breakdown cannot come from two different points in time (Consistency Boundaries).
Who owns what, and where the seams fall
Responsibilities decide boundaries; boundaries decide what an interface has to say.
- The design owns the *shape* of the work: how many round trips, how much data crosses each boundary, what is bounded and by what.
- Measurement owns which of the remaining costs actually matters. These are different jobs and the second cannot substitute for the first.
- Whoever writes an interface owns making its cost legible to callers, because a caller cannot see past a method name (Designing a Module Interface).
- The costs worth designing for are the ones that cross a boundary: a network hop, a disk read, a serialisation, a lock. Work inside a function is where profiling belongs; work across a boundary is where design does (What Changes at the Network Boundary).
- The seam that matters most is the one between "the caller decides how much" and "the callee decides how much" — everything in this module is a variation on that line.
What the design decides, and what a profiler can tell you afterwards
The costs below are not equally reversible, and that is the whole point of separating them. Some are local: a slow loop is fixed in the function that contains it, and profiling is exactly the right way to find it. Others are structural: they were decided when someone chose an interface, and no amount of measurement inside that interface will surface them.
Read the last column carefully. A profiler is excellent at the first two rows and close to useless at the last three, because what it reports — time spent here — is a fact about the design you have, not evidence about the design you could have had.
| Cost | Decided by | When it becomes visible | What measurement tells you |
|---|---|---|---|
| CPU inside a function | The implementation | Under load, on the hot path | Exactly where it is. This is what profilers are for, and tuning here is cheap and local. |
| Allocation and garbage | The implementation, mostly | Under sustained load, as pauses | Which sites allocate. Usually fixable without touching the design (Allocation and Copies). |
| Round trips to a store or service | The interface shape | When a customer has more rows than a fixture | That the query is slow. Not that there is one per row — the trace shows it, the profiler does not (N+1 as a Design Problem). |
| How much data crosses a boundary | Who decides the bound: caller or callee | When a table grows, which is later than launch | That serialisation is hot. The fix is upstream of where the time is spent (Cost-Aware Interfaces). |
| Copies at boundaries | Ownership and immutability choices | Under memory pressure, rarely as latency | That copying is expensive. Whether it is *necessary* is a design question (Immutability). |
| Lock contention | What state is shared, and by whom | Under concurrency, non-linearly | That threads are waiting. Which shared state should not have been shared is a design answer (Concurrency by Design). |
The order these decisions actually get made in
It is worth walking the dashboard requirement forward, because the expensive commitments are made early and casually — usually in the first hour, usually by someone who is thinking about correctness and has no reason to think about anything else.
None of these steps is a performance decision in the sense the team would recognise. Each one narrows what is possible later.
- 1Model the data
Decide what an activity is and how it relates to a customer.
fails by A model that requires a join per panel to answer any question, which sets the round-trip floor for every future feature (Choosing the Model).
- 2Write the repository method
Name the read the dashboard needs.
fails by
findActivity(customerId)— no bound, no cursor, and a signature that has nowhere to put one (Cost-Aware Interfaces). - 3Assemble the page
Fetch what each panel needs.
fails by Each panel fetches independently, so the request cost grows with the number of panels rather than with the data (Fan-in and Fan-out).
- 4Aggregate
Turn rows into the numbers shown.
fails by In application memory over everything loaded — correct, testable, and linear in a quantity nobody controls.
- 5Ship
Eleven customers, forty rows, instant.
fails by Nothing. This is the honest part: the design is indistinguishable from a good one for the first year.
- 6Grow
The largest customer accumulates four years of activity.
fails by One customer times out. The fix is not a tuning change, because the interface has no bound and eleven call sites assume a full list (Change Amplification).
The moment worth intervening on is step two, and it costs one parameter. Every later intervention costs more, and the last one costs a migration of callers under incident pressure.
Which costs to think about now
The useful rule is not "think about performance" — it is a test of reversibility, exactly as in the rest of this domain. If getting it wrong is fixed by changing one function, defer it and measure. If getting it wrong is fixed by changing every caller, decide it now.
That test happens to sort the costs almost perfectly into the two columns above, which is why it is worth applying rather than arguing about principles.
If this turns out to be wrong, what does fixing it touch?
when A loop, a string concatenation, a sort, an allocation in a hot path.
cost Wait. Measure first. This is the case the premature-optimisation warning is about, and acting early here is a pure loss (Premature Optimization, Reclaimed).
when Aggregation strategy behind a stable interface; a cache inside a repository.
cost Wait, but keep the seam. Cheap to change later precisely because the interface hides it (Encapsulation Radius).
when Whether a read is bounded; whether a method returns a list or a page; whether an id or an object is passed.
cost Decide now. The parameter costs nothing today and cannot be added cheaply once there are twenty call sites (Cost-Aware Interfaces).
when Precomputed rollups, denormalised counters, search index sync.
cost Decide deliberately and usually decide against, for now — this adds a consistency problem to buy a read cost you have not yet measured (Backend Engineering owns the failure that follows).
when Whether history is append-only, whether tenancy is a column, whether time is a dimension.
cost Decide now with real care. This is the "unlikely but catastrophic to retrofit" category, and it is decided in the first week by people who are thinking about something else (The Cost of Change).
How to build it
Most important first.
- Ask, for every read path: what bounds this? If the answer is "how much data the customer has", the design has no bound and the only question is when it becomes a problem (Invariants).
- Count round trips at design time. It is a number you can compute by reading, before anything runs, and it is the single most predictive thing about a page's latency (N+1 as a Design Problem).
- Put the bound in the signature. A method that cannot be called without a limit cannot be called unboundedly, which is the difference between a guideline and a design (Cost-Aware Interfaces).
- Decide where aggregation happens — in the store, in the application, or precomputed — because that is a structural choice with three different change profiles, not a tuning knob (Choosing the Model).
- Leave the rest alone. Loop bodies, allocation counts and string handling are exactly what "premature optimisation" was written about, and they are cheap to change later because they are local (Premature Optimization, Reclaimed).
- Write down the assumption that makes the design fine — "bounded because a customer cannot have more than N open orders" — so that the day it stops being true, someone can find it (Decision Records).
What the next change costs
The field this whole domain exists for. A structure is only better if it makes the change after this one cheaper — and it is worth saying which changes it does not help.
- Adding a panel to the dashboard: under a bounded, batched design it is one more query with a known shape. Under the naive design it is one more round trip per request, and the cost of the page grows with the number of features on it — which is the definition of a design that gets worse as it succeeds.
- Introducing pagination later: with the bound already in the signature, it is a parameter change. Without it, every caller of the method has to be found and re-examined, and the ones that used the full list for something else have to be redesigned (Change Amplification).
- Moving aggregation from the application into the store: one module if aggregation was a named responsibility, and a search-and-replace across handlers if it was not.
- The honest cost of the bounded design: every caller now handles "there is more", including the twelve that never will have more. That verbosity is paid on every call site forever.
- Designing for cost up front spends complexity on a problem you may never have. On a system that stays small, the unbounded version was correct and cheaper for its whole life.
- Bounded interfaces are more verbose at every call site, and the verbosity is real work for readers as well as writers.
- Thinking about round trips early biases toward batching, which biases toward coarse interfaces — and coarse interfaces are harder to change than fine ones (Module Granularity).
What can go wrong
- Cost-aware design becomes cost-obsessed design: every interface grows pagination, batching and cursors, including the ones over a table with nine rows in it (Speculative Generality).
- The bound exists and is a default —
limit = 1000— so nothing fails, it just silently truncates, which is worse than being slow because the answer is wrong (Swallowed Errors). - The design is right and the ORM undoes it: a lazily-loaded association turns one query into one per row without any code changing (Backend Engineering owns the ORM mechanics).
- Precomputation is introduced, and now there are two sources of truth for the same number with no reconciliation (Consistency Boundaries).
- Every panel that reads through a bounded interface depends on the bound, which is a deliberate constraint pointing inward rather than an optimisation bolted on.
- Aggregation in the database couples the design to the store's capabilities and its query planner; aggregation in the application couples it to how much data crosses the wire. Neither is free and the choice should be made knowingly (Dependency Direction).
- A precomputed rollup adds a write-path dependency: every write now has a second consequence, which is a much larger change than it looks (Backend Engineering owns what a second write costs).
- "So we should optimise early after all." No. Nothing here says tune anything. It says an interface that cannot express a bound, and a read with no bound, are structural decisions you are making whether or not you notice (Premature Optimization, Reclaimed).
- "Measure first" means do not think first. Measurement answers "which of these costs matters"; it cannot answer "which costs did our structure make unavoidable", because the alternative structure does not exist to be measured.
- "The database will handle it." It will, for a while, and the moment it does not, the fix is a design change under load rather than a configuration change.
- "This is capacity planning." Capacity planning asks how much hardware a known shape needs. This asks what shape you just committed to, which is upstream of that question (capacity planning belongs to Observability & Performance).
- primitive-obsession
- shotgun-surgery
Testing it, and how it ages
- Assert the shape, not the speed: a test that a dashboard request issues a bounded number of queries is stable, fast and catches the regression that matters (N+1 as a Design Problem).
- Test with a volume the design claims to handle. A fixture with three rows exercises no bound and is why unbounded reads reach production (Testing as Design Feedback).
- Assert the failure of the bound explicitly — what happens on the 1001st row — because a silent truncation passes every test that only looks at the first page.
- Leave latency assertions to the performance environment. A timing assertion in a unit test measures the CI machine (Observability & Performance catalogues why that number means nothing).
- Read paths get bounds retrofitted under incident pressure, which is the expensive moment to do it. The teams that avoid that are not the ones that guessed better; they are the ones whose interfaces had somewhere to put a bound.
- As the product succeeds, the ratio changes: the design that was fine at forty rows per customer meets the customer with four hundred thousand, and this is always a specific customer rather than an average (this is why an average never warns you, which Observability & Performance treats properly).
- Eventually some panel genuinely needs precomputation, and the design ages well if aggregation was already one module rather than a query inside a handler.
Where this applies
This domain's advice is contested more than most. These labels say what each claim is specific to — and where CONTESTED appears, the note gives the strongest form of the opposing view rather than a caricature.
- GENERALThat crossing a boundary costs orders of magnitude more than not crossing one — memory to disk to network — holds across every stack; what varies is the constant, not the ranking, which is why the shape argument transfers when a number would not.
- SCALE-SPECIFICUnder a few thousand rows per tenant an unbounded read is genuinely fine and the bounded version is wasted work; the argument turns on whether any customer's data grows without limit over time, which is a product question rather than an engineering one.
- CONTESTEDThe strongest opposing view: teams systematically over-predict which paths will be hot, and cost-aware structure imposed up front usually lands on the wrong ones — so the discipline of building the simple thing, measuring, and fixing the two paths that actually matter delivers better systems for less total effort. That is well-supported for tuning and for caching. Its weakness is the small set of decisions that measurement cannot reverse cheaply: interface shape, ownership of aggregation, and whether the API can express a bound at all.
Where the depth lives
This domain teaches the codebase-level structure and hands the rest off.
- — System Design — capacity, replication and cache placement decide the constants this module treats as fixed, and the two views meet at the question of how much work one request is allowed to cause.