Outcome vs Output
Output is what shipped; outcome is what changed because it shipped. Being measured on the second changes what you build, how small you build it, and what you do after launch.
The ask, the obvious response, and how it goes wrong
Every lesson starts where the work starts: someone asked for something, and the first response that comes to mind has a problem.
When this quarter ends, will we describe it by what we shipped or by what changed?
The quarterly review slide reads: "Shipped one-click reorder, guest checkout redesign, new order-status page. 42 tickets closed." The engineering manager asks the team to "show more impact next quarter".
Show more impact by shipping more and shipping bigger. Push velocity up, close more tickets, and put every launch on the slide with a screenshot.
Velocity goes up and nothing else does. The team ships more features nobody measured, and the next review cannot say which of them mattered either.
- Velocity goes up and nothing else does. The team ships more features nobody measured, and the next review cannot say which of them mattered either.
- One-click reorder is used, but nobody recorded the reorder rate before launch. There is no way to tell whether it created reorders or just moved existing ones onto a faster button.
- The order-status page shipped and "where is my order" is still the largest support category, because the page shows when the parcel left the warehouse and shoppers want to know when it will arrive.
- Finance asks what the guest checkout redesign did to conversion. The honest answer is "we do not know", and the next conversation about the team's headcount goes worse than it should.
What is really going on
- Output is under the team's control and visible the day it happens. Outcomes are partly outside the team's control and show up weeks later. Organisations drift to reporting output because it is cheap and immediate, not because it is what anyone cares about.
- An outcome is a change in someone's behaviour or situation: shoppers reorder more often, support gets fewer "where is my order" tickets, finance closes the month a day earlier. It names a person and a change, and it can turn out false.
- What you are measured on changes what you build. Measured on output, the rational move is more and bigger features. Measured on outcomes, the rational move is the smallest change that moves the number — and removing things that did not.
- Outcomes need a baseline recorded before launch. Without "before" there is no "after", only a number with nothing to compare it to. That is why Instrumentation First is part of this habit, not a separate tool.
- Output still matters. A team that ships nothing moves no outcomes. The point is not to stop counting output, but to stop presenting it as the result.
Two ways to describe the same quarter
The slide in the requirement is not wrong. Everything on it happened. The problem is that it cannot be wrong, which is exactly why it cannot guide anything: no line on it tells the team what to do next.
Shipped one-click reorder, guest checkout redesign and the new order-status page. 42 tickets closed, velocity up.
Order-status page: where-is-my-order tickets unchanged — the page shows warehouse dispatch, shoppers want the delivery date; fixing that next. One-click reorder: used, but the 60-day reorder rate is flat, so it sped up reorders that were happening anyway. Guest checkout: no baseline was recorded, so we cannot say; we are recording one before the next change.
The output version is comfortable everywhere and informative nowhere. The outcome version is uncomfortable in three places, and each of those places is a decision for next quarter.
The outcome line and the baseline
The outcome line is one sentence in the ticket: who, what changes, how you will see it. The baseline is the query that measures it, run before launch and saved where the "after" run will find it.
The query does not need to be clever. It needs to be the same query both times, with its window defined carefully enough that "after" is not a different number for a boring reason.
1-- Outcome: customers who ordered before reorder within 60 days more often.2-- Run before launch and again six weeks after. Same query both times.3SELECT4 date_trunc('week', o.created_at) AS order_week,5 count(DISTINCT o.customer_id) AS customers,6 count(DISTINCT r.customer_id) AS reordered_60d,7 round(100.0 * count(DISTINCT r.customer_id)8 / nullif(count(DISTINCT o.customer_id), 0), 1) AS reorder_rate_pct9FROM orders o10LEFT JOIN orders r11 ON r.customer_id = o.customer_id12 AND r.created_at > o.created_at13 AND r.created_at <= o.created_at + interval '60 days'14WHERE o.created_at < now() - interval '60 days' -- only orders old enough to have a full window15 AND o.created_at >= now() - interval '180 days'16GROUP BY 117ORDER BY 1;The window matters more than the syntax. Orders younger than 60 days cannot have a 60-day reorder yet; including them makes every recent week look worse and would make the launch look like it hurt.
Reporting a launch that did not work
The moment that makes outcome reporting trustworthy is the first time the team says, unprompted, that something did not work. It is also the moment most teams avoid.
Six weeks after the order-status page shipped, you look at the support queue. Where-is-my-order is still the largest category. The team demoed the page at the all-hands.
"The order-status page is live and getting good traffic. We are continuing to iterate on it."
"The order-status page has not reduced where-is-my-order tickets. Reading twenty of them, shoppers are asking when the parcel will arrive, and the page tells them when it left the warehouse. The carrier already sends us an estimated delivery date we are not showing. That is about three days of work, and I would like to do it before anything else in this area."
Where output is the honest thing to report
Some work has no outcome in the sense this lesson means, and inventing one is worse than admitting it. A database upgrade, a security patch, a tax-rule change and a migration off a payment API that is being retired are done when they are done.
For that work, report the reason and the risk removed: "Upgraded before end of support; no customer-visible change, which was the goal." That is honest output, and it belongs on the slide next to the outcomes, labelled as what it is (Tech Debt Is a Product Decision).
How to do it
Most important first.
- Before starting any piece of work, write the outcome in one line: who, what changes, how you will see it. "Customers who ordered before reorder within 60 days more often; we will see it in the 60-day reorder rate."
- Record the baseline before shipping. A few weeks of the number is better than none; a saved query in the ticket is the minimum (Picking a Metric That Moves When the Product Gets Better).
- Put a review date in the calendar on launch day — two to six weeks later, depending on how often the behaviour happens — and look at the number on that date.
- Report launches as outcome statements, including the ones that did not move. "Order-status page shipped; where-is-my-order tickets unchanged; the page shows the wrong date" is worth more than a screenshot.
- When an outcome takes months to show, name a leading signal you can see sooner, say plainly that it is a proxy, and add a guardrail so the proxy cannot win at the outcome's expense (Guardrail Metrics).
- Let outcomes change the backlog. Work whose outcome did not move is a candidate for iteration or removal, not a finished item (The Cost of a Feature).
How to explain the decision
The sentences, the order, and what to lead with — for someone who did not make the call.
- Lead with the change, then the thing that caused it: "Where-is-my-order tickets fell from the largest category to the third after we started showing the carrier's delivery estimate."
- Be honest about attribution: "The sale ended that same week, which lowers volume anyway, so part of the drop is seasonal. The category's share of the queue is the cleaner number, and that dropped too."
- Report non-results as findings: "One-click reorder is used, but the 60-day reorder rate has not moved. It looks like it moved existing reorders onto a faster path. That is a small win for shoppers; it is not growth."
- End with what it means for the next decision: "So I would not invest more in reorder this quarter. I would spend the time on delivery-date accuracy, which is still generating tickets."
- "Leadership wants to see what we shipped." Show them both, outcome first and launch second. Most people would rather know the thing worked than see that it exists.
- "We cannot control outcomes, so it is unfair to be measured on them." Partly true — which is why you report the outcome alongside what else changed, and why a missed outcome is a finding, not a performance review.
- "This will slow us down." It adds about a day to the next feature. It may save the team building a second version of something that did not work the first time.
What can go wrong
- Outcome-washing: renaming outputs as outcomes. "Improved the checkout experience" is still an output with a nicer verb.
- Claiming every movement. Reorders rose in the month the sale ran; the button may have done nothing. Say what else changed at the same time.
- Choosing an outcome the team cannot influence, like company revenue, and so never learning anything from it.
- Gaming the number: pushing reorder prompts until shoppers unsubscribe, so the metric rises while the thing it stood for falls.
- Dropping output tracking entirely, so nobody can tell whether the team is blocked or simply waiting for numbers to arrive.
- "Output does not matter." It does. A team measured only on outcomes that never ships has neither.
- "Outcomes are just OKRs." OKRs are one way to write them down. The habit is naming who changes and how, which works with or without the framework.
- "Every ticket needs an outcome." Maintenance, fixes and compliance need a reason, not a metric. Forcing a number onto them teaches the team to invent numbers.
- "The number went up after we shipped, so we caused it." A change that coincides with a launch date is the weakest evidence there is; see Reading an Experiment Honestly.
Knowing whether it worked
- The quarterly review has fewer lines, and each one has a before and an after.
- Someone on the team says a launch "did not work" in a meeting, and it is treated as information rather than blame.
- Tickets in the backlog start with a "so that" clause naming a person and a change.
- The team removes or reverses something because its outcome did not move. It is the clearest sign the measurement is real.
- A small team can hold its outcomes in its head and check them at standup. At 10x team size outcomes need writing down, an owner and a review cadence, or they decay back into launch lists.
- At 10x users, small effects become visible within days, and experiments that were statistical noise at the old volume become possible (Reading an Experiment Honestly).
- At company scale outcomes nest: a team outcome such as fewer where-is-my-order tickets should connect to a company one such as lower support cost per order. If you cannot state the link, one of the two is wrong.
- Outcomes are slower and noisier to report than output, and some quarters the honest report is "we do not know yet".
- Being measured on outcomes exposes the team to things outside its control: a carrier outage can wreck a delivery metric the team did nothing to.
- Baselines and review dates are overhead on every piece of work, including the ones where nobody ends up looking.
- It makes necessary work look bad. Security fixes, migrations and compliance changes rarely have outcomes in this sense, and pretending they do is its own distortion.
Where this applies
Product advice is context-sensitive. These labels say what each claim is specific to, and where a different stage, team or product would differ.
- GENERALThe distinction applies to any team that ships to people. What differs is the timescale: a checkout change shows its outcome in days, a retention change in months.
- STAGE-SPECIFICBefore product-market fit the outcome is often simply "did anyone use it twice", and formal baselines are overkill. At a scale-up with steady traffic, experiments and precise baselines become both possible and expected.
- PRODUCT-SPECIFICFor platform and infrastructure teams the outcome is other teams' speed or incidents avoided, not shopper behaviour; for regulated work it can be "passed the audit". The order-page metrics here are the wrong level for them.