Instrumentation First
Decide how you will know before you build, and ship the events a cycle ahead of the feature — because "measure it later" has no baseline, and a number with no before is a number with no meaning.
The ask, the obvious response, and how it goes wrong
Every lesson starts where the work starts: someone asked for something, and the first response that comes to mind has a problem.
If this ships next month and someone asks whether it worked, what will you compare it against?
The PM says: "Let's get the new shipping-options step out first and add tracking afterwards — analytics is a follow-up ticket."
Ship the feature, then add analytics. Tracking is not user-facing, it can always be added later, and nobody wants to delay a launch for events.
After launch there is only an "after". The new step shows a 70% pass-through, and nobody knows whether the old flow was at 60% or 85%. The number is real and means nothing.
- After launch there is only an "after". The new step shows a 70% pass-through, and nobody knows whether the old flow was at 60% or 85%. The number is real and means nothing.
- The follow-up ticket loses every prioritisation fight to the next feature, because by then the launch has been celebrated and nobody is waiting on the answer.
- When the events are finally added, they are added to the new flow only. There is no way to recreate the old flow's behaviour, so the comparison can never be made.
- Events designed after the fact are designed around what the feature does, not around the question it was meant to answer. You get "button_clicked" and not "reason people left at shipping".
What is really going on
- Measurement is a comparison, and a comparison needs two sides. The "before" side can only be recorded before the change. Every day of delay is a day of baseline you cannot get back.
- Baselines have cycles. A store has weekday and weekend shapes, pay-day spikes and month-end dips. A baseline shorter than the cycle you care about is an anecdote.
- The question comes first, the events second. An event exists to answer a question; one written without a question gets named after UI elements and answers nothing when the UI changes.
- Instrumentation is product code with product failure modes: ad blockers drop client events, retries double-count, a renamed step silently breaks a funnel. Treating it as a side task is how you end up trusting numbers that are wrong.
Later has no baseline
The reason instrumentation goes first is not discipline. It is that the before only exists before. Once the new shipping step replaces the old one, the old flow's behaviour is gone; nothing you add afterwards can record it.
The order below is the whole lesson. Each arrow is a step that cannot be reordered without losing something.
An event is a question, written down
Name events for the job, not the widget. The widget will change in the next redesign; the job — getting through the shipping step — will not. Write the question the event answers next to it, so the person who wants to delete it in a year knows what they would lose.
1event: checkout_step_viewed2question: At which step of checkout do people stop?3owner: checkout-team4emitted_from: server # client events are dropped by blockers and double-fire on refresh5properties:6 checkout_session_id: string7 step: [cart_review, address, shipping, payment, confirm]8 cart_value_band: [under_25, 25_to_100, over_100]9personal_data: none # no email, no address, no user id beyond the session10since: 2026-09-0111verified_against: orders table, daily count of 'confirm' vs placed ordersThe question line is the one people skip and the one that matters most. The value band instead of the exact cart value answers the likely follow-up without storing more than needed.
Asking for a week before the feature
The hard part is not writing events. It is asking for them to go out before the feature, when everyone wants the feature.
Planning. The PM wants the new shipping-options step live in three weeks and has put "add analytics" as a follow-up ticket for the sprint after.
"We really should do analytics properly this time. Can we make it part of the definition of done?"
"Can I pull three events out of the follow-up and ship them this week, on the current flow? They're half a day. That gives us two full weeks of how many people leave at shipping today, so when the new step lands we can say whether it helped. If they slip, the feature doesn't wait — we just won't have a before."
Trust the events only after reconciling them
An event stream that has never been checked against something else is a rumour. Before any decision rests on it, compare the count of confirm-step events with the orders table, day by day. The gap will not be zero — sessions time out, clients disconnect — but it should be stable and explained. A gap that jumps on a release day is a tracking bug, and it will be mistaken for a product effect if nobody looks.
Checkout completion jumped from 61% to 68% the day the new step shipped. Great result.
Completion events jumped the day the step shipped, but placed orders did not. The new step fires "confirm" on page load, not on submit. After fixing it, completion moved from 61% to 62%, within normal weekly range — we need two more weeks to read it.
The first reading credits a tracking change to the product. Only a check against an independent source separates the two.
How to do it
Most important first.
- Write the question and the metric first (Picking a Metric That Moves When the Product Gets Better). Then list the smallest set of events that would compute it, and nothing else.
- Ship the events against the current flow a cycle before the feature — ideally one or two full weeks — so the baseline covers the weekly shape. It is usually a small change and can go out behind nothing.
- Emit the events that decide the result from the server where you can; use the client for things only the client knows, and record which is which.
- Give each event an owner, a written meaning and a stable identity that survives UI changes: "checkout_step_viewed { step: shipping }", not "blue_button_clicked".
- Verify the events against a source of truth before trusting them: do "order_placed" events match the orders table, day by day? If they do not, fix that before building anything.
- Keep personal data out of events by default. The question is almost never "which person"; it is "how many, at which step".
How to explain the decision
The sentences, the order, and what to lead with — for someone who did not make the call.
- Lead with the question the PM will be asked: "In six weeks someone will ask if the new shipping step helped. Right now we can't answer, because we don't know how many people leave at shipping today."
- Make the cost concrete and small: "It is three events and half a day. If we ship them this week, we get two full weeks of the old flow before the new one lands."
- Name what you lose by waiting: "If tracking comes after launch, we only ever see the new flow. We'll have a number and nothing to compare it to."
- Offer the trade: "The feature does not wait for this — the events go out on their own, before the feature is ready. If they slip, we ship anyway and say the result is unreadable."
- "Just ship it, we can measure later." Later has no baseline. Three events this week are the difference between knowing and arguing.
- "It's obviously better, why do we need numbers?" If it is obviously better, the numbers will say so cheaply. If it isn't, we will be glad we found out before building the next thing on top of it.
- "Privacy said no more tracking." Good — then we design events with no personal data: step names and counts. Ask privacy to review those specifically; it is usually a different conversation.
What can go wrong
- Instrumenting everything "in case": hundreds of events nobody owns, a bill nobody reads, and still not the one event the next question needs.
- A baseline recorded during an unusual week — a sale, an outage, a holiday — and then compared against a normal week after launch.
- Changing the event's meaning when the feature ships (the "shipping" step now includes address) so before and after measure different things under the same name.
- Treating the instrumentation PR as a blocker for the feature PR, and slipping the launch to perfect tracking. The point is a usable baseline, not a complete one.
- "Instrumentation first means instrument everything." It means instrument the question. Five well-defined events beat two hundred that nobody can explain.
- "We can reconstruct the baseline from the database." Sometimes, for orders. Almost never for the steps people abandoned, because an abandoned step writes nothing.
- "Tracking is the analytics team's job." The engineer changing the flow is the only person who knows exactly what changed; an event someone else adds later will measure a guess.
Knowing whether it worked
- At launch there is a chart with a line before the change and a line after it, on the same definition.
- The event counts reconcile with the orders table within a margin you have written down, and someone checks it after each release.
- Nobody on the team has to say "we'll know once we add tracking" in a launch review.
- The next question about checkout can be answered from events that already exist, because they were named for the job rather than the UI.
- At 10x traffic, sending every client event becomes a cost and a performance line item; sampling and server-side events become the default, and the sampling rate must be recorded with the data.
- At 10x team size, event names collide and drift. A shared schema with owners and a review step — the same as for an API — is what keeps "checkout_started" meaning one thing.
- At 10x revenue, the baseline itself becomes valuable for more than one team, and removing an event needs the same care as removing an endpoint.
- It moves work earlier, when the feature is least certain. Sometimes you instrument for a feature that is then cancelled, and the events answer a question nobody asks.
- It needs a cycle of lead time. A feature that must ship this week cannot have a two-week baseline, and pretending otherwise delays something real.
- Server-side events are more trustworthy and more work than dropping a client-side snippet in, and they miss things only the browser sees.
Where this applies
Product advice is context-sensitive. These labels say what each claim is specific to, and where a different stage, team or product would differ.
- GENERALA before-and-after comparison needs a before, for any change meant to move behaviour. It does not apply to changes whose success is binary and visible, like a compliance fix that either passes the audit or does not.
- PRODUCT-SPECIFICWith daily consumer traffic, one or two weeks of baseline is readable. For a B2B product with monthly usage cycles, the baseline needs to cover a month or more — or the comparison is better done through accounts and conversations than through rates.
- SIMPLIFIEDA before/after comparison is weaker evidence than a controlled experiment, because anything else that changed at launch time gets credited to the feature. Where you can split traffic, do; the baseline is still what tells you what normal looks like.
Where the depth lives
This domain teaches the product-side judgement and hands the mechanism off.