advancedMeasuring What Matters

The Free-Shipping Experiment

Decide what you would do from the brief alone. Everything below it is available, but the exercise stops working if you open it first.

The situation you were handed

PM: "Great news — the lower free-shipping threshold won the A/B test. Completion is up. Let's ship it to everyone on Monday."

The trap — the move that looks right

Shipping on Monday because the primary metric won and the dashboard said significant. The team celebrates higher completion while each extra order costs more to ship than it earns, and by the time finance notices the margin drop it is attributed to the season.