Retries are unavoidable; duplicates are the tax

Idempotency & Delivery

6 lessons. Every one names the guarantee it claims, what a node can know, and how it fails.

The Retry Is a Decision, Not a Reflex▶ lab

A timeout leaves the outcome unknown. Retrying and not retrying are both guesses, with different costs — and the moment you retry, the receiver has no way to tell your second attempt from a genuine second request unless you gave it one before the first attempt left.

Q · The outcome is unknown. Should I retry — and what have I actually done to the receiver when I do?

Idempotent Is a Property of the Whole Effect, Not the Write▶ lab

An operation is idempotent when applying it twice leaves the world in the same state as applying it once. The trap is scope: the database write may be idempotent while the handler around it — which also emits an event, increments a counter and sends an email — is not. Idempotence is only as strong as the least idempotent thing the handler does.

Q · What makes an operation actually safe to repeat — and why do handlers that "are idempotent" still produce duplicate effects?

What Counts as the Same Operation?▶ lab

Deduplication rests on a claim that two requests are "the same". That claim is made across a machine boundary, using only what the caller sent, by a store that has its own partitioning, retention and failure domain. Every one of those is a scope boundary, and a duplicate slips through wherever two of them disagree.

Q · Two requests arrive carrying the same key. Under what conditions is that enough to conclude they are the same operation?

Where You Put the Acknowledgement Decides Everything▶ lab

At-most-once and at-least-once are not two configuration options with a third, better one hiding behind a paywall. They are the two possible positions of the acknowledgement relative to the processing — and since ack and processing are separate actions that a crash can land between, the choice is forced by physics, not preference.

Q · Why are there exactly two delivery semantics, and what decides which one I have?

Exactly-Once Is a Scope, Not a Guarantee▶ lab

Three different things get called exactly-once: message delivery, processing attempts, and business effect. Delivery cannot be exactly-once over an unreliable network — that is a proof. What real systems provide is exactly-once *effect* for outputs inside one transactional boundary, and at-least-once for everything outside it.

Q · Systems advertise exactly-once. What are they actually offering, and where does it stop?

Deduplication: Bounded Memory Against an Unbounded Stream▶ lab

Detecting a repeat is a set-membership test over a stream that never ends, using memory that does. Every design choice — where the check happens, how exact it is, how long it remembers, how it is partitioned — is a way of trading one of those against the others, and each has a failure that looks like nothing at all.

Q · Given that duplicates will arrive, where and how do I detect them without storing every identifier forever?