Messaging

Putting a broker between two services is usually sold as "decoupling". The precise trade is narrower and more useful: you convert a live availability dependency on the consumer into a durability dependency on the broker, and you pay for it with unbounded staleness and a new operational surface.

Q · Both services are running. Why not just call the other one directly?

A work queue distributes tasks across a pool of interchangeable workers. Each message is meant to be handled once, by whichever worker grabbed it. That single design choice — a claim, then a destructive completion — determines the scaling behaviour, the ordering behaviour, and every failure mode in the module.

Q · I have more work than one worker can do. What does a queue actually give me, and what does it take away?

In pub/sub the publisher announces that something happened and does not know or care who listens. Each subscriber gets its own copy and its own fate. The value is that adding a consumer requires no change to the producer; the cost is that the producer can no longer tell you whether anything downstream worked.

Q · Three teams need to know when an order is placed. How do I tell all of them without the order service knowing they exist?

The choice is usually decided by a single question: should one worker handle this, or does every interested party need to see it? That rule gets you the right answer nearly always — and then consumer groups over a log give you both at once, which is why the distinction has been quietly collapsing for a decade.

Q · Do I need a queue or a topic — and why does the answer keep being "a log"?

Receive, process, acknowledge. Where you put the acknowledgement relative to the processing is not a detail — it is the entire choice between at-most-once and at-least-once, and there is no arrangement of those three steps that gives you exactly-once.

Q · When exactly should my consumer tell the broker it is done — and what does the answer cost me?

When a consumer receives a message, the broker does not give it away — it hides it for a bounded period. If processing outlives that period, the message reappears and a second worker starts on it while the first is still running. This is not a rare edge case; it is a concurrency bug the broker will manufacture for you on a timer.

Q · My handler sometimes takes longer than expected. What does the broker do about it, and what does my code have to survive?

Retrying a transient failure is correct. Retrying a message that can never succeed is an infinite loop that consumes your consumers, your logs and your budget, and in an ordered stream it stops everything behind it. Telling the two apart is the whole problem, and you cannot do it perfectly.

Q · This message fails every single time. How long should I keep trying, and what stops it from taking the pipeline down?

Routing failed messages to a DLQ protects the pipeline, and that is the easy half. The DLQ is only a safety mechanism if someone is alerted, can inspect what failed, can fix the cause, and can make an explicit decision to replay or discard. A DLQ nobody reads is not a safety net — it is a silent data-loss mechanism with a reassuring name.

Q · The message failed five times and went to the DLQ. Now what — and who finds out?

An append-only log is an immutable, ordered sequence that consumers read by advancing a position. Nothing is removed when it is read; retention is governed by time or size, not by consumption. That single structural difference — a cursor over immutable data instead of a claim over a mutable set — is why a log supports replay, multiple independent readers, and per-key ordering, and why a queue never can.

Q · Everyone calls Kafka a message queue. What is actually different, and why does it change my design?