One Slow Vendor Took Down Everything
Read what each party saw, commit to a cause, and only then find out which of them was right. The root cause, why the obvious reading was wrong, and the fixes are all held back until you have answered.
The symptom
What was reported, before anyone knew what was happening.
What each party saw, and what each concluded
The evidence, in the form it actually arrived in — several parties, several partial views, several confident conclusions.
| Who | What they could actually see | What they concluded | Verdict |
|---|---|---|---|
| Tax vendor | Its own degradation, disclosed 20 minutes later. | A partial incident affecting tax calculation. | ✓ right |
| Application servers | All 200 worker threads occupied waiting on outbound HTTP. | Fully utilised; queue incoming requests. | ✕ wrong |
| Home page handler | No thread available to run on. It makes no external calls at all. | Nothing — it never executed. | ✓ right |
| On-call engineer | Total outage; static pages failing. | A platform or infrastructure failure — CDN, load balancer or DNS. | ✕ wrong |
2 of 4 parties reasoned correctly from what they could see and still reached the wrong conclusion. Nobody in this table is careless. Each one acted on complete-looking local information, and the information was local. That gap — between what a node can observe and what is true — is the whole domain, and one of these readings will usually be yours.
Commit before you read on
Only one dependency degraded, and it is used by one flow. Commit before reading on: name the shared resource that carried the failure from that flow to unrelated ones, and say which of the two obvious fixes — a timeout or a circuit breaker — you would deploy first, and why.
Write it down even if you are unsure. An unwritten guess quietly becomes “that is what I thought” the moment you read the answer.