Design a Concurrent Image Pipeline

0 of 5 classified. Two of the five compute and three of them wait — which is why one pool sized for either one of those is wrong for the other.

1. 1Classify every stageWhich stages wait, and which stages compute? Nothing else on this list can be answered before this one.
   Must cover

   • • Each of the five stages labelled I/O-bound or CPU-bound, with the reason.
   • • The consequence: I/O-bound stages want concurrency, CPU-bound stages want parallelism, and they want different worker counts.
   • • What happens if you run them all in one pool sized for one of the two — the classic mistake.
2. 2Decide what runs concurrently and what runs in parallelWhich stages overlap in time, and which genuinely execute simultaneously?
   Must cover

   • • Downloads and uploads overlap freely; decodes and transforms occupy cores and cannot exceed the core count.
   • • A pipeline lets different images occupy different stages at the same time — the throughput win that does not require any single stage to be parallel.
   • • The distinction stated out loud: overlapping progress is not simultaneous execution, and the design depends on which one each stage needs.
3. 3Choose worker counts per stageHow many workers does each stage get, and what does the number depend on?
   Must cover

   • • CPU stages: on the order of one worker per core, then measured — never a formula stated as universal.
   • • I/O stages: bounded by what the dependency will accept, not by the core count.
   • • Why a single global pool for both leaves cores idle during the I/O stages and oversubscribed during the CPU stages.
   • • What signal you would use to adjust the numbers after launch.
4. 4Place the queuesWhere does work wait between stages, and who owns each queue?
   Must cover

   • • A queue between each pair of stages with different throughput, because that is what a stage boundary is for.
   • • What the queue depth tells you about which stage is the constraint.
   • • The one thing the queue must never be: unbounded.
5. 5Bound everythingWhat limits the number of in-flight jobs, and what happens to the ones that do not fit?
   Must cover

   • • A concrete bound on in-flight jobs, queue length and memory held by decoded pixel buffers — which are far larger than the compressed input.
   • • The behaviour at the bound: reject, block the producer, or shed. Naming which, and why.
   • • The failure you are preventing: a bounded queue turns an overload into a decision, an unbounded one turns it into an out-of-memory kill.
6. 6Name the shared stateWhat do multiple workers touch, and what invariant must hold over it?
   Must cover

   • • Every piece of genuinely shared mutable state, named individually — counters, caches, the job status, connection pools.
   • • The invariant each one must preserve under every interleaving.
   • • Which of them could simply stop being shared — per-worker accumulation, immutable configuration, a copy instead of a reference.
7. 7Decide what needs a lock, and how wideWhich critical sections exist, and how much work is inside each one?
   Must cover

   • • The minimal region each lock protects — not the function, the region.
   • • An explicit statement that no lock is held across a network call or a disk write.
   • • Where you chose an atomic, a per-worker counter or an immutable snapshot instead of a lock, and why.
8. 8Assess the deadlock riskCan a cycle form? Between what?
   Must cover

   • • The lock order, written down, including the database connection as a resource that is acquired and held.
   • • The classic shape in a pipeline: a worker holding a pool slot while waiting to push into a full downstream queue, and the downstream worker waiting on the same pool.
   • • Which of the four conditions you chose to break, and what that choice costs.
9. 9Design cancellationA job is cancelled mid-pipeline. What stops, and what has to be cleaned up?
   Must cover

   • • Propagation to children: the in-flight download, the decode, the partial upload.
   • • What survives cancellation and must be reconciled — a partially written object, a job row in a running state, a queue entry.
   • • A deadline for the whole job, distinct from a timeout on each call.
10. 10Define behaviour under overloadTen thousand jobs a second becomes thirty thousand. What does the system do?
    Must cover

    • • The intended behaviour, chosen deliberately: shed, queue with a bound, or degrade to a cheaper transform.
    • • What the producer sees, and how fast it finds out.
    • • Why "slow down for everybody" is a decision even when nobody made it.
11. 11Measure contentionHow would you know that workers are waiting on each other rather than working?
    Must cover

    • • The specific signals: lock wait time, queue depth and age per stage, pool checkout wait, effective parallelism against core count.
    • • The signal that stays reassuringly green — CPU utilization looks fine precisely because everyone is blocked.
    • • How you would tell a slow dependency apart from internal contention, using data you already collect.