FundamentalsIntermediate

Why did you add that box?

“A candidate draws a load balancer, a cache, a queue and a CDN before writing down a single number. How should each of those be justified? Walk me through a system that grows from browser → backend → database, adding components only when a measured problem appears.”

What this tests

  • Architecture as evolution caused by problems, not a target picture
  • Whether every component is tied to a symptom and a metric
  • Awareness that each addition introduces a new problem
  • Estimation habit: numbers before boxes

Answers by level

Read the beginner answer first and notice what is missing.

Every box must answer: what symptom did we measure, what does this fix, and what new problem does it add. Browser → backend → database is a complete architecture for a lot of products. The first real symptom is usually CPU saturation or p99 latency on the single backend under load; the fix is a second instance behind a load balancer, and the new problems are session state and deploys across instances (so the service must be stateless).

Next symptom: the database is read-heavy and its CPU is the bottleneck — say 90% reads, the same 1,000 products fetched repeatedly. A cache in front of it cuts DB load, and adds staleness, invalidation and stampedes. A queue arrives when a request does work the user does not need to wait for (sending email, generating a PDF) or when bursts exceed what synchronous processing can absorb; the cost is eventual consistency and an invisible backlog. A CDN is justified by static bytes or geographic latency (150 ms round trip to a far origin), and adds purge and cache-key problems.

The point is not to avoid these components; it is that each rung of the scaling ladder buys capacity by adding a problem, and adding the problem before the capacity is needed is pure cost.

Green flags · Red flags

Strong green flag · Says the queue must be justified by retryable, deferrable work — not by the word "decoupling".
Green flags
  • Starts from the three-box system and treats it as legitimate
  • Names a measured symptom before each addition (CPU, p99, read ratio, byte size, geography)
  • States the new problem each component introduces
  • Does a rough traffic estimate unprompted
  • Profiles before caching (index vs cache)
Red flags
  • "You should always have a load balancer and a cache; it is best practice."
  • Adds components with no metric attached
  • Cannot name what a cache or queue makes worse
  • Designs for a scale two orders of magnitude above anything mentioned

Follow-up questions

F1
The database is at 95% CPU. Cache or read replica?
F2
What new failure mode does the load balancer introduce?
F3
When is the queue premature?

Scenario

A startup with 300 daily active users has a design document proposing an API gateway, six services, Kafka, Redis and a CDN before launch. The single engineer building it estimates four months. Rewrite the plan: what ships in week one, and what specific measurement would trigger each later addition?

Learn this topic