How do consumers avoid reading a half-written dataset?
Whether the candidate knows that a run failing loudly and a consumer seeing a broken state are independent problems, and can name mechanisms for making a publish atomic.
The situation behind the question
Interviewers ask this because it happened to them.
The nightly build writes a fact table in several statements. A scheduled report runs on its own timer and occasionally captures a state that never existed as a consistent whole.
A strong answer
Flags
Green flags
- Knows freshness and completeness are consumer-visible properties that need an explicit publish boundary.
- Names a concrete mechanism appropriate to the storage layer rather than describing the goal.
- Identifies the delete-then-insert window and what a consumer sees inside it.
- Wants an assertion that the expected period exists before the swap, not only after.
Red flags
- Assumes a failed run means nothing was published.
- Solves it by scheduling the consumer later, which is coordination by hope.
- Cannot distinguish "the partition is missing" from "the partition is present and empty".
- Treats transactional guarantees in the warehouse as automatically covering a multi-statement pipeline.
Follow-ups
Where the conversation goes if the first answer holds up.
- Your storage layer has no atomic rename. What are your options?
- A backfill is rewriting the same partitions consumers are reading. Does your mechanism still hold?
- How does a consumer tell a genuinely quiet day from a partition that was never written?