CDC or a polling extract — which, and why?
Whether the candidate understands what each mechanism can and cannot observe, and whether they know the specific way a timestamp-based extract loses rows.
The situation behind the question
Interviewers ask this because it happened to them.
A source PostgreSQL database holds orders that are updated and occasionally deleted. The current extract polls WHERE updated_at > :last_run every hour. Someone proposes change data capture instead.
A strong answer
Flags
Green flags
- Knows the commit-versus-timestamp race and can explain why it produces a permanent loss.
- Understands the source database well enough to talk about the log, the slot and retention.
- Names deletes as the thing polling structurally cannot observe.
- Preserves raw and replayable data, and treats log retention as the recovery window.
Red flags
- "Data engineers don't need to understand databases." Everything about this decision is database internals — commit visibility, log retention, replication slots.
- Prefers CDC because it is real-time, with no mention of what it costs the source.
- Cannot describe what happens when the connector is down for longer than log retention.
- Believes a polling extract with a small overlap window is equivalent to CDC.
Follow-ups
Where the conversation goes if the first answer holds up.
- The connector was down for four hours and the log has since rotated. What do you do?
- How would you detect the slow loss from the timestamp predicate, given that it never alerts?
- The source is a SaaS API with no change log. What is the best available answer?