Data Observability
Pipeline health is not data health. Freshness, volume, schema and quality as monitored signals, and the upstream walk that turns "revenue looks wrong" into a cause.
Pipeline health and data health are two different systems. A platform that watches only the first finds almost none of the incidents anyone cares about.
What an orchestrator genuinely knows, what it structurally cannot know, and how to make a task-level signal say something about data.
Rows processed, bytes processed, duration, failures, retries and lag — what each one detects, what moves it for boring reasons, and what none of them can see.
Freshness is a per-dataset property. Averaging it across a platform hides the one table that has not updated since Friday — and the false-positive rate decides whether anyone still reads the alert in six months.
Comparing today with the same weekday historically is the cheapest broad detector there is — and it misses every error that preserves row count, which is most value-level bugs.
A dashboard says revenue dropped eighty percent overnight. Seven different causes produce that symptom, and telling them apart is the job.
Consumer symptom to serving dataset to transformation to upstream dataset to ingestion to source. Debug upstream, always — and diagnose from the set of checks that failed, not from the first one.
Click a dashboard metric and walk it back — tile, metric definition, mart, model, staging, raw, change capture, production database — then turn around and ask what else this feeds.