The Data Quality Dashboard
One row per dataset — pipeline, freshness, completeness, status — and a hard rule that a green row is a statement about the checks you wrote, not about the data.
Who needs this, what one row is, and why the obvious build breaks
Every lesson starts from the consumer, because designing from the source outward is this domain's characteristic mistake.
What should a single screen show so that a consumer can decide, in five seconds, whether to trust the table they are about to query?
Two audiences with opposite needs. A consumer wants one verdict per dataset, immediately, in the place where they read the data. An operator wants the underlying series, the history and the drill-down. A dashboard designed for one of them and given to the other is the usual outcome (Dashboards Built Around Questions).
One row per serving dataset, not per check and not per pipeline. A consumer queries a table, so the verdict must be attached to a table; rolling several tables into a pipeline row hides the one that is broken, and splitting one table across twenty check rows asks the consumer to do the aggregation themselves.
Show the orchestrator's DAG status. Green boxes, red boxes, one screen, already built and already integrated. It answers the question people ask most often — "did it run" — and it costs nothing to produce.
Every box is green and the payments table has not received a row since Friday, because the run succeeds against an empty source (Freshness Checks).
- Every box is green and the payments table has not received a row since Friday, because the run succeeds against an empty source (Freshness Checks).
- The DAG is a pipeline view, so a consumer looking for
dim_customershas to know which DAG builds it. Most do not, and the dashboard goes unread by the audience it was built for (Data Discovery). - One overall platform health percentage is displayed. It is high, it is always high, and it is high on the day the revenue model is duplicating rows, because one broken table among two hundred barely moves an average (The Average Was Fine and Users Were Not).
- The dashboard shows checks that passed and omits checks that did not run, so a suite that silently stopped executing displays as perfect health (Data Tests).
- Everything is green because the only checks configured are not-null assertions on columns that cannot be null. The dashboard is accurate, comprehensive and meaningless (Data Quality).
What is actually happening
- The dashboard is a projection of the check portfolio, so its coverage is exactly the portfolio's coverage. Green means "no check that we wrote found a problem", which is a much weaker statement than the visual grammar of a green tile implies (Data Quality).
- Four columns carry most of the value: whether the pipeline ran, how fresh the data is, how complete the most recent closed period is, and a rolled-up status. Each answers a different failure class, and the roll-up is only as honest as the worst of them.
- The status column is a policy decision rendered as a colour. Deciding that 99.8% completeness is a warning and 75% is an error is a business judgement about tolerance, and the thresholds belong next to the dataset's declared SLO rather than inside the dashboard's code (The Freshness SLO).
- Three states are the minimum, and the third state that actually matters is unknown — no checks configured, or checks that did not run. Collapsing unknown into green is the single most common way these dashboards lie (Pipeline Observability).
- The dashboard is not a monitoring system. Nobody watches a screen; alerts do the waking up. What the dashboard provides is context during an incident and a trust signal before a query, and designing it as if it were an alert channel produces something that serves neither purpose (Quality Alerting).
One row per dataset
The shape below is deliberately unambitious: the dataset, whether the pipeline ran, how fresh the data is, how complete the last closed period was, and a status. Four facts and a verdict. Most quality dashboards fail by being either a DAG view, which answers the wrong question, or a wall of individual check results, which asks the consumer to do the aggregation.
Read the three rows as three distinct failure shapes. Orders is healthy. Customers is late but complete — the pipeline is running and something upstream is slow, so consumers can use it while knowing it lags. Payments is both stale and materially incomplete, which is not a degradation but an outage wearing a percentage.
The figures are illustrative and must not be copied. What transfers is the structure: a value next to its target in every cell, a period named for the completeness figure, and a status derived from declared thresholds rather than from a number that looked reasonable to whoever built the page.
The fourth row is the one most dashboards omit and the one that matters most. A dataset with no checks configured is unknown, not healthy, and rendering it green is how a quality dashboard ends up certifying the datasets it has never examined.
Every pipeline in this table succeeded. Three of the four rows have a problem, and the orchestrator can see none of them — which is the entire argument for a dashboard organised by dataset rather than by DAG.
| Dataset | Pipeline | Freshness (target) | Completeness, last closed day | Status |
|---|---|---|---|---|
fct_orders | Ran 04:12, succeeded | 5 min (target 1 h) | 100% against source count and net sum | GREEN |
dim_customers | Ran 04:15, succeeded | 3 h (target 1 h) | 99.8% against source count | WARN — fresh enough to read, late enough to mention |
fct_payments | Ran 04:20, succeeded | 8 h (target 1 h) | 75% against provider control total | RED — do not report from this table |
stg_sessions | Ran 04:22, succeeded | not measured | no check configured | UNKNOWN — the state most dashboards render as green |
What a green row is actually claiming
The visual grammar of a dashboard is stronger than any caveat written on it. A green tile reads as "this data is correct", and no amount of hover text changes that. So the honest design question is not what to write in the tooltip but what a green row must require before it is allowed to be green.
The weak version requires that no configured check failed. Under that rule, a dataset with one vacuous not-null test is green, and so is a dataset with no tests at all if unknown collapses into pass. The strong version requires that a minimum portfolio exists and passed — freshness, structure, and one check with an external reference — and renders anything less as unknown.
The strong version makes the dashboard look considerably worse on the day it launches. That is not a drawback; it is the dashboard doing its first useful piece of work, which is telling you how much of the platform has never been measured (Data Quality).
Roll up whatever checks happen to exist per dataset. A dataset with one not-null assertion is green. A dataset with no assertions inherits the platform default and is also green. The page is reassuring and the coverage is unknown.
A dataset is green only if it has a freshness check against a declared target, a structural check on its business key, and one check with an external reference — and all of them ran and passed. Anything less is UNKNOWN, in its own colour, listed first.
The purpose of the row is to let a consumer decide whether to trust a table, and a colour derived from an arbitrary set of assertions cannot support that decision. Requiring a defined minimum makes green mean the same thing on every row, which is the only property that lets a consumer read the page quickly. It also converts the dashboard into a coverage report, which is the artefact that actually drives the platform forward.
Reading a red row
A status is a starting point, not a diagnosis. The value of the four columns is that their combination narrows the cause before anyone opens a query editor, and the combinations are few enough to enumerate.
The pattern to internalise is the second row below: pipeline succeeded, freshness stale. That pair means your platform is healthy and the source is not, and it is the case most often misattributed to the data team. The third row is its mirror — fresh but incomplete — which almost always means a window closed before the data arrived.
Each row names a response rather than an explanation, because during an incident the useful output of a dashboard is the next action (Debugging a Data Incident).
| Trigger | Symptom | Cause | Response |
|---|---|---|---|
| Pipeline failed, freshness stale | The obvious case: a red DAG task and a table holding the previous period. | The run raised and nothing was published, so the previous correct partition is still in place (When a Task Fails Mid-DAG). | Fix and re-run. The data is stale rather than wrong, which is the recoverable failure — confirm the old partition was not partially overwritten (Atomic Publish). |
| Pipeline succeeded, freshness stale | Everything green in the orchestrator, newest record hours old. | The run carried nothing: the source stopped, the capture connector is behind, or the extract window returned empty (CDC Failure Modes and the Retention Deadline). | Do not re-run. Check the source first — this is a producer-side incident and the routing should already have sent it there (Who Owns Data Quality). |
| Freshness green, completeness low | Recent data present, last closed period materially short against its reference. | The window closed before events arrived, or a capture gap lost a range that later runs did not backfill (Late-Arriving Data). | Reconcile the period to size the gap, then backfill that range specifically with an idempotent merge (Planning a Backfill). |
| Everything green, a consumer reports a wrong number | No check disagrees with the data and the number is still wrong. | A failure class nobody asserted — most often an arithmetic or definitional change that leaves every structural property intact (Two Dashboards, Two Numbers). | Reconcile against the source for a closed period. If that also passes, the disagreement is definitional and belongs in the metrics layer (The Metrics Layer). |
| A row has been green for months and nothing else has | One dataset with a perfect record on a platform where everything else fluctuates. | Its checks are vacuous, or its suite stopped running and unknown is being rendered as pass (Data Tests). | Check that the expected number of checks executed, then break the dataset in staging and confirm the row turns red. |
| The whole dashboard is green and unchanged since yesterday | Uniform health, including for datasets known to be broken. | The dashboard itself stopped recomputing and is displaying its last state (Health Checks). | Show the dashboard's own last-computed time on the dashboard, and alert on it. The monitoring needs monitoring, and this is the cheapest place to add it. |
What the dashboard costs to keep honest
The page is a read model over a set of checks, so its cost is entirely the cost of the checks and the frequency at which they are recomputed. That means the design lever is not the dashboard — it is deciding which cell needs to be current and which cell can be a day old.
Freshness can be recomputed constantly because it reads a maximum or a table snapshot rather than the data. Completeness against an external reference is the expensive cell, because it queries a second system, and it only changes once per closed period — so recomputing it hourly buys nothing and costs everything (Reconciliation).
The weights below are relative and directional, not measurements. Their ordering is the teaching: the cell that carries the most information is the most expensive one, and the cheapest cells are the ones people already have.
Reads a second system, often an operational one that was not built for aggregates. Scope tightly to one closed period and run once per period, never on the dashboard's refresh interval (Workload Isolation).
Multiplies every check's cost directly. The commonest waste in these dashboards is recomputing a per-period figure many times within the period it describes.
Uniqueness and referential tests over all history rather than the new partition. Scoping them to the partition usually removes most of this without weakening the assertion for the period being published (Partition Pruning).
Per-dataset, per-check series are cheap; adding a per-category dimension multiplies them and is the usual cause of a metrics store growing faster than the data platform (Cardinality: The Label That Took Down Monitoring).
Metadata lookups and a maximum over a partitioned timestamp. Effectively free, which is why there is no defensible reason for these two columns to be missing.
Relative weights, not measurements. The ordering is what transfers between platforms; the magnitudes do not, and this domain never publishes a price.
The ordering is the point: the most informative cell is the most expensive one and changes least often, so cache it per period. The two cheapest cells are the two most often absent.
How to build it
Most important first.
- One row per serving dataset, with the dataset name a consumer would recognise, not the model name only its author knows (The Data Catalog).
- Show freshness with the target beside it, not merely the value. "3 h" means nothing; "3 h against a 1 h target" is a verdict a consumer can act on (Freshness Checks).
- Show completeness for the most recent closed period and say which period it is. Completeness on an open period is always low and always meaningless (Reconciliation).
- Render unknown as its own state with its own colour. A dataset with no checks is not healthy; it is unmeasured, and that is information a consumer needs before they trust it.
- Put the verdict where the data is read — in the catalog entry, in the BI tool's dataset description, in the table's comment — because a consumer will not visit a separate site before running a query (Data Discovery).
- Link every row to its checks, its series and its owner, so the dashboard is the entry point to an investigation rather than the end of one (Lineage Debugging).
What this actually promises
Naming the guarantee you do not have is worth more than naming the one you do — everything downstream inherits the weakest promise in the chain.
- A green row guarantees that the checks configured for that dataset ran and passed. It guarantees nothing about failure classes nobody asserted, which is where most incidents live.
- The freshness column is a genuine measurement and the strongest cell on the screen. It is also the one least related to correctness (Freshness Checks).
- The completeness column is only a guarantee if it is computed against an external reference. Completeness derived from the pipeline's own row counts is circular and guarantees nothing at all (Reconciliation).
- A red row guarantees only that a check disagreed with the data. Whether the check or the data is wrong is a human judgement the dashboard cannot make (Distribution Tests).
Can I trust it?
A green pipeline is evidence that code ran. These four fields are the evidence that the data is right.
- The check on the dashboard is coverage: how many serving datasets have at least one check of each class, and how many have none. That number is usually much worse than the colour distribution suggests.
- Also verify that a broken dataset actually turns the row red end to end, by breaking one in staging. A dashboard nobody has seen fail is a dashboard nobody should trust.
- It misses a dataset that has checks, passes them, and is wrong in a way nothing asserted — which the dashboard is structurally incapable of showing and should therefore say so in its own header (Data Quality).
- The dashboard itself has a freshness, and it is routinely forgotten. A quality dashboard computed hourly can show green for an hour after a dataset broke, and it should display its own last-computed time.
- Freshness and completeness answer questions on different timescales — one about now, one about the last closed period — and putting them in adjacent columns invites the reading that they refer to the same moment. Label the period explicitly.
- A dashboard that only recomputes when the pipeline runs cannot show that the pipeline did not run. The status computation has to be on its own schedule (Orchestration).
- New datasets appear without checks and should appear on the dashboard immediately as unknown. A dashboard that only lists datasets somebody configured hides the growing unmeasured tail (Metadata: Technical, Operational and Business).
- Thresholds must move with the dataset's SLO, and both should live in the same declaration. A dashboard with thresholds configured in its own UI drifts from the commitments it is displaying (Data Contracts).
- When a dataset is deprecated the row must not silently vanish, or consumers still reading it lose their only signal (Deprecation as a Process, Not a Label).
- The dashboard is a read model; recovery is recomputation. What matters is that it recomputes on its own schedule so it can report on a pipeline that is not running (Pipeline Observability).
- During an incident the dashboard's job is to answer "what else is affected", which it can only do if rows are linked to lineage. Otherwise the blast radius is reconstructed by hand (Impact Analysis).
- After an incident, the row that stayed green is the most valuable artefact in the review: it names the check that should exist and does not (Data Incidents).
What can go wrong
- Unknown rendered as green, so unmeasured datasets are indistinguishable from healthy ones.
- A single platform-wide health figure, which is high on every day including the bad ones (The Average Was Fine and Users Were Not).
- A pipeline-shaped view given to consumers who think in datasets.
- Thresholds configured in the dashboard rather than declared with the dataset, so the displayed status and the promised SLO diverge.
- A dashboard that stops updating and displays its last state indefinitely — the mitigation failing in the same silent way as the pipelines it watches.
- Two hundred green rows produced by two hundred vacuous checks, which is worse than an empty dashboard because it is convincing (Data Tests).
- "All green, so the data is good." All green means no check that exists found a problem. Ask what fraction of datasets have any check at all before reading the colours (Data Quality).
- "Completeness is 99.8%, so almost nothing is missing." Ask what it was compared against. If it is a non-null rate, it is a validity measure and says nothing about rows that never arrived (The Dimensions of Data Quality).
- "The dashboard is our monitoring." Nobody watches a dashboard at 3 a.m. Detection is alerting; the dashboard is context and trust signalling (Quality Alerting).
- "Our platform health is 98%." Aggregated health hides the single broken table, which is the only thing anybody wanted to know (The Average Was Fine and Users Were Not).
Operating it
- Coverage: datasets with a freshness check, with a uniqueness check, with a completeness or reconciliation check, and with none. Four counts, tracked over time (The Data Catalog).
- The dashboard's own last-computed timestamp, displayed on the dashboard (Health Checks).
- Distribution of statuses over time, which shows whether the platform is improving or whether the checks have been loosened (Alert Fatigue: The Page Nobody Reads).
- Click-through from the dashboard to a dataset's checks, as a crude but real measure of whether anyone is using it (Data Discovery).
- At tens of datasets the dashboard is a table anyone can read. At hundreds it needs grouping by domain and ownership, and a default view that shows only what is not green (Data Ownership).
- At thousands, per-dataset rows stop being browsable and the useful artefact becomes the coverage report plus a search — the dashboard becomes a query interface rather than a screen (Data Discovery).
- Check count scales faster than dataset count, so the roll-up rule from checks to a dataset status becomes the most consequential piece of logic on the page (Impact Analysis).
- The dashboard is cheap; the checks behind it are not. Cost is dominated by whichever check scans the most, which is normally a completeness or reconciliation query over a closed period (Scan Cost).
- Recomputing every row on a short interval multiplies the cost of every check by the frequency. Freshness can be recomputed constantly because it reads metadata; completeness cannot (Partition Pruning).
- Series cardinality is the hidden driver. Per-dataset, per-check, per-period series are fine; adding a per-category dimension to each multiplies the count and is the usual cause of a metrics store outgrowing its budget (Cardinality: The Label That Took Down Monitoring).
- A single status per dataset is readable and lossy. It compresses several independent questions into one colour, and the compression rule is where the disagreements will happen.
- Showing unknown honestly makes the dashboard look worse on the day it ships, which is exactly why most implementations quietly do not.
- Putting the verdict inside the BI tool reaches consumers where they are and couples the quality system to that tool. The alternative reaches fewer people and stays portable (The Data Catalog).
Platform quality dashboard
Change an input and watch which number moves — and which one does not. Everything here comes from a model in this repository, not from a measurement.
| Dataset | Layer | Owner | Freshness | Completeness | Checks | Status |
|---|---|---|---|---|---|---|
| raw.orders | Raw landing | platform | 12 minsim | 100.0%sim | 6/6 | healthy |
| stg.orders | Staging | platform | 12 minsim | 111.1%sim | 4/6 | degraded |
| fct_orders | Model | analytics-eng | 12 minsim | 100.0%sim | 5/6 | degraded |
| fct_orders_eu | Model | analytics-eng | 12 minsim | 83.6%sim | 3/6 | degraded |
| mart.revenue_daily | Serving | analytics-eng | 1,452 minsim | 0.0%sim | 2/6 | stale |
| mart.orders_by_country | Serving | analytics-eng | 12 minsim | 100.0%sim | 5/6 | degraded |
A healthy run in this model has a freshness lag of 12 minsim, and a run that never published carries the previous period forward — which is what a stale row is: a table with rows in it, rendering confidently.
Where this applies
Almost nothing here is universal. These labels say what each claim is specific to, and where a different engine, format, warehouse or scale would differ.
- GENERALOne row per serving dataset with freshness, completeness and a status is a shape that works regardless of stack, because it is organised around what consumers query rather than around how the platform is built.
- SIMPLIFIEDThe figures in the example dashboard are illustrative values chosen to show three distinct states, not measurements and not recommended thresholds. Real thresholds come from each dataset's declared SLO, and a completeness percentage is only meaningful once you say what it was compared against.
- ORG-SPECIFICWhether the verdict lives in a dedicated quality tool, in the data catalog, or in the BI tool's dataset metadata depends on where consumers actually look, which differs by company. The wrong choice does not produce a wrong dashboard, it produces an unread one.
Where the depth lives
This domain teaches how data moves and how you know it arrived intact. It hands the rest off by name.
- — DevOps / Production Engineering owns service dashboards and the discipline of designing one screen per audience. The failure modes are shared — aggregated health hiding a single broken component, a dashboard that stops updating and keeps rendering — and the fix is the same in both domains.