The Dimensions of Data Quality
Completeness, accuracy, freshness, uniqueness, validity, consistency — defined precisely, with how each is measured and which one is almost always asserted instead.
Who needs this, what one row is, and why the obvious build breaks
Every lesson starts from the consumer, because designing from the source outward is this domain's characteristic mistake.
When somebody says the data is "good quality", which of six quite different claims are they making, and which of them did anyone actually verify?
Anyone who has to decide whether a dataset is fit for a purpose: an analyst choosing between two tables, a platform team writing an SLA, a model owner deciding whether last quarter is trainable. All of them need a vocabulary in which "the data is fine" decomposes into claims that can individually be true or false.
Every dimension is evaluated at a declared unit and a declared period. Completeness of fct_orders for a closed day is a claim about orders in that day; the same word applied to the table as a whole, with no period, is not a claim at all. Naming the unit and the boundary is half of making these dimensions operational (Grain: What Does One Row Represent?).
Adopt the standard six-word list from a data governance deck, put it on a slide, and score each dataset red/amber/green against it. This is a genuine improvement on having no vocabulary at all, and it is why the list is popular — the failure is not the list, it is stopping there.
The scorecard says accuracy is green. Nobody can say what was compared against what. In practice "accuracy: green" means "no one has complained", which is a statement about attention, not about data.
- The scorecard says accuracy is green. Nobody can say what was compared against what. In practice "accuracy: green" means "no one has complained", which is a statement about attention, not about data.
- Completeness is reported as a percentage of non-null cells. That measures population, not arrival: a table missing an entire day of orders can be 100% non-null in every column it does contain (Missing Rows).
- Freshness is reported for "the warehouse" as an average. Averaging hides the single table that has not updated since Friday, which is the only fact anybody needed (The Average Was Fine and Users Were Not).
- Uniqueness is asserted on a surrogate key generated at load time, which is unique by construction and therefore always passes. The business key — the thing that would reveal a duplicate order — is untested (Surrogate Keys).
- Validity passes because the column is a valid
DECIMAL. It holds a price in a currency the model does not know about, and no type system in existence will object (Semantic Changes). - Consistency is claimed because two dashboards agree. They agree because they read the same model, which is exactly what you would expect if that model is wrong (Two Dashboards, Two Numbers).
What is actually happening
- Completeness asks whether everything that happened is present. It is measurable only against an external count — the source, an event log, or a control total supplied by the producer. Non-null rates measure something else entirely and are frequently substituted for it (Reconciliation).
- Validity asks whether each value conforms to its declared domain: type, range, format, allowed set, referential existence. It is fully measurable from inside the dataset, which is why it is the dimension every tool implements first and the dimension that catches the least (Data Tests).
- Uniqueness asks whether each real-world entity appears once. It is measurable, but only against a key that identifies the entity in the world rather than in your load process — which is a modelling problem before it is a testing one (Deduplication).
- Freshness asks whether the newest data is recent enough for the decision it drives. It is measurable and cheap, and it has two distinct clocks — when the data was loaded, and when the newest event in it happened — that are routinely conflated (Freshness Checks).
- Consistency asks whether the same fact agrees wherever it appears: within the row (a
shipped_atbefore anordered_at), across tables (every order references a customer that exists), and across systems (the warehouse and the payment provider agree on yesterday). Only the third is hard, and only the third is worth much (Relationships, Keys and Constraints). - Accuracy asks whether the value matches reality. There is no way to measure it from inside the data, ever, because reality is not in the table. What is actually measurable is agreement with an independent authority, which is a weaker and much more honest claim.
Six dimensions, six different questions
The value of the list is that it separates claims that people routinely bundle. A dataset can be complete and inaccurate, valid and stale, fresh and duplicated, unique and inconsistent with the system that produced it. Every one of those combinations is common, and each has a different cause and a different fix.
Read the fourth column first. What a passing dimension does *not* mean is more operationally useful than what it does mean, because that is the part people extrapolate from. A green validity score reliably gets read as "the values are right", which is a claim about accuracy that validity is in no position to make.
Note also the third column's asymmetry: three of the six are measurable entirely from inside the dataset, two require a second system, and one is not measurable at all.
Three dimensions measurable from inside, two requiring a second system, one not measurable at all. A quality programme that only implements the first three has implemented the cheap half and reported it as the whole.
| Dimension | The precise question | Measured by | A pass does not mean |
|---|---|---|---|
| Completeness | Is every record that happened present, for a closed period? | Comparing a count from the source or an independent control total with the count in the serving table. Requires a second system. | That the values in the rows are right, or that a still-open period is complete. |
| Validity | Does every value conform to its declared type, range, format, allowed set and references? | Assertions inside the dataset: not-null, range, regex, accepted values, referential existence. | That the value is correct. A well-formed number in the wrong currency is perfectly valid. |
| Uniqueness | Does each real-world entity appear once and only once? | A count of distinct business keys against a count of rows, at a declared grain. | That a duplicate under a new key was caught, or that the key you tested identifies the entity at all. |
| Freshness | Is the newest data recent enough for the decision this drives? | Now minus the maximum event time, and separately now minus the load time. Two clocks, both needed. | That the newly loaded data is different from what was there before, or that any of it is correct. |
| Consistency | Does the same fact agree within a row, across tables, and across systems? | Intra-row predicates, referential-integrity counts, and cross-system aggregate comparison for a closed period. | That two systems agreeing are both right — a shared upstream error agrees with itself. |
| Accuracy | Does the value match what actually happened in the world? | Nothing internal. Only agreement with a named independent authority, which is a strictly weaker claim. | Anything, unless the authority is named. An unnamed accuracy score is an opinion. |
Accuracy is asserted, not measured
Every other dimension has a query behind it. Accuracy does not, because the comparison it needs — value against reality — has no second operand available to a database. Reality is not a table you can join to.
What gets reported as accuracy is therefore always a proxy, and the proxies vary enormously in strength. The weakest is absence of complaints. Stronger is internal plausibility: values in range, no impossible combinations. Stronger still is agreement with an independent system that observed the same events — a payment provider's settlement report, a warehouse management system's stock count, a manual sample audited by a human who knows the domain.
None of these is accuracy. They are agreements, each inheriting the errors of whatever it agreed with. That is not a reason to skip them; it is a reason to name them, because a claim with a named reference has a knowable blind spot and a claim without one has an unknowable one.
The practical rule is short: if you cannot say what the number was compared against, do not report a score for it. "Unverified" is a legitimate and informative value for this dimension, and it is the correct one far more often than dashboards admit.
A dimension score on a scorecard, computed as the proportion of rows passing the platform's validity and plausibility rules, presented next to five other dimensions in the same visual grammar.
An explicit statement that no general accuracy measure exists, plus one named agreement with a stated reference, a stated period boundary and a stated scope — one measure, one authority, one blind spot.
The percentage is a validity measure wearing an accuracy label, and it invites exactly the wrong inference: that 0.6% of values are wrong and the rest are right. The named agreement supports a different and defensible inference — this measure matches an independent observer for closed days, and everything else about this dataset is unverified. The second is less reassuring and much more useful during an incident, because it tells you where to look.
Consistency: three meanings, one word
Consistency is the dimension that causes the most argument, because it names three different checks with very different costs and very different value.
Intra-row consistency is a predicate over columns of the same row: a shipment date not before its order date, a discount not exceeding the total, an end timestamp after a start. These are cheap, catch real upstream bugs, and are almost never written.
Referential consistency is the classic foreign-key question: does every customer_id in the fact table exist in the dimension? Analytical stores frequently do not enforce this, so it becomes a test rather than a constraint, and its violation is the single most common cause of rows quietly disappearing from a dashboard through an inner join (Dimension Tables).
Cross-system consistency asks whether the warehouse agrees with a system it does not control, for a closed period. This is the expensive one, the one with the fewest blind spots, and the one that becomes Reconciliation — which is why that lesson exists separately.
| Check | Expresses | Catches | Still misses |
|---|---|---|---|
Intra-row predicate: shipped_at >= ordered_at where both are present | Two columns of the same row describe a physically possible sequence of events. | Timezone handling applied to one column and not the other; a backfill that populated one field from the wrong source; clock skew at the producer. | Everything about rows where either column is null, which is usually where the bug lives. It also cannot see two timestamps that are consistently ordered and both wrong. |
Referential: every customer_id in fct_orders exists in dim_customers | The dimension covers every key the fact table references, so an inner join loses nothing. | A dimension load that ran late or partially; a new customer created after the dimension snapshot; a key type mismatch that silently fails to match (Surrogate Keys). | A dimension row that exists and is wrong — the join succeeds and attributes the order to the wrong country. Referential existence says nothing about referential correctness. |
| Referential in reverse: dimension keys with no facts | The dimension is not accumulating rows that describe nothing, which usually signals a duplicated or mis-keyed load. | A dimension that doubled after a non-idempotent re-run; keys generated by the load process rather than by the source. | Legitimately unused dimension members — new products, dormant customers — so it is a warn-level signal and never a blocking one (Quality Alerting). |
| Cross-system: net revenue for a closed day, warehouse versus payment provider | Two independent observers of the same events produce the same total. | Missing rows, duplicated rows, a join at the wrong grain, an arithmetic change in the transformation — the entire class that internal checks cannot see. | Anything wrong identically in both systems, anything in an open period, and any column it does not sum. It also fails for legitimate definitional differences, which is why the definition must be agreed before the check is written (Reconciliation). |
Ordered by cost and by coverage, which happen to run in the same direction. The first three are cheap and narrow, the fourth is expensive and broad, and a platform that has only the first three has not tested the failures that matter most.
How to build it
Most important first.
- Name the dimension whenever you make a quality claim, and refuse the ungrammatical version. "The data is good" cannot be verified; "orders for closed days reconcile to the source on count and net amount" can.
- Measure completeness against a count the pipeline did not produce. A completeness figure derived from the same extract that might have dropped the rows is circular (Reconciliation).
- Publish freshness per dataset with both clocks — load time and maximum event time — because a table loaded five minutes ago containing nothing new is a distinct and common failure (Freshness Monitoring).
- Assert uniqueness on the business key, and if there is no business key, say so publicly rather than testing a surrogate and reporting green (Surrogate Keys).
- For accuracy, name the authority explicitly: "matches the payment provider's settlement report for closed days". Then it is a testable claim with a known blind spot instead of an assertion of virtue.
- Record every dimension as a series over time, not a boolean. Drift within tolerance is the earliest available warning and a pass/fail check discards it entirely (Pipeline Metrics).
What this actually promises
Naming the guarantee you do not have is worth more than naming the one you do — everything downstream inherits the weakest promise in the chain.
- Validity and internal consistency can be genuinely guaranteed by construction — a constraint in the warehouse, a contract at the boundary — because both are properties of the copy (Database Constraints).
- Completeness and cross-system consistency can only be *measured*, never guaranteed, and only for periods that are closed. An open period is expected to be incomplete and any check that says otherwise is wrong (Late-Arriving Data).
- Freshness can be committed to as an SLO, which is a promise about the pipeline's behaviour rather than about the data's truth. A perfectly fresh table can be entirely wrong (The Freshness SLO).
- Accuracy is never guaranteed by a data platform. The strongest available statement is agreement with a named external reference, which inherits that reference's own errors.
Can I trust it?
A green pipeline is evidence that code ran. These four fields are the evidence that the data is right.
- The meta-check: for each dimension, can you name the query that produced the score and the reference it compared against? A dimension with a green score and no query behind it is decoration.
- Run that audit once a quarter across the datasets people actually use. It reliably finds accuracy scores backed by nothing and completeness scores backed by non-null rates.
- It misses a dimension that is measured correctly and irrelevantly — validity checks on columns nobody reads produce impeccable evidence about nothing (Dataset Documentation).
- Freshness is the only dimension that is itself a latency, which is why it gets confused with the others. It answers "how old", not "how right", and the two are independent.
- Every other dimension has a freshness *cost*: a completeness check that waits for a period to close cannot report on today, so the strongest checks are structurally the slowest.
- That produces the standing tension in this module — the checks with the least blind spots run last, and the checks that run first have the most. A portfolio needs both and should say which is which (Data Quality).
- Adding a column adds an untested surface in every dimension at once, and the scorecard usually stays green because it averages over columns.
- A renamed column vacates its tests silently. The suite still passes; it is now asserting properties of a column that no consumer reads (Breaking Schema Changes).
- A change in *meaning* moves accuracy and consistency without touching validity, completeness, uniqueness or freshness. Four green dimensions and a wrong number is a normal outcome of a semantic change (Semantic Changes).
- Recovery differs sharply by dimension. Validity failures are usually repairable in place by fixing the cast and recomputing. Completeness failures need the missing rows, which may or may not still exist upstream (Replay from the Log).
- Uniqueness failures need a decision before a repair: which of the duplicates is authoritative, and can the survivors be identified deterministically? Deleting the wrong copy is worse than keeping both (Upserts and Merges).
- Accuracy failures are the ones with no technical recovery. If the value was wrong at the source, the platform faithfully carried a wrong value and the repair is upstream.
What can go wrong
- Substituting a cheap measurable dimension for an expensive important one — non-null rate for completeness is the canonical case.
- Scoring dimensions on a table rather than for a period, which makes every score unfalsifiable.
- Testing uniqueness on a key generated by the pipeline, which is unique by construction.
- Averaging a dimension across datasets so the one broken table disappears into a platform-level green (The Data Quality Dashboard).
- Treating the six dimensions as a checklist to complete rather than as questions to answer, which produces evidence without coverage.
- "Accuracy is one of the six dimensions we monitor." Almost certainly not. Check what it is compared against; the usual answer is nothing.
- "Completeness is the percentage of non-null values." That is a validity measure. Completeness is about rows that never arrived, and non-null rates cannot see them.
- "Consistency means the dashboards match." Internal agreement is the weakest form of consistency. The valuable form is agreement with a system you do not control.
- "Freshness tells us the data is current." It tells you when it was last written. Whether what was written is a new period or a re-write of the old one is a separate question (Stale Dashboards).
Operating it
- One time series per dataset per dimension, with the reference named in the metric's labels so it is obvious what a green means (Label Sets That Survive a Year).
- The gap between load-time freshness and event-time freshness, which separates "the pipeline ran" from "there was new data" (Freshness Monitoring).
- Referential-integrity violation counts against dimension tables, which is the cheapest cross-table consistency signal there is (Relationships, Keys and Constraints).
- At 10x datasets, per-dimension scoring by hand collapses and the dimensions have to be derived from declared metadata: grain, keys, nullability, expected freshness (Data Contracts).
- At 100x rows, completeness and uniqueness stop being exhaustive queries and become sampled or sketch-based, which changes their guarantee from "no duplicates" to "no duplicates found in the sample".
- Cross-system consistency scales worst of all, because every additional source is another pair of definitions that must be reconciled by a human before any query can be written.
- Validity and uniqueness are cheap: they scan one partition and read a few columns. They are also the dimensions with the smallest blind-spot coverage, which is why cost and value point in opposite directions here.
- Completeness and cross-system consistency are expensive because they require reading a second system and agreeing on a boundary. That expense buys the coverage nothing else provides (Reconciliation).
- The cheapest dimension to measure well is freshness — two timestamps and a subtraction — and it remains one of the most under-implemented (Freshness Checks).
- A precise vocabulary makes disagreement visible, which is uncomfortable. "Accuracy: unverified" is more useful and less popular than "accuracy: green".
- Measuring completeness properly requires source access, which crosses a team and often a security boundary. The honest alternative is to say the dimension is unmeasured rather than to substitute a proxy (Data Access Control).
- Recording every dimension as a series costs storage and cardinality in the metrics system, and pays for itself the first time you need to know when a drift started (Cardinality: The Label That Took Down Monitoring).
Where this applies
Almost nothing here is universal. These labels say what each claim is specific to, and where a different engine, format, warehouse or scale would differ.
- GENERALThe six dimensions and their measurability are properties of the questions, not of any tool. Different frameworks split them differently — some separate timeliness from currency, some fold uniqueness into validity — but no framework can make accuracy measurable from inside the dataset.
- ORG-SPECIFICWhich dimensions carry a formal commitment is an organisational choice. A regulated finance environment usually commits to completeness and accuracy against a named external report; a product analytics team usually commits to freshness and shape and explicitly does not commit to per-row accuracy.
Where the depth lives
This domain teaches how data moves and how you know it arrived intact. It hands the rest off by name.
- — Distributed Systems owns a different sense of the word "consistency" — the agreement between replicas of a single system under concurrent writes. That is not this dimension, and conflating the two is a persistent source of confusion in design reviews.