Source of Truth
One authoritative system per business concept, everything else explicitly a copy — and the discipline that follows once you have said which is which.
Who needs this, what one row is, and why the obvious build breaks
Every lesson starts from the consumer, because designing from the source outward is this domain's characteristic mistake.
customer_country exists in the CRM, on the orders table and in the customer dimension, and all three disagree. Which one is right, and how would anybody know?
The analyst whose regional revenue split depends on which of the three they happened to join to; the finance controller who reports one of those splits externally; and the engineer who has to decide, during an incident, whether a mismatch is a bug or the expected behaviour of three systems that were never supposed to agree.
The unit is the business concept, not the table and not the column. "Customer country" is one concept that happens to be materialised in three places; declaring truth per column produces a policy nobody can follow, and declaring it per system produces one that is obviously false the moment two systems both hold customers.
Whichever system you happen to be querying is the truth for that query. Everyone knows roughly where things live, the numbers are usually close, and nobody has ever had to write it down.
Two dashboards split revenue by country differently because one joined to the CRM extract and one used the value denormalised onto the order. Both are defensible, both have been shown to executives, and neither author knew the other existed (Two Dashboards, Two Numbers).
- Two dashboards split revenue by country differently because one joined to the CRM extract and one used the value denormalised onto the order. Both are defensible, both have been shown to executives, and neither author knew the other existed (Two Dashboards, Two Numbers).
- A customer moves country. The CRM updates, the orders placed last year keep the value they were written with, and the dimension takes whichever arrived last. Every historical split now depends on an ingestion race (Slowly Changing Dimensions).
- A correction is applied in the warehouse because that is where the wrong number was noticed. The next load from the CRM overwrites it, the fix disappears, and the incident is reopened a week later (Full Refresh vs Incremental).
- A deletion request is satisfied in the CRM. The value survives in the orders table, the dimension, a materialised mart and an extract in someone's notebook, because nothing recorded that those were copies (Deletion Requests).
- Someone proposes reconciling the three systems automatically, and the proposal stalls because nobody can say which direction the reconciliation should run.
What is actually happening
- Every copy of a value has a write path and the paths differ. The CRM is written by a human sales operator, at an arbitrary time, with no versioning. The order row is written once at checkout, from whatever the application knew then, and never updated. The dimension is written by a pipeline, on a schedule, from one of the other two.
- Because the write paths differ, the values diverge *correctly*. The order's country is not a stale copy of the CRM's — it is a different fact, "the country at time of order", which the CRM does not store at all. Most "data quality problems" of this shape are two different facts sharing one name (Semantic Changes).
- A source of truth is therefore a declaration about which write path is authoritative for which question. It is not a statement that other copies are wrong; it is a statement that when they disagree, one of them is the one to fix and the others are the ones to refresh.
- The declaration only has force if copies are marked as copies. An unmarked copy accumulates its own edits — a correction here, a manual fix there — and becomes a second source of truth by drift rather than by decision (Data Ownership).
- Warehouses make this worse in a specific way: they are the one place all the copies meet, so they are where the disagreement becomes visible, and they are the most tempting place to "fix" it. A fix applied downstream of the authoritative system is overwritten by the next load, or worse, is not (Reconciliation).
Three systems, three answers, and all of them defensible
Start with the concrete case, because the abstract version is easy to nod along to and impossible to act on. customer_country exists in the CRM, on each order row, and in the customer dimension. A query joining any two of them can produce a different regional revenue split, and all three values were written by a process that was working correctly.
The table below is the whole lesson in one view. Read the "when written" column: three different write paths, three different times, three different notions of what the value means. The divergence is not a bug in any of them — it is the predictable consequence of storing what looks like one fact in three places with three lifecycles.
The moment you see it this way, the question changes. It stops being "which value is correct" — a question with no answer — and becomes "which fact is this query about", which has a precise one. Regional revenue for a historical period almost certainly wants the order-time value; a mailing list wants the current one (Event vs Snapshot Modeling).
| Where it lives | What the value actually means | When it is written | Authoritative for | How it goes wrong |
|---|---|---|---|---|
CRM account.country | The customer's country *now*, as last known to a human operator. | Whenever a sales or support person edits the record. No versioning, no history. | Anything about the present: who to contact, which team owns them, which tax regime applies today. | It is edited without an audit trail, so a wrong value has no "before", and a corrected value silently changes every downstream copy on the next load. |
orders.shipping_country | The country the order was shipped to, as known at checkout. | Once, at checkout, from whatever the application knew then. Never updated. | Anything about that order: fulfilment, tax on that transaction, historical regional revenue. | It is treated as "the customer's country" and joined as if it were current, which mixes a point-in-time fact into a present-tense question. |
dim_customer.country | Whichever of the two the pipeline last loaded, flattened to one value per customer. | On the pipeline's schedule, from one declared upstream — if anyone declared one. | Nothing by itself. It is a copy, and it is authoritative only if the model deliberately made it a versioned record (SCD Type 2 in Practice). | It gets corrected in place during incidents, acquiring a write path of its own, and becomes an undeclared third source of truth. |
1-- Do not ask which is right. Ask how many disagree, and in which direction.2WITH crm AS (3 SELECT customer_id, country AS crm_country FROM stg_crm_accounts4), ord AS (5 SELECT customer_id,6 max_by(shipping_country, ordered_at) AS latest_order_country7 FROM fct_orders GROUP BY customer_id8), dim AS (9 SELECT customer_id, country AS dim_country FROM dim_customer10)11SELECT12 count(*) AS customers,13 count(*) FILTER (WHERE crm_country <> dim_country) AS crm_vs_dim,14 count(*) FILTER (WHERE crm_country <> latest_order_country) AS crm_vs_order,15 count(*) FILTER (WHERE dim_country IS NULL) AS missing_in_dim16FROM crm17 LEFT JOIN ord USING (customer_id)18 LEFT JOIN dim USING (customer_id);Run it on a schedule, not during the argument. The level tells you how contested the concept is; the slope tells you whether something broke last night. crm_vs_order is expected to be non-zero forever — those are two different facts — and a design that drives it to zero has destroyed information.
Choosing which system is authoritative
There is no general answer, and the temptation is to pick the system that is easiest to query — which is almost always the wrong one, because ease of querying is a property of the copy, not of the write path.
The criteria below are ordered roughly by how often they are decisive. Accountability first: the authoritative system should be the one whose users are answerable for the value being right. If nobody in the CRM is accountable for country being accurate, declaring the CRM authoritative just relocates the problem.
Note that the last option — no single authority, because they are genuinely different facts — is not a failure to decide. It is often the correct answer, and it obliges you to name and model both facts rather than to reconcile them into one (Dimension Tables).
Where is this fact created and edited by whoever is accountable for it being right?
when A person whose job includes being right about this value edits it there — a CRM for account attributes, an HR system for employment, a billing system for plan.
cost Downstream fixes become upstream requests, which is slower and correct. You also inherit that system's weaknesses: usually no history and no audit trail, so "what was it in March" may be unanswerable (Slowly Changing Dimensions).
when The question is about a specific event and the value was captured at that moment — shipping country on an order, price at time of sale, tier at time of signup.
cost You must resist joining it as though it were current state. It answers point-in-time questions precisely and present-tense questions wrongly (Fact Tables).
when Every change to the concept is emitted and retained, so any past state can be reconstructed by replay.
cost Reconstruction is work, retention is a recovery-window decision rather than a cost decision, and consumers need a materialised view to query at all (Replay from the Log).
when The concept is *created* by the transformation — a derived segment, a computed lifetime value, a metric definition. Nothing upstream holds it.
cost The model now has upstream obligations it did not have: an owner, a contract, a versioning story, and a definition reviewable by the business (The Metrics Layer).
when The values disagree because they answer different questions, and reconciling them would destroy information.
cost Two names, two columns, two definitions and one explanation you will give repeatedly. Cheaper than the alternative, which is a reconciliation that quietly picks a winner (Event vs Snapshot Modeling).
What the declaration obliges you to do
A source-of-truth declaration is not a label; it is a set of ongoing obligations, and a declaration without them is worse than nothing because it creates confidence without evidence.
The chain below is what the obligations attach to. Every node downstream of the authoritative system is a copy, every copy has exactly one legitimate way to change — refresh from upstream — and every node has a specific way it can corrupt the concept if that rule is broken.
The obligation people skip is the last one: maintaining the list of copies. Copies are created constantly and cheaply — a mart, an extract, a dashboard cache, a training set, a notebook. Each is legitimate; each inherits the concept's classification and retention; and none of them is on anyone's list unless lineage is generated rather than remembered (Data Lineage).
- CRM `account.country` (declared authoritative)
holds The current value, as edited by the accountable human. No history.
could corrupt An operator edit with no audit trail; a bulk import that overwrote curated values; a merge of two accounts that picked one arbitrarily.
↑ reads from - Ingestion extract
holds A point-in-time capture of the authoritative value.
could corrupt An incremental predicate that misses edits — the CRM updates a row without touching the column the watermark reads (Incremental Extraction).
↑ reads from - `stg_crm_accounts`
holds One row per account, cleaned and typed, still recognisably the source shape.
could corrupt A cast or a trim that normalises two distinct values into one; a deduplication that picks the wrong survivor.
↑ reads from - `dim_customer`
holds One row per customer with the current value, or one row per customer-version if it is an SCD2.
could corrupt A manual correction applied here during an incident, which either vanishes on the next load or survives forever with no explanation (SCD Type 2 in Practice).
↑ reads from - `fct_orders.shipping_country`
holds A different fact: the country at order time, not a copy at all.
could corrupt A well-meaning backfill that "corrects" historical orders to match the current CRM value, destroying the point-in-time record permanently.
↑ reads from - `revenue_by_region` mart
holds Pre-aggregated revenue keyed by whichever country column the model chose.
could corrupt Choosing the dimension's current value for a historical split, so last year's regional revenue changes whenever a customer moves.
↑ reads from - Extracts, notebooks, training sets
holds Whatever the value was on the day someone exported it.
could corrupt Everything downstream of them, invisibly and indefinitely — these are copies with no refresh path and no entry in the catalog (Data Discovery).
One authority, six copies, and one node that is not a copy at all. The fct_orders row is the one people get wrong in both directions: they treat it as a stale copy to be reconciled, or they treat the dimension as a valid substitute for it.
Catalog and lineage products differ substantially in whether they capture column-level edges and whether they see BI-layer copies at all. Evaluate any of them by asking whether it would have listed the last row of this table, since that is the row that determines whether a deletion request can be answered honestly.
How to build it
Most important first.
- Name one authoritative system per concept and write it down where an engineer will find it — the catalog entry, not a wiki page from 2023 (The Data Catalog).
- Prefer the system where the fact is created and edited by whoever is accountable for it. Truth follows the write path, not the query path, and certainly not the system that is easiest to query (Data Ownership).
- When two systems hold what looks like the same field, check whether they hold the same *fact*. "Country now" and "country at order time" are two concepts and both may be authoritative — for different questions (Event vs Snapshot Modeling).
- Mark every other copy explicitly, in the catalog and ideally in the column name or description. A copy that is labelled will be refreshed; a copy that is not will be edited (Dataset Documentation).
- Never repair data downstream of its source. Fix it at the authoritative system and let the correction propagate, and if that is impossible, record the override as data — a mapping table with a reason and an owner — rather than as an edit (Data Quality).
- Reconcile in the declared direction, on a schedule, and alert on divergence. A declaration nobody measures decays into folklore within two quarters (Reconciliation).
What this actually promises
Naming the guarantee you do not have is worth more than naming the one you do — everything downstream inherits the weakest promise in the chain.
- Declaring a source of truth guarantees nothing about the values agreeing. It guarantees that when they disagree there is an unambiguous answer to "which one do we change", which is the only guarantee available here.
- It does not guarantee the authoritative value is correct. The CRM can be wrong; being authoritative means being the place to fix it, not being right (Data Quality).
- The propagation from the authoritative system to its copies inherits whatever the pipeline promises — usually at-least-once with a delay, never instantaneous consistency. Consumers reading a copy are reading a past state of the truth (Eventual Consistency in Practice).
- Nothing guarantees a new copy will not appear. Every extract, every mart, every notebook cache is a copy, and the set of copies grows unless somebody actively maintains the list (Impact Analysis).
Can I trust it?
A green pipeline is evidence that code ran. These four fields are the evidence that the data is right.
- The check is a scheduled three-way comparison: count rows where the authoritative value differs from each copy, per concept, and trend that count. A one-off comparison during an incident tells you nothing about whether the gap is new.
- Alert on the *rate of change* of divergence as well as its level. A steady small gap is usually a legitimate semantic difference; a gap that jumped overnight is an incident (Volume Anomalies).
- It misses the case where the authoritative system itself is wrong — every copy agrees, the reconciliation is clean, and the number is false. It also misses concepts nobody declared, which is where most of the real divergence lives.
- A copy is always behind its source by the interval of the pipeline that maintains it, plus every hop in between. Publishing that lag per copy is what lets a consumer decide whether the copy is good enough for their question (Freshness Monitoring).
- Freshness expectations differ per concept in ways that surprise people. A customer's country can be a day stale with no consequence; their consent flag cannot be stale at all, because acting on a stale one is a compliance event (Data Access Control).
- The dangerous case is a copy that is fresh for most rows and badly stale for a few — an incremental refresh whose predicate misses updates. It looks current in aggregate and is wrong exactly where somebody changed something (Incremental Extraction).
- Authority moves. A concept owned by a spreadsheet becomes owned by a CRM, then by a billing system, and each migration leaves a copy behind that still gets written to by somebody's habit (Impact Analysis).
- When authority moves, the old system must lose its write path, not merely its designation. A system that is "no longer the source of truth" but still accepts edits is a second source of truth with a disclaimer.
- Semantic evolution is the harder case: the authoritative system starts recording billing country where it used to record shipping country. The schema does not change, every copy propagates the new meaning, and every historical comparison silently breaks (Semantic Changes).
- Recovery is always a re-propagation from the authoritative system, never a repair of the copy. If a copy has drifted, refresh it; if the refresh cannot be trusted to fix it, the copy has its own write path and that is the real bug (Full Refresh vs Incremental).
- When the authoritative system was wrong, correction propagates forward and history has to be decided explicitly: restate the past, or keep the value as recorded and note the correction. Both are defensible; doing it silently is not (Planning a Backfill).
- Manual overrides applied during an incident must be recorded as data with an owner and an expiry, or they become permanent unexplained differences that the next engineer will treat as a bug (Data Quality).
What can go wrong
- A copy acquires its own write path — a manual correction, a one-off script, a "temporary" mapping table — and becomes an undeclared second source of truth.
- The authoritative system is chosen for convenience: the warehouse, because it is easiest to query, even though nobody edits customers there (Data Platform Anti-Patterns).
- Two facts share a name, the disagreement is treated as a defect, and a reconciliation is built that destroys the distinction between "country now" and "country at order time".
- The declaration exists and nothing measures it, so it decays into folklore and the next engineer rediscovers the problem from scratch.
- A concept has no authoritative system at all — it is computed differently in three models — and the resolution is an organisational argument nobody has authority to settle (Who Owns Data Quality).
- "The warehouse is the source of truth." The warehouse is where copies meet. It is authoritative only for concepts that are *created* there, which is a short list — usually derived metrics and nothing else (The Metrics Layer).
- "If two systems disagree, one is broken." Very often both are correct and they hold different facts. Diagnose the semantics before diagnosing the pipeline (Semantic Changes).
- "Source of truth means single copy." It means single *authority*. Copies are how analytics works at all; the discipline is labelling them, not eliminating them.
- "We can fix it in the dimension." You can, and the next load will either overwrite the fix or preserve it forever without explanation. Both outcomes are worse than fixing it upstream (Data Quality).
- The authoritative system is where a deletion or rectification request must be satisfied first, and the copy list is what determines whether it was satisfied at all. A concept with no declared authority has no defensible deletion process (Deletion Requests).
- Classification follows the concept rather than the column: if a field is personal data in the authoritative system, every copy inherits that classification, including the ones in marts and extracts that were created before anyone thought about it (Data Classification).
- Access control diverges between a system and its copies by default. The CRM restricts who can see a customer's country; the warehouse copy usually does not, and that gap is created the moment the copy is made (Row and Column Security).
Operating it
- Divergence count per concept per copy, as a time series. Level tells you how bad it is; slope tells you whether it is an incident (The Data Quality Dashboard).
- The catalog field that names the authoritative system, and the proportion of concepts that have one filled in. An empty field is the observation (The Data Catalog).
- Write paths per table: which jobs, services and humans have written to a copy in the last month. A copy with more than one writer has stopped being a copy (Data Lineage).
- At small scale one system usually is the truth for everything, and the lesson is trivial. It stops being trivial the moment a second system can create a customer.
- At 10x concepts the declaration must be recorded in a catalog rather than remembered, because nobody can hold fifty concept-to-system mappings and everyone will guess (Data Discovery).
- At large scale the pattern becomes federated: each domain team is authoritative for its own concepts and publishes them as products with contracts. The number of concepts is what forces this, not the volume of rows (Data Mesh).
- Reconciliation costs scans, once per concept per run, and the scan is usually cheap because it touches two columns and a key (Scan Cost).
- The real cost is coordination: deciding authority means someone loses the right to edit their copy, and that conversation is slower than any query (Data Ownership).
- Not declaring it costs more and later — duplicated corrections, restated reports, and the meetings that discover two teams have been reporting different regional splits for a year.
- Declaring authority makes disagreements resolvable and makes some teams slower — the team that can no longer fix a value in their own copy now files a request against the owning system.
- Reconciliation gives you evidence and costs a scheduled scan plus an alert that will sometimes fire on a legitimate semantic difference. Tuning it towards silence is how it becomes useless.
- Keeping two facts distinct — country now and country at order time — is correct and doubles the modelling work, the documentation and the number of columns an analyst must choose between (Dimension Tables).
Dataset review questions
This lesson uses the shared review exercise.
Where this applies
Almost nothing here is universal. These labels say what each claim is specific to, and where a different engine, format, warehouse or scale would differ.
- GENERALThe need for one authority per concept holds anywhere a value exists in more than one system, including a company with one database and one spreadsheet. What varies is how many concepts are contested, which is a function of how many systems can create the same entity.
- ORG-SPECIFICDeclaring authority takes editing rights away from a team, so it is an organisational negotiation rather than a technical change. In a single-team company it costs one sentence; across departments with separate budgets it can take a quarter, and it usually needs one visible incident behind it.
- SOURCE-SPECIFICA SaaS source you do not control cannot always be made authoritative in practice — you may have no way to write corrections back, no history, and an API that reflects current state only. That changes the design from "fix it upstream" to "record the override as data", which is a different and worse position.
Where the depth lives
This domain teaches how data moves and how you know it arrived intact. It hands the rest off by name.
- — Distributed Systems owns why two systems holding the same value cannot be kept identical without giving something up, and what the available trade-offs are. This lesson takes that as given and asks the organisational question it forces: which copy is the one to fix.
- — DevOps / Production Engineering owns the change-management side — how a manual override applied during an incident is recorded, reviewed and expired rather than becoming a permanent unexplained difference.