Metadata, Catalog & Lineage
Data about data, and the graph that connects it. Discovery, ownership, column-level lineage and impact analysis — the difference between a warehouse and a landfill.
Schema, owner, description, freshness, lineage, tags and quality — split by where each comes from, because that is what predicts which of them is still true.
Four questions it must answer, and the honest failure mode: a catalog nobody populates is worse than no catalog, because it looks authoritative.
orders DB to stg_orders to fct_orders to revenue_daily to the executive dashboard — and why that graph is a debugging tool rather than documentation.
orders.amount to revenue to monthly_revenue. Much harder to produce than table-level lineage, and the only granularity that answers the question an incident actually asks.
The same graph read the other way. If I change this column, what breaks — answered before the change rather than discovered afterwards.
Every important dataset has a clear owner. The failure to design against is "nobody knows where this table came from" — and it is an organisational problem with a technical trigger.
How someone finds the right dataset among hundreds. Search over descriptions fails; search over what people actually query works.
Documentation generated from the transformation graph stays true. Documentation written separately does not — and the distinction decides what is worth writing down at all.