Transformation
Cleaning, casting, joining, aggregating and deduplicating — expressed as a dependency graph of tested, documented models rather than a pile of scheduled scripts.
Clean, cast, join, aggregate, deduplicate, enrich, filter, normalize, denormalize — nine operations, each with a way of being wrong that does not raise an error.
The five-line aggregate everyone writes, and the thirty-line one that is still correct after duplicates, refunds, currency and late data exist.
Transformations as version-controlled, tested, documented models whose dependencies are inferred rather than declared — and materialisation as a choice you make on purpose.
raw_orders to stg_orders to int_orders_enriched to fct_orders to customer_metrics — five nodes, four edges, and everything you can ask of a graph you did not have to draw.
Node, edge, no cycles — the whole structure. Why every question a data platform asks about itself turns out to be a standard graph traversal, and why a cycle is almost always a modelling error.
If A feeds C and B feeds C, then A and B can run together and C must wait. What that ordering buys — parallelism, correctness of order, selective rebuild — and the one thing it emphatically does not buy.
Staging renames and types one source. Intermediate joins and reshapes. Marts face the business. The rule that makes it work is that consumers depend only on marts.
Business logic copied into twenty dashboards produces twenty definitions of revenue, all defensible. A metric defined once, with an owner, is the only fix — and it does not fix everything.