Cost Engineering
Data platforms get expensive quietly. The drivers — bytes scanned, bytes shuffled, bytes retained, hours held, work repeated — and the design decisions that move each one.
Storage, scans, shuffle, compute hours, network, retention, file count and repeated work — put in the order they actually move the number.
What a query actually has to read, and why column selection and partition pruning are the two cheapest fixes in the entire domain.
Rebuilding history that did not change, refreshing models nobody reads, holding capacity nobody uses, and shuffling data that did not need to move.
Hot to warm to cold to archive to deleted, driven by how the data is actually read — and the retrieval cost that makes archive a trap for anything you might read again.
You cannot manage what you cannot attribute — and in a shared platform every cost belongs to everyone, which means it belongs to nobody.
The most direct trade in the platform: every increment of freshness is bought with compute that runs more often, longer, or continuously. The right freshness is set by the decision the data drives, never by what the stack can achieve.