Data Platform Anti-Patterns
The six failures that are organisational rather than technical. None of them is visible in a query plan, all of them are cheap in a small company, and each is what a large one means when it says the data cannot be trusted.
Who needs this, what one row is, and why the obvious build breaks
Every lesson starts from the consumer, because designing from the source outward is this domain's characteristic mistake.
Nothing in the platform is technically wrong, every pipeline works, and nobody trusts the data. What is the organisation doing that no engineering fix will reach?
Everyone who has to use data produced by a team they do not sit with: an analyst hunting for the right table, an engineer trying to find who owns a schema, a finance team asking what the platform costs and who spent it. The technical layer serves them fine; the coordination layer does not exist.
The unit here is the team, not the dataset or the decision. Every entry below is a property of how work is divided between groups, and each becomes visible only when the number of independent groups crosses a threshold that nobody announces (Data Platform Engineering).
Let each team solve its own data problems. It is the fastest way to unblock people, it needs no central group, it avoids the queue that a platform team inevitably becomes, and it respects the teams' own knowledge of their sources. For two or three teams it is not merely acceptable — it is faster and better than any alternative, and centralising early is its own failure mode.
Six teams have each built an ingestion path for a source, and four of those sources are the same source. When it changes, four pipelines break in four different ways and three teams find out from a consumer (Data Ingestion).
- Six teams have each built an ingestion path for a source, and four of those sources are the same source. When it changes, four pipelines break in four different ways and three teams find out from a consumer (Data Ingestion).
- Nobody can find a dataset without asking a person. Discovery happens in chat, so the answer depends on who is online, and three teams end up building the same table because none knew the others existed (Data Discovery).
- A table breaks and the investigation stalls on the question of who owns it. The team that wrote it has reorganised, the team that reads it assumed the first team maintained it, and the platform team is told to fix a dataset it has never seen (Data Ownership).
- Each team has its own idea of what a tested dataset means. One asserts uniqueness and freshness; another has no tests and excellent documentation; a third has forty tests that have been failing for months and are muted (Who Owns Data Quality).
- A transformation is deployed by someone pasting SQL into a console. It works, it is not in version control, and six months later nobody can say what changed on the day the metric moved (The Transformation DAG).
- The platform bill grows and no line of it can be attributed. Every team believes their usage is small, all of them are right about their own share, and nobody owns the total (Cost Attribution).
What is actually happening
- These are coordination costs, and coordination cost grows with the number of pairs of teams rather than with the number of teams. Two teams have one relationship to maintain; eight have twenty-eight. That is why a platform that worked perfectly with three groups becomes unmanageable with eight without anyone doing anything differently.
- Duplication of ingestion is the clearest case. The work is proportional to teams times sources, and only a small part of each connector is team-specific — the schedule and the destination. The rest is the source's pagination, its rate limits, its schema drift and its failure behaviour, which is identical for everybody who reads it (Ingestion Sources).
- A catalog is not documentation; it is the index that makes a search possible at all. Without it, discovery is O(people you know) and the platform effectively has as many datasets as you can remember (The Data Catalog).
- Ownership is the load-bearing one, because every other item degrades to it. Quality standards without an owner are advice; a catalog without owners is a phone book with no numbers; cost visibility without owners is a report nobody acts on (Data Ownership).
- Manual deployment is not primarily a speed problem. It removes the artefact that makes an incident explicable — a versioned diff with an author, a review and a time — and it means the running transformation and the repository can differ without anything detecting it (dbt Concepts).
- Cost invisibility is a commons problem with a specific mechanism: shared compute makes every query's cost everyone's and therefore nobody's, and no individual query is ever the problem. Attribution is what converts a shared bill into a set of decisions somebody can make (Cost Attribution, What Actually Drives Data Platform Cost).
Six ways a platform becomes a landfill
None of the six below is a mistake in the sense of being wrong when it was made. Each is what happens when the thing that used to make coordination unnecessary — a small number of people who all knew each other — stops being true, and nothing replaced it. There is no day on which the decision was made, which is why there is no day on which anyone reviewed it.
The third column is the one to read carefully. These do not become expensive gradually and proportionally; they become expensive when a threshold in team count is crossed, and then they become expensive very quickly, because coordination cost grows with pairs of teams. That is the specific reason a platform can feel fine for two years and unmanageable within one quarter (Data Platform Engineering).
The fourth column matters because these are diagnosed by symptom rather than by audit. Nobody notices that there is no catalog; they notice that finding a table takes a day. Nobody notices the absence of ownership; they notice an incident that spends its first two hours establishing who to talk to (Debugging a Data Incident).
| Platform anti-pattern | Why it was tempting | What it costs once team count crosses the threshold | The symptom you will actually see first |
|---|---|---|---|
| Every team builds its own ingestion | It unblocks people immediately, needs no central group and no queue, and respects that the team knows its own source best. With two teams it is unambiguously faster (Data Ingestion). | Work proportional to teams times sources, several copies of the same source with different schedules and different bugs, and an upstream change that breaks four pipelines in four ways on four different days (Ingestion Sources). | Two dashboards disagree, and the investigation discovers that they read two different copies of the same source with different completeness (Source of Truth). |
| No shared catalog | Everyone already knows where the data is, and a catalog is a product to buy, populate and maintain for a problem that has not appeared yet. For a while that is simply true (The Data Catalog). | Discovery becomes proportional to who you know. Datasets are rebuilt because nobody found the existing one, and the platform grows tables monotonically because nothing can establish that a table is unused (Data Discovery). | A new analyst takes weeks to become productive, and the onboarding fix proposed is a wiki page — which is the same anti-pattern with more effort (Dataset Documentation). |
| No dataset ownership | Ownership feels like bureaucracy when everyone is helpful, and naming an owner for a table three people maintain informally seems like ceremony (Data Ownership). | Every incident begins with an unowned question, every deprecation stalls, every quality standard is advice, and reorganisations silently orphan datasets that consumers still depend on (Who Owns Data Quality). | An incident whose first two hours are spent finding somebody who will accept the ticket, while the wrong number stays on the dashboard. |
| No quality standards | Each team knows what its own data needs, and imposing a uniform standard would slow the teams that are already careful. Local judgement is genuinely better than a bad global rule (Data Quality). | A consumer cannot tell a tested dataset from an untested one, so they either trust everything or nothing — and both are wrong. Tests that exist are muted rather than fixed, because nobody owns the failure (Data Tests). | Consumers build their own defensive checks on top of your datasets, which is a precise measurement of how much they trust you (The Data Quality Dashboard). |
| Manual pipeline deployment | Pasting the corrected SQL into a console fixes production in a minute, at 2 a.m., when the alternative is a pipeline that takes twenty. That is a good trade exactly once (The Transformation DAG). | The running logic and the repository diverge with nothing detecting it. Incidents lose their most useful artefact — a reviewed diff with an author and a timestamp — and the question of what changed becomes unanswerable (Data Incidents). | A metric moves and there is no deploy to correlate it with, so the investigation searches the whole platform instead of one diff (Lineage Debugging). |
| No cost visibility | The platform bill is one line, it was small, and attributing it needs tagging, query labels and a report nobody has asked for. Optimising an unfelt cost is a poor use of a quarter (What Actually Drives Data Platform Cost). | A shared bill makes every cost everyone's and therefore nobody's. No individual query is ever the problem, so the total grows without any decision, and the eventual response is a blunt cut rather than a fix (Cost Attribution). | The bill doubles in a quarter with no change in headcount and nobody can say which driver moved (Scan Cost, Compute Waste). |
Read the second column as sympathy and the third as arithmetic. Every one of these is correct at small team counts, and the mistake is never the original decision — it is the absence of anything that revisits it when the count changes.
The duplication nobody decided
Take the first entry seriously for a moment, because it is the one with the clearest arithmetic. Four teams need data from the payment provider. Each writes a connector. Each connector has to handle the same pagination, the same rate limits, the same at-least-once webhook redelivery, the same schema drift, and the same awkward definition of a transaction. Only two things differ between them: the schedule and where the output lands (Data Ingestion).
The result is not merely four times the work. It is four *different* answers to every hard question the source poses, so the four copies diverge — one deduplicates on the provider's event id, another on its own ingestion id, a third not at all — and consumers reading different copies get different totals. What began as duplicated effort ends as a correctness problem that looks technical and is not (Deduplication).
The alternative is not a central team that owns everyone's pipelines, which recreates the queue. It is one supported ingestion for each source, owned by whoever knows the source, landing raw where anyone can read it — with each consuming team still owning its own models on top. The shared part is the part that is identical for everybody; the divergent part stays where the knowledge is (The Raw Landing Zone, Data Products).
Each team builds and operates its own extraction for every source it needs. Nobody is blocked, nothing needs a central decision, and each pipeline is shaped exactly to the team that wrote it.
Each source is read once by a supported ingestion with a named owning team, landing raw exactly as received where every team can read it. Deduplication, schema drift and failure recovery are solved once, at the boundary. Each consuming team still writes and owns its own models on top of raw, on its own schedule, with its own definitions.
The hard parts of an ingestion are properties of the *source* — its pagination, its delivery semantics, its schema drift, its idea of a transaction — and are therefore identical for every consumer of it. The parts that genuinely differ per team are the models, and those stay decentralised. Splitting the work along that line removes the duplication without creating a central queue, which is the failure mode of the obvious alternative (The Self-Service Data Platform).
The paved road is a product, not a policy
The fix for the remaining entries is one mechanism rather than four: a supported path through the platform that is easier than not using it. Standards that are harder than the workaround produce documented non-compliance, and every organisation that has tried to fix this with a policy has learned that in about a quarter (The Self-Service Data Platform).
The road below is described as a pipeline of stages, each with what it does, what it guarantees to the team using it, and how it fails. The guarantee column is the product: a team should be able to say precisely what they get by staying on the road, and a platform group should be able to say precisely what they are on the hook for. Vague mutual expectations are how a platform team becomes a queue with a roadmap (Data Platform Engineering).
Note what the road deliberately does not guarantee. It does not promise that a dataset is correct — the producing team owns that, and no central group can own it — and it does not promise a response time on someone else's data. Being explicit about the boundary is what makes the promise credible (Who Owns Data Quality).
- 1Register
Declare a dataset: name, owning team, declared grain, classification, freshness target, retention.
guarantees The dataset is discoverable by every team from the moment it exists, and has an addressee during an incident.
fails by Being a form rather than a step in the build, at which point registration drifts from reality and the catalog becomes fiction (The Data Catalog).
- 2Ingest
Read a source once, through a supported connector with a named owner, landing raw exactly as received.
guarantees Delivery semantics, schema-drift behaviour and failure recovery are solved once per source rather than once per team (The Raw Landing Zone).
fails by Not covering an awkward source, so a team builds a shadow connector — and the shadow is invisible to lineage, cost and governance at once.
- 3Contract
Pin the schema the producer agreed to, and check arriving batches against it at the boundary.
guarantees A shape change is rejected loudly rather than absorbed silently. It does not guarantee that values are right (Contract Enforcement).
fails by Being enforced only where the producer cooperated, which is exactly where it was least needed (Data Contracts).
- 4Build
Transform in version-controlled, reviewed, tested models with a dependency graph the tool derives (The Transformation DAG).
guarantees Every running transformation corresponds to a reviewed commit, and lineage is generated rather than maintained (Data Lineage).
fails by Allowing a console edit as an emergency escape hatch that is never reconciled back, after which the repository and production differ silently.
- 5Test and publish
Run the dataset's declared assertions, then publish atomically or not at all.
guarantees Consumers never observe a half-written dataset, and a failing assertion has an agreed consequence decided by the owner (Atomic Publish).
fails by Assertions that were added centrally, fail, and get muted rather than fixed — leaving coverage that reports a standard nobody enforces (Alert Fatigue: The Page Nobody Reads).
- 6Observe and attribute
Publish freshness against target, volume, assertion outcomes and cost per dataset and per consuming team.
guarantees Every dataset has a public health record and a cost line its owner can see (The Data Quality Dashboard, Cost Attribution).
fails by Being used to rank teams rather than to locate drivers, after which the numbers get disputed instead of acted on.
Every stage is optional individually and the road only works as a whole: a team that registers but deploys by hand, or builds properly but ingests through a shadow connector, gets none of the guarantees and appears in every coverage report as compliant.
Who owns what, and the honest options
Ownership is the entry the other five depend on, and it is the one most often deferred because it is a decision about people rather than about systems. The question is not who *does* the work — that varies — but who is accountable when a dataset is wrong, and who a consumer can reach (Data Ownership).
The options below are not a ladder from bad to good. A single embedded data engineer inside one product team is genuinely the right answer for a company with three teams, and a federated model with a platform group is genuinely wrong for it. What decides between them is team count, how much domain knowledge a dataset requires, and whether the organisation can actually staff domain teams with the capability the model assumes (Data Mesh).
Two cautions that apply to every option. Assign ownership to a team with a rota, not to a person, or the next reorganisation invalidates the whole map. And separate the two obligations that get bundled: the producing team owns whether a field is correct, while a platform group can own whether it is monitored, catalogued, deployed and attributed. Bundling them is how the data team ends up accountable for data it has never seen (Who Owns Data Quality).
How many independent teams produce data, how much domain knowledge does a dataset require to be correct, and can you actually staff the model you are choosing?
when Few producing teams, a small number of sources, and domain knowledge that one group can genuinely hold. Most companies, for longer than they expect (The Central Warehouse).
cost Becomes a queue as team count grows, and the group ends up accountable for the correctness of fields it does not produce and cannot judge. Scales to about the point where you stop knowing every dataset by name.
when Domain knowledge dominates and each team's data problems are genuinely different, but there are not yet enough teams for duplication to hurt.
cost Every team solves the same source-level problems independently, and practice diverges immediately. This is the structure that produces the duplication in the previous section (Data Ingestion).
when Enough teams that duplication is expensive, and enough capability in the domain teams to own their own models against a standard (Data Products).
cost The platform group must build a road that is genuinely easier than the workaround, and must keep migrating consumers as it evolves. This is a permanent product commitment, not a project (The Self-Service Data Platform).
when Many domains, each with real data engineering capability, and central enforcement of interface, quality and discoverability rather than of implementation (Data Mesh).
cost Requires every domain team to acquire and retain data engineering skill, which is an organisational investment with a hiring plan. Without the standard actually enforced, it is decentralisation with a nicer name (Data Governance).
when Never as a choice — but it is the honest description of what many platforms have, and naming it is more useful than pretending an org chart implies ownership.
cost Every incident starts with an unowned question, deprecations never complete, and the datasets that matter most are the ones nobody will claim because claiming them is expensive.
Catalog, lineage and cost-attribution products change their coverage boundaries frequently — which transformation tools they can parse, which query engines they can attribute, whether BI-layer logic is visible to them. Treat any specific coverage claim as something to verify against current documentation, and measure your own coverage directly rather than assuming the vendor's.
How to build it
Most important first.
- Build a paved road rather than a policy. One supported way to ingest a source, define a model, deploy it, test it and publish it — and make that way easier than the alternatives. Governance that is harder than the workaround produces documented non-compliance (The Self-Service Data Platform).
- Make ownership a required field rather than a cultural expectation. A dataset published without a named owning team should fail the same way a model without a schema fails, and the owner should be a team with a rota rather than a person who may leave (Data Ownership).
- Generate the catalog from what actually runs — models, schemas, lineage, freshness, owner, tests — so it cannot drift. A hand-maintained catalog is out of date within weeks and then actively misleading, which is worse than absent (Metadata: Technical, Operational and Business, Data Lineage).
- Define a minimum standard for a dataset that other teams may depend on: declared grain, an owner, a freshness target, a uniqueness assertion on its key, and documented known gaps. Distinguish it explicitly from a team's internal datasets, which need none of that (Data Products, Dataset Documentation).
- Version and deploy transformations exactly like application code: reviewed diff, tested, promoted through environments, with the running version recorded on the output. A data model is software and deserves the same delivery discipline.
- Attribute cost to datasets and teams before you try to reduce it. The first attribution report changes behaviour more than any optimisation, because it converts an argument about frugality into a list of specific queries (Cost Attribution).
What this actually promises
Naming the guarantee you do not have is worth more than naming the one you do — everything downstream inherits the weakest promise in the chain.
- A platform team can guarantee the road: that a dataset built the supported way is deployed, tested, catalogued, monitored and attributed. That is a real and useful guarantee and it is entirely about mechanism.
- It cannot guarantee correctness of data it does not produce. The team that owns a field owns whether it is right; a central group can only measure and report, and placing the whole obligation downstream guarantees it fails (Who Owns Data Quality).
- A catalog guarantees discoverability of what is registered in it. Anything created outside the road is invisible to it, so coverage is the number that matters and it is rarely published (Data Discovery).
- Ownership guarantees an addressee, not a response time. That is why an owner without an SLO is only half of the fix, and why the freshness and quality targets belong to the owner rather than to the platform (The Freshness SLO).
- Nothing here guarantees adoption. Every mechanism on this list can be bypassed, and a platform whose road is optional governs exactly the teams that were already careful.
Can I trust it?
A green pipeline is evidence that code ran. These four fields are the evidence that the data is right.
- The platform-level check is coverage, not correctness: what fraction of serving datasets declare an owner, a grain, a freshness target and at least one assertion, and what fraction were built off the road entirely. It is the one number that tells you whether the mechanism is working (The Data Quality Dashboard).
- It misses quality itself completely. A dataset can satisfy every coverage requirement and be wrong, and a platform that reports coverage as though it were trust has built a very convincing dashboard for the wrong quantity (Data Quality).
- Pair it with a small set of genuinely enforced assertions on the datasets that leave the company, because coverage measures the road and reconciliation measures the data (Reconciliation).
- Duplicated ingestion produces duplicated freshness: several copies of a source, each with its own schedule and lag, and consumers picking whichever they found first. The disagreement between them looks like a data quality problem and is an organisational one (Source of Truth).
- Without published per-dataset freshness targets, consumers assume the platform's best case and discover the worst case during a decision. A freshness number that exists only in a monitoring system reaches the wrong audience (Freshness Monitoring).
- Manual deployment adds an unbounded and invisible delay to every fix: the time until the person who knows how to deploy is available. That delay does not appear in any lag metric and is often the largest term during an incident (Data Incidents).
- Duplicated ingestion multiplies the cost of every upstream schema change by the number of copies, and guarantees the copies diverge — because each team absorbs the change differently and only some of them notice (Schema Evolution).
- Without ownership there is no addressee for a deprecation, so nothing is ever removed. Platforms accumulate datasets monotonically, and the catalog fills with tables that have not been written to in a year and are still read by something (Data Discovery).
- The road itself evolves, and every change to it applies to teams who adopted it in good faith. Migrating consumers off an old pattern is the recurring cost of having a platform, and pretending otherwise is how a second road appears (Data Contracts).
- Recovery here is consolidation, and it is slow. Pick the sources duplicated most, build one supported ingestion for each, migrate consumers one at a time, and delete the old paths only once nothing reads them — which requires the lineage you probably do not have yet (Impact Analysis).
- Assigning ownership retroactively is a week of unpleasant meetings and the highest-leverage week available. Start from the datasets that feed external reporting and work down; datasets nobody will claim are usually datasets nobody needs (Data Ownership).
- Bringing manually deployed logic under version control begins with capturing what is currently running, which is archaeology rather than engineering. Do it before the next change, because after the change the original is gone.
What can go wrong
- The platform team becomes a queue. Every dataset needs its attention, throughput is bounded by its headcount, and teams route around it — which recreates the duplication the team was formed to remove (The Self-Service Data Platform).
- The road is built and adoption is voluntary, so it is adopted by the teams that were already disciplined and the coverage number improves without the problem changing.
- A catalog is bought, populated once by an import, and never regenerated. Within a quarter it disagrees with reality and people learn to distrust it, which is harder to undo than never having had one (The Data Catalog).
- Ownership is assigned to individuals rather than teams, and the reorganisation makes the whole map wrong in one afternoon.
- Quality standards are set centrally without the producing teams agreeing to them, so tests are added, fail, and are muted — leaving a platform that reports coverage and enforces nothing (Alert Fatigue: The Page Nobody Reads).
- Cost attribution is built, published, and used to rank teams rather than to fix drivers, after which the numbers are disputed rather than acted on (Cost Attribution).
- "We need a data platform team." Perhaps. First name which of these six is currently costing you, and what the team would do about it in its first quarter. A platform team formed without that answer becomes a ticket queue.
- "A catalog will fix discovery." A generated catalog helps enormously; a purchased one populated by hand becomes wrong and then ignored. The property that matters is whether it is derived from what runs (Metadata: Technical, Operational and Business).
- "Data mesh means every team does its own thing." It means domain teams own data products against a standard that a platform group provides and enforces. Federation without the standard is the duplication anti-pattern with a better name (Data Mesh).
- "Ownership is a cultural problem." It is a structural one. Make owner a required field on a published dataset and the culture follows the mechanism, which is the reverse of the usual advice and the direction that actually works.
- "Our costs are fine, no query looks unreasonable." That is exactly the signature of the commons problem. The bill is the sum of reasonable things, and only attribution decomposes it (Cost Attribution).
- "These are big-company problems." They are team-count problems. A company of forty people with six product teams has all six, and a company of four hundred with one data team has almost none of them.
- Every entry here is also a governance failure. Without ownership there is no one to approve access; without a catalog there is no inventory to classify; without a supported ingestion path there is no boundary at which PII can be detected or minimised (Data Classification, PII in Pipelines).
- Duplicated ingestion multiplies the copies of sensitive data and hides them from the deletion process, so a request that reaches the supported path leaves the shadow pipelines untouched. The blast radius of a deletion request is exactly as large as your lineage coverage (Deletion Requests, Data Access Control).
Operating it
- Coverage: datasets with an owner, with a declared grain, with a freshness target, with at least one assertion, and the count built outside the supported path (The Data Catalog).
- Duplication: distinct ingestion pipelines per upstream source. Any source read by more than one connector is a consolidation candidate and usually a divergence already (Data Ingestion).
- Deployment provenance: the fraction of running transformations whose code version is recorded and matches a reviewed commit.
- Cost by dataset and by consuming team, published on the same cadence as the bill and readable by the teams themselves (Cost Attribution).
- Time-to-owner during incidents: how long between "this table is wrong" and reaching the person accountable for it. It is the single most honest measure of whether ownership is real (Debugging a Data Incident).
- Below about three independent producing teams, none of these are anti-patterns. Duplication is cheap, everyone knows every dataset, ownership is obvious, and a platform team would be a bottleneck (The Central Warehouse).
- Between roughly three and ten teams, all six appear at once and are usually attributed to growth rather than to structure. This is the range in which building the road is highest-value and lowest-cost.
- Above ten teams, the question shifts from whether to centralise to what exactly is central. Standards, catalog and cost attribution centralise well; domain knowledge and dataset ownership do not, which is the observation the mesh argument is built on (Data Mesh).
- At 100x datasets, discovery and lineage stop being conveniences and become the only way any question can be answered at all. Everything else scales; the graph is what does not scale in a human head (Data Lineage).
- Duplicated ingestion costs engineering time linearly in teams times sources, and costs the source system repeated reads of the same data — which is the part the source owner notices (Ingestion Sources).
- The absence of cost visibility is itself the largest cost driver, because no driver can be reduced until it can be seen. Attribution is the cheapest intervention on this list and is almost always deferred (What Actually Drives Data Platform Cost).
- A platform team is a fixed cost that replaces a variable one. It pays off above the team count where duplication dominates, and below that it is overhead with a roadmap (Data Platform Engineering).
- Manual deployment costs incident duration rather than compute, and incident duration is paid in consumer trust, which is the one currency in this domain that does not recover quickly.
- A paved road costs the freedom teams had before it and the platform team that maintains it. Both costs are real, permanent, and worth paying only above the scale where duplication and divergence cost more.
- Centralised standards make datasets comparable and make the standard the bottleneck. The mitigation is to standardise the interface — grain, owner, freshness, tests — and not the implementation (Data Products).
- Cost attribution creates accountability and creates an argument about allocation that will consume real time. It is still worth it, and it is not free, and pretending it is free is how it gets built without the conversation that makes it useful.
- Mesh-style federation gives domain teams the ownership that makes quality possible and requires every domain team to acquire data engineering capability. That is an organisational investment, not an architecture diagram, and it is routinely presented as the latter (Data Mesh).
Dataset review questions
This lesson uses the shared review exercise.
Where this applies
Almost nothing here is universal. These labels say what each claim is specific to, and where a different engine, format, warehouse or scale would differ.
- ORG-SPECIFICAll six are properties of how work is divided between groups rather than of any technology, and every one of them disappears in a single-team platform regardless of scale. The same warehouse, the same tools and the same volume produce a healthy platform or a landfill depending only on how many independent producers write into it.
- SCALE-SPECIFICThe threshold is team count rather than data volume: below roughly three independent producing teams, decentralised ingestion and informal ownership are faster and cheaper than any alternative, and above roughly ten the informal versions have stopped working entirely. Volume moves none of these thresholds.
- GENERALThe mechanism — coordination cost growing with the number of pairs of teams rather than with the number of teams — is the same in any organisation and is why the transition feels sudden. What differs is which entry becomes painful first, which depends on whether sources or consumers grew faster.
Where the depth lives
This domain teaches how data moves and how you know it arrived intact. It hands the rest off by name.
- — DevOps / Production Engineering owns the delivery mechanics the paved road depends on — version control, review, environment promotion, artefact identity and rollback — applied here to models and schemas rather than to services.
- — Distributed Systems owns the coordination arguments underneath federation: what a shared standard can and cannot guarantee when the participants are independent, and why a contract enforced by convention degrades differently from one enforced by a gate.