Contract Enforcement
Where the check actually runs — producer CI, the ingestion boundary, the entry to transformation — and the trade every enforcement point makes: a silent wrong number becomes a loud failure, which is correct and will still page someone.
Who needs this, what one row is, and why the obvious build breaks
Every lesson starts from the consumer, because designing from the source outward is this domain's characteristic mistake.
At which point in the pipeline does a contract violation stop the data, who gets woken up when it does, and what happens to the batch that was refused?
Two consumers with opposed interests. The analytical consumer wants wrong data to be stopped, because a wrong number costs them more than a missing one. The operational consumer — the team on call for the pipeline — wants the pipeline to keep running, because a stopped pipeline is their night. Enforcement design is the negotiation between those two, and pretending it is not is how enforcement gets switched off (Who Actually Consumes This Data).
The unit of enforcement is one batch or one record, at one boundary. That choice matters: rejecting a whole batch for one bad record is a very different operational posture from routing that record to quarantine and letting the rest through, and the right answer depends on whether the bad record can distort an aggregate.
Check at the end. Run data tests after the transformation has published, alert when they fail, and fix forward. This is where almost every platform starts, because tests are easy to add to a transformation tool and the boundary is somebody else's system (Data Tests).
The test fails after the table is published, so consumers have already read the wrong numbers. Detection happened; prevention did not (Atomic Publish).
- The test fails after the table is published, so consumers have already read the wrong numbers. Detection happened; prevention did not (Atomic Publish).
- The transformation has already coerced the bad values by the time the test runs, so the test sees a column of nulls and cannot tell you what the producer actually sent (Breaking Schema Changes).
- The failing test names the model, not the producer, so the incident is routed to the data team, who cannot fix it. The team that can fix it learns about it second-hand, hours later (Who Owns Data Quality).
- Twenty models downstream have already consumed the bad table by the time anyone reads the alert, so the repair is a dependency-ordered backfill rather than a re-run (Impact Analysis).
- The raw payload has been transformed in place, so there is no copy of what arrived and the conversation with the producer becomes an argument rather than a diff (The Raw Landing Zone).
What is actually happening
- Enforcement is a gate: a point where data can be refused. Every other quality mechanism in the domain is a detector, and the difference is whether the data continues on failure. A test that alerts and lets the publish proceed is a smoke alarm; a check that refuses the batch is a fire door (Data Quality).
- Gates are cheaper the earlier they sit, and the earliest possible gate is before the data exists: a contract test in the producer's continuous integration, which fails a build rather than a batch (Contract Tests Between Services).
- The next gate is the ingestion boundary, where an arriving batch is validated against the declared schema and either lands or goes to quarantine. This is the only gate available for a producer you do not control, and it is the last point at which nothing downstream has been touched (Transport Validation).
- The third is the entry to transformation — the promotion from raw into modelled data. Raw has already landed, which is what you want, and nothing modelled has been written yet, which is what makes it still cheap (Raw, Staging, Curated: Layers by Purpose).
- Every gate converts one failure into another. It takes a silent wrong number and produces a loud missing one. Nothing about enforcement makes data correct; it makes incorrectness *visible*, and visible incorrectness is an outage (The Pipeline Succeeded. The Data Is Wrong.).
- Because a gate produces outages, its social durability depends entirely on its false-rejection rate. A gate that has ever refused good data during an incident will be disabled during the next one, and gates that get disabled are worse than gates that were never built, because the platform was designed assuming they existed (Alert Fatigue: The Page Nobody Reads).
Where the check runs
There are only a few places a contract can be enforced, and they differ in what they can see, what they cost, and who they can stop. The pattern across all of them: the earlier a gate sits, the cheaper the failure and the narrower the coverage.
The earliest gate is in the producer's own build, where a failure costs a red pipeline and nothing else. The latest useful gate is at the promotion into modelled data, where a failure costs a stale table. Anything after that is a detector rather than a gate, because the data has already been published and consumers have already read it.
Most platforms should have two: one in the producer's build for the producers they control, and one at the ingestion boundary for everyone else. Having only the second is normal and workable. Having only a post-publish test is the state this lesson exists to move you out of.
What is the earliest gate you can actually place, given who owns each hop?
when You control the producing service, or the producing team is willing to take the dependency. Always do this where it is possible.
cost Covers only producers you control, and only what the contract declares. Failure costs a red build and zero freshness — by far the cheapest failure available (Contract Tests Between Services).
when Always, and it is the only option for third-party and SaaS sources. This is the last point before anything of yours has been written.
cost A refused batch is a freshness incident, and the remedy for an external producer is a support ticket rather than a fix. Requires a quarantine path or you are choosing between wrong data and no evidence.
when Raw should always land regardless of conformance, so this is where strictness belongs in a lake-first platform.
cost Raw is already written, so a violation costs storage and an unpromoted period. Nothing modelled is touched, which keeps the blast radius at one dataset (Raw, Staging, Curated: Layers by Purpose).
when As a detector alongside the gates, never as a substitute for them. It is the only place cross-dataset invariants can be checked at all.
cost Consumers have already read the data. This catches things no earlier gate can see and it cannot prevent anything (Data Tests).
when For a consumer whose stakes are much higher than the platform's default — a financial close, a regulatory report.
cost Duplicated logic per consumer, and it protects exactly one consumer. It is nonetheless the right call when the cost of one wrong number is much higher for that consumer than for anyone else (Trusting Data).
The gates in order, and what each one still cannot see
Reading the gates as a sequence rather than as alternatives is what makes the design decisions obvious. Each one catches a class the one before it could not reach, and each one is more expensive when it fires.
Pay attention to the guarantees column and to how narrow each promise is. No gate anywhere in this table promises that the data is correct. They promise, in order: that the producer's build matches the contract; that what arrived conforms structurally; that what is promoted conforms; and that the assertions someone wrote hold. Correctness is not among them and cannot be (Semantic Changes).
The last row is the one that keeps enforcement honest. However many gates you build, a consumer with high stakes should still check the properties they personally depend on — because your gates encode what the platform agreed to, and their requirements may be stricter than that agreement.
- 1Producer build
Runs a contract test against the payload the service emits and fails the build on a breaking change.
guarantees That a declared breaking change from a producer you control cannot reach production unnoticed.
fails by Covering only declared fields, and only producers who run it. Everything undeclared passes.
- 2Ingestion boundary
Validates each arriving batch against the declared schema, types and allowed values; routes violations to quarantine.
guarantees That structurally non-conforming data does not enter the platform, and that the refused payload is retained as evidence.
fails by Being configured to infer rather than assert; dropping rather than quarantining; or blocking a stream partition with no dead-letter path.
- 3Raw landing
Writes what arrived, unchanged, including anything the boundary flagged.
guarantees That the original bytes survive the incident, which is what makes reprocessing and the producer conversation possible (The Raw Landing Zone).
fails by Being transformed on ingest, which destroys the only copy of what the producer actually sent.
- 4Promotion to modelled
Asserts conformance, grain and required fields before raw becomes a staging or curated model.
guarantees That non-conforming data does not become something a consumer queries. Raw keeps it; models do not.
fails by Running after a permissive cast has already turned bad values into nulls, so it inspects the damage rather than the cause (Breaking Schema Changes).
- 5Post-publish tests
Runs assertions across published datasets, including cross-dataset invariants and reconciliations.
guarantees That the assertions someone wrote hold. Nothing more, and nothing about the assertions nobody wrote.
fails by Firing after consumers have read the data, and being routed to whoever owns the model rather than whoever caused the problem (Who Owns Data Quality).
- 6Consumer assertions
A high-stakes consumer checks the specific properties its own decisions depend on.
guarantees That this consumer will not act on data violating its own requirements.
fails by Existing in one consumer and nowhere else, so everyone with the same requirement and less diligence is unprotected (Trusting Data).
Four gates and two detectors. The two detectors are not redundant — they see cross-dataset and cross-time properties no single-batch gate can express — but they cannot stop anything, and a platform that has only them has detection without prevention.
The trade, stated honestly
src/de/sim/pipeline.ts under a fixed seed, with the schema-change fault and the contract-check mitigation; the check outcomes are computed by that model rather than measured on a real platform, and scripts/de-sim.test.ts pins both halves so the claim cannot drift from the code.The in-repo pipeline model makes the trade concrete. Under its schema-change fault with no enforcement, the run publishes normally: every row is present, unique and fresh, and the revenue figure is zero against a true total that is not. Two of the model's six checks fail, and they are the two checks most platforms have not implemented.
Turn on the contract check and the batch is rejected, the run fails, and nothing is published. Now four of the six checks fail — including freshness, because the serving table still holds the previous period. The count of failing checks went up, the platform got safer, and the reason is that the failures moved from a category nobody watches into a category everybody watches (Breaking Schema Changes).
That is the whole trade and it should be said out loud when enforcement is proposed: incidents go up, wrong numbers go down, and the person paged is frequently not the person who caused it. A team that adopts enforcement expecting a quieter on-call rotation will disable it within a quarter, and a disabled gate is worse than one that was never built, because the platform around it was designed assuming it was there.
The cost shape below orders what enforcement actually costs, relative to itself. The compute is not the story. The operational and coordination costs are, and they are the ones that decide whether the gate is still enabled next year.
The dominant cost, paid in on-call time and stale tables. It scales with how often producers violate the contract and with how quickly they respond, both of which are organisational rather than technical.
The mechanism working as intended: a change that used to cost the producer nothing now costs a discussion. Scales with the number of producer teams, not with data volume.
Ongoing and unglamorous, and it grows with dataset count rather than with volume. A gate cannot enforce more than the contract declares, so this work sets the ceiling on coverage.
Proportional to how long the violation went unresolved, which makes producer response time a compute cost as well as a trust cost.
Proportional to records inspected and to inspection depth. Structural and type checks are small next to any transformation over the same data; full value-level validation of every field is not.
Refused volume times retention. Negligible until a producer breaks something for a week, at which point it is still the smallest line here.
Relative weights, not measurements. The ordering is what transfers between platforms; the magnitudes do not, and this domain never publishes a price.
Relative weights for a typical platform, shown to establish an ordering rather than a magnitude — not measurements and not transferable to any specific system. The ordering is the teaching: the cost of enforcement is operational and organisational, and the line people try to optimise first is the smallest one.
The gate refuses the batch, logs the rule that fired, and discards the payload. The pipeline is protected, the table is stale, and there is no copy of what arrived.
The gate refuses the batch, writes the payload unchanged to a quarantine location with the rule that fired and the contract version it was checked against, and alerts the producing team with a link to it. Once the producer ships a fix, the quarantined range replays through the same path.
The refused payload is the only evidence of what the producer actually sent, and it is the input to both the conversation and the recovery. Without it the discussion with the producing team is two teams describing what they believe happened, and the recovery depends on the source still holding the range — which for a stream past its retention window it does not. Quarantine costs a little storage and converts an unresolvable argument into a diff.
How to build it
Most important first.
- Put a contract test in the producer's pipeline first. It is the only gate that fails a build instead of a batch, and it is the only one that costs a consumer nothing (Contract Tests Between Services).
- Enforce structure and types at the ingestion boundary, and route refused batches to quarantine rather than dropping them — a refused batch is evidence, deleting it destroys the only proof of what the producer sent, and a retained one replays through the same path once the producer ships a fix (The Raw Landing Zone).
- Land raw permissively and promote strictly. The raw layer's job is to be an accurate record of what arrived, including things that should not have arrived (Keeping Raw History: The Recovery Position and the Liability).
- Decide per field whether a violation refuses the record or the batch. A malformed optional attribute can be quarantined per record; a malformed measure that feeds a published aggregate should stop the batch, because a partial aggregate is worse than none (Atomic Publish).
- Route the alert to the producing team, with the offending payload attached. An enforcement point that pages the data team about somebody else's bug converts a fixable incident into a relay race (Data Ownership).
- Track false rejections explicitly and treat a rise in them as a defect in the gate. The gate's credibility is a real asset and it is spent every time it is wrong (Quality Alerting).
What this actually promises
Naming the guarantee you do not have is worth more than naming the one you do — everything downstream inherits the weakest promise in the chain.
- A gate guarantees that data violating the *declared* rules does not pass it. It guarantees nothing about rules nobody declared, which is where the module's worst failures live (Semantic Changes).
- It guarantees that a failure is visible, which is the actual product. It does not guarantee anyone acts on it — that depends on routing and on whether the alert reaches someone who can fix it.
- It explicitly does not guarantee that data will arrive. Refusing bad data means having no data, and a consumer who was promised freshness has to be told which of the two they are getting (The Freshness SLO).
- It guarantees nothing about periods already published under the old regime. Turning on enforcement today says nothing about last quarter (Reconciliation).
Can I trust it?
A green pipeline is evidence that code ran. These four fields are the evidence that the data is right.
- The check is the gate itself: validate the payload against the declared contract and refuse what does not conform (Data Contracts).
- It misses everything the contract does not declare — semantics, units, cross-field invariants, and anything about a field nobody wrote down.
- It also misses absence entirely. A gate evaluates what arrived; a producer that sends nothing sends nothing that violates the contract, which is why a freshness and completeness check must sit beside it rather than behind it (Freshness Checks).
- Enforcement trades freshness for correctness, deliberately and every time. A refused batch means a stale table, and staleness is the visible symptom that makes the failure loud enough to act on.
- The freshness cost is bounded by how quickly the producing team can respond, which makes routing part of the freshness design rather than an operational detail (Data Incidents).
- Producer-side gates cost no freshness at all, because they act before the data exists. That is the argument for pushing enforcement as far upstream as the organisation allows (Contract Tests Between Services).
- A gate on a stream stops consumption at the offending record unless there is a quarantine path, so the freshness cost there is unbounded rather than one batch — which makes per-record quarantine substantially more important on streams than on files (Offsets and Commits).
- The gate has to evolve with the contract, in the right order: a producer shipping a legitimate contract change before the gate is updated is refused for doing exactly what was agreed. Contract, gate and producer are one change and should be deployed as one (Schema Evolution).
- Tightening an existing gate is a breaking change for producers who have been relying on it accepting more than the contract said. Announce it and run it in warn-only mode first.
- A gate in warn-only mode is a detector, not a gate. Running one indefinitely because nobody wants the outages is a decision to accept silent wrongness, and it should be made explicitly rather than by default.
- Once the producer is fixed, replay the quarantined range through the same path. This works only if quarantine retained the payloads and the pipeline is idempotent over the range (Idempotent Data Pipelines).
- If enforcement was added after a period of silent violations, the recovery is a backfill of that period from raw, in dependency order, validated before publishing (Planning a Backfill).
- If a gate was disabled during an incident, treat re-enabling it as its own change with its own verification. Disabled gates are re-enabled far less often than they are disabled.
What can go wrong
- The gate is placed downstream of the coercion, so it inspects nulls rather than the values that produced them.
- The gate refuses a batch and drops it, destroying the evidence and making the producer conversation unresolvable.
- The gate routes its alert to the data team rather than to the producer, so the fix is always second-hand.
- The gate is disabled during an incident to restore the pipeline and is never re-enabled — the most common way enforcement ends, and it leaves a platform designed around protection it no longer has.
- The gate is run in warn-only mode indefinitely, which is a detector wearing a gate's name in architecture diagrams — or, on a stream, blocks consumption at the offending record with no quarantine path, so one bad record stops the topic (Consumer Groups and the Parallelism Ceiling).
- The contract and the gate drift apart, so the gate enforces last quarter's agreement.
- "We have data tests, so we enforce contracts." A test that alerts after publishing is a detector. Enforcement is the property that the data does not continue, and most test suites do not have it (Data Tests).
- "Enforcement means fewer incidents." It means *more* incidents and fewer wrong numbers. A platform that adopts enforcement should expect its incident count to go up and its restatement count to go down, and should say so before adopting it.
- "Reject and drop is the safe default." Dropping destroys evidence. Quarantine is barely more work and it is the difference between a diff and an argument (The Raw Landing Zone).
- "The gate protects the platform." It protects the paths that go through it. Consumers reading a warehouse table populated by some other route are entirely outside it, and coverage discussions routinely assume otherwise (Schema Registry).
- "If it pages too often we will loosen it." Loosening a gate until it stops firing is a decision to accept silent wrongness. Sometimes that is the right call for a low-stakes dataset, and it should be recorded as a decision rather than arrived at by attrition.
- An enforcement point is a natural place to check classification rules as well as structure — a field arriving with personal data where the contract declared none is a governance incident, and the boundary is where it is cheapest to catch (PII in Pipelines).
- Quarantined payloads are real data with real obligations: they inherit the retention and access rules of the dataset they failed to join, and a quarantine bucket nobody governs is a compliance gap that grows quietly (Data Retention).
Operating it
- Rejections per producer per day, with the rule that fired. A rising rate is a producer making a change nobody announced (CDC and Schema Drift).
- Quarantine depth and age — how much refused data is waiting, and how long it has been waiting. A growing quarantine is an unresolved conversation.
- False-rejection count, tracked deliberately, because it is the number that predicts whether the gate will still exist in a year (Alert Fatigue: The Page Nobody Reads).
- Time from rejection to producer acknowledgement, which is the real freshness cost of enforcement and is an organisational metric rather than a technical one.
- Which gates are currently enabled, as a monitored fact rather than a configuration file. A disabled gate should be as visible as a failing one.
- At 10x volume, per-record validation cost starts to matter and the structural checks stay cheap while value-level ones do not, so gates tend to be trimmed to types and required fields.
- At 10x producers, the routing problem dominates: a gate that cannot name the owning team for a rejection is a gate that generates unassigned tickets (Data Ownership).
- At 100x datasets, hand-written gates are impossible and enforcement has to be generated from the contract, which is the point at which contracts must be machine-readable or the whole mechanism collapses (The Self-Service Data Platform).
- Validation compute at the boundary is proportional to records inspected and to how deep the inspection goes. Structural and type checking is cheap relative to any transformation over the same data; full value-level validation of every field is not (Scan Cost).
- Quarantine storage is proportional to refused volume times how long it is retained, and it is small until a producer breaks something for a week.
- The dominant cost is neither of those. It is operational: the outages enforcement creates, the on-call time they consume, and the coordination that a refused change forces between teams. That is the cost that decides whether enforcement survives contact with a quarter's deadlines.
- Enforcement converts a silent wrong number into a loud failure. That is the correct trade for anything feeding a published metric, and it is still an outage that will page someone — often someone who did not cause it, at an hour when the person who did is asleep.
- Earlier gates are cheaper and cover less. A producer-side gate costs nothing and only covers producers you control; a boundary gate covers everyone and costs freshness. Most platforms need both and get one.
- Per-record quarantine keeps the pipeline moving and lets a partially-complete period publish, which is exactly what you do not want for an aggregate. Per-batch rejection protects the aggregate and stops everything. The right choice is per field, which means someone has to actually make it per field.
Where this applies
Almost nothing here is universal. These labels say what each claim is specific to, and where a different engine, format, warehouse or scale would differ.
- GENERALThe three candidate gate positions — before the data exists, at the boundary where it arrives, and at the promotion into modelled data — exist in every platform, because they follow from the shape of the pipeline rather than from any tool. What varies is which of them you can actually put a check in given who owns each hop.
- SOURCE-SPECIFICA producer you own can be gated in its own build, before any data exists; a third-party SaaS export cannot be gated at all upstream, so the ingestion boundary is the only available point and the only remedy for a violation is a support ticket and a stale table.
- BROKER-SPECIFICOn a stream, refusing a record blocks the partition unless there is an explicit dead-letter path, so one bad record can halt a topic; on a file drop, refusing a batch leaves the rest of the world unaffected. The same policy therefore has very different operational consequences depending on the transport.
- SIMULATEDThe comparison in the third section between an unenforced and an enforced run comes from the in-repo pipeline model under its schema-change fault and contract-check mitigation, not from a measurement. The model is deterministic and the check outcomes are computed from it rather than asserted.
Where the depth lives
This domain teaches how data moves and how you know it arrived intact. It hands the rest off by name.
- — DevOps / Production Engineering owns the delivery half of this: running the contract test as a required check on the producer's merge, deploying the contract and the gate together, and the runbook for a rejected batch at 3 a.m.
- — Distributed Systems owns the dead-letter pattern that makes per-record rejection possible on a stream without halting a partition, and what redelivery of a quarantined record actually guarantees.