TestingSCALE-SPECIFICGENERALCONTESTED

Contract Tests

A contract test is the thing that keeps a double honest. It is also a design decision: writing one is a declaration that this seam is a contract and not an implementation detail.

The requirement, the obvious build, and why it breaks

Every lesson starts where the work starts: someone asked for something, and the first implementation that comes to mind survives until the requirement changes.

The question

Two sides of a boundary each have green tests. What proves they agree?

The requirement

Checkout mocks the inventory module. Inventory changed reserve() to return a reservation id instead of a boolean six weeks ago. Both test suites are green and have been the entire time.

The obvious build

Both sides have thorough tests, and the interface is written down in a shared type or an OpenAPI file. The compiler and the schema will catch a mismatch.

Why it breaks

A shared type catches shape, and most breakages are not shape. reserve() still returns a string; it now returns a reservation id where it used to return a status code, and every type checks.

How it breaks as requirements change
  • A shared type catches shape, and most breakages are not shape. reserve() still returns a string; it now returns a reservation id where it used to return a status code, and every type checks.
  • Schemas do not carry semantics: which fields are optional in practice, what an empty list means, whether the call is idempotent, what happens on a retry, which error is retryable (Retryability: Telling Clients What To Do Next).
  • A shared type creates a coupling of its own — both sides now depend on a third artefact, and updating it is a coordination event that people avoid by adding fields instead of changing them (Shared Libraries).
  • It fails hardest exactly where it matters most: at the moment the two sides stop deploying together, which is the moment the shared type stops being shared reality and becomes shared intention.
RequirementConstraintsInvariantsResponsibilitiesBoundariesInterfacesStateDependenciesFailureImplementationTestsFeedbackEvolution

What limits the solution, and what must never stop being true

This domain leads with these two. A design that ignores its constraints is not a design, and an invariant nobody named is one nothing is protecting.

Constraints
  • The two sides are owned by different people, and increasingly by different teams.
  • They release independently, so "just deploy them together" is not available and will get less available.
  • The provider cannot be run inside every consumer build once there are eight consumers.
Invariants
  • If a consumer's double claims the provider behaves a certain way, something must check that claim against the real provider.
  • A provider must not break an expectation a consumer actually relies on without that being a visible, deliberate act (Backward Compatibility as a Constraint).

Who owns what, and where the seams fall

Responsibilities decide boundaries; boundaries decide what an interface has to say.

Responsibilities
  • The consumer owns declaring what it actually relies on — which is always a subset of what the provider offers, and usually a small one.
  • The provider owns satisfying every declared expectation, and owns being told before it breaks one.
  • The contract owns being the executable form of that agreement. If it is a document rather than a test, it decays like a document (Documentation Decay).
Boundaries
  • A contract test marks a seam as a contract. That is a design decision with consequences: contracts are expensive to change, and declaring one on an internal seam you would rather stay fluid is a mistake (Stable Boundaries).
  • The right seams are the ones where the two sides change on different schedules or under different ownership. Everything else should stay refactorable (Internal Module Contracts).
  • Inside a single deployable with a single owner, the contract test is usually unnecessary — you can call the real thing, and that is a stronger check for less work (Mocking).

The mechanism, in five steps

The whole idea is that one suite runs in two places. Everything else — brokers, versioning, can-i-deploy gates — is logistics around that single fact, and a team that gets the two runs working has most of the value before any of the logistics.

The step that gets skipped is the fourth, and skipping it removes the entire benefit: if the provider does not run the consumer's expectations in its own pipeline, breakage is detected after it ships.

Consumer-driven contract testing
  1. 1
    Consumer declares

    Writes only what it actually relies on: "reserve(sku, 2) returns a reservation with an id and a status of HELD".

    fails by Declaring the whole provider surface, which freezes everything and gives the provider no room (API Stability)

  2. 2
    Consumer verifies its double

    Runs the expectations against the double it uses in its own tests, so the double is proven to satisfy them.

    fails by Stopping here — this is a mock with paperwork and checks nothing about reality

  3. 3
    Contract is published

    The expectations are stored somewhere the provider's build can fetch — a broker, or a file in a shared repo.

    fails by A manual copy step, which decays within two sprints

  4. 4
    Provider verifies

    The provider's pipeline replays every consumer's expectations against the real implementation, before merge.

    fails by Running it nightly or post-deploy, which converts prevention into an alert

  5. 5
    Deploy gate

    A provider version may deploy only if every consumer version currently in production still verifies against it.

    fails by Treating a red contract as advisory, at which point the whole mechanism is documentation again

Steps one and four carry all the value. If your budget only covers two steps, run the consumer's expectations against the real provider in the provider's build and skip the broker — a directory of JSON files works (Reversible and Irreversible Decisions).

What belongs in a contract, and what does not

A contract should say what the consumer would break on and nothing else. That is a narrower set than most people write, and the narrowness is not a limitation — it is what leaves the provider free to change everything nobody depends on.

The distinction to hold: shape is what a schema already checks; semantics is what nothing checks. Spend the contract on semantics.

A consumer's expectations of the inventory module
1// what checkout actually relies on — and nothing more
2contract('checkout -> inventory', (inv) => {
3 it('a successful reservation yields an id we can cancel', async () => {
4 const r = await inv.reserve({ sku: 'A1', qty: 2 })
5 expect(r.status).toBe('HELD')
6 expect(typeof r.id).toBe('string') // shape
7 await inv.cancel(r.id) // and the id is usable
8 })
9
10 it('reserving the same requestId twice reserves once', async () => {
11 const req = { sku: 'A1', qty: 2, requestId: 'r-1' }
12 const a = await inv.reserve(req)
13 const b = await inv.reserve(req)
14 expect(b.id).toBe(a.id) // semantics: idempotency
15 })
16
17 it('insufficient stock is a typed refusal, not an exception', async () => {
18 const r = await inv.reserve({ sku: 'EMPTY', qty: 1 })
19 expect(r.status).toBe('REJECTED') // semantics: expected failure
20 })
21})
22
23// NOT in the contract: field order, the exact wording of messages,
24// how reservations are stored, anything checkout never reads.

The second and third cases are the ones a schema cannot express and a shared type cannot enforce, and they are the ones that break silently. Note also what is absent: no assertion on internal storage, no assertion on fields checkout does not read — every extra assertion is a freedom the provider loses (Exposing Too Much).

When this is the wrong tool

Contract tests declare a seam to be a contract, and that declaration has a cost that compounds: the seam becomes something you negotiate rather than something you refactor. On an internal boundary that is a self-inflicted wound.

The honest decision is about ownership and release cadence, not about how the code is organised. Two modules in one repository, owned by one team, deployed together, do not need this — a direct call is stronger evidence and free.

Does this seam deserve a contract test?

Can the two sides be changed and released together by people who talk to each other?

Same deployable, same owner

when Two modules in one repo, one team, one release.

cost No contract test. Call the real implementation in the test — stronger evidence, zero infrastructure, and the seam stays free to move (Mocking).

Same deployable, different owners

when A modular monolith where a platform module has several consuming teams.

cost A shared suite run against both the real module and its fake. Cheap version of the idea; keeps the fake honest without a broker (Internal Module Contracts).

Separate deployables, few consumers

when Two or three services released independently.

cost Contract tests are worth it, and a directory of contract files is enough infrastructure. Cost: pipeline wiring on both sides.

Separate deployables, many consumers

when A platform service with eight or more consumers.

cost Contract tests plus a broker and a deploy gate. Cost: real infrastructure, plus the ongoing prune of expectations nobody relies on any more (Deprecation).

A third party you do not control

when A payment provider, a partner bank.

cost You cannot make them verify your expectations, so run your suite against their sandbox on a schedule and treat drift as an incident. Weaker evidence, and the honest name for it is monitoring (Designing for Failure).

How to build it

Most important first.

  • Write the contract from the consumer's side: only what this consumer uses. A consumer-driven contract is small, honest and lets the provider know exactly what it is allowed to change (Consumer-Driven Evolution: Telemetry Before Breakage).
  • Run the same suite twice — against the double the consumer uses, and against the real provider. Those two runs are the entire mechanism; a contract test that only runs against the double is a mock with paperwork.
  • Put the provider-side run in the provider's pipeline, so breaking a consumer fails the *provider's* build. If the failure lands anywhere else it arrives too late to prevent the deploy.
  • Cover semantics, not shape: what a second identical call does, what an empty result means, which errors are retryable, whether ordering is guaranteed (Idempotency by Design).
  • Keep the contract narrow deliberately. Every expectation you assert is something the provider can no longer change freely, so an over-broad contract freezes the provider by accident (API Stability).
  • For an in-process module, use the same suite against the real implementation and its fake — the same idea at a much lower price (Test Doubles, Precisely).

What the next change costs

The field this whole domain exists for. A structure is only better if it makes the change after this one cheaper — and it is worth saying which changes it does not help.

Cost of the next change
  • Without contract tests: a provider change that breaks a consumer costs a production incident, a rollback, and a cross-team debugging session in which the first hour goes on establishing which side changed. Call it a day, plus the customer impact.
  • With contract tests: the same change costs a red build on the provider's branch with a message naming the consumer and the expectation. Minutes, before merge.
  • The recurring cost is not small: every genuinely intentional provider change now requires updating the contract and coordinating with consumers, and that is friction on every release, including the ones nobody was going to break.
  • The next consumer added costs one small contract file, not another end-to-end environment — which is the property that makes this scale where integration environments do not.
What the recommended approach costs
  • Contract testing is infrastructure: a broker or a repository, pipeline wiring on both sides, and a convention nobody can violate. That is a real fixed cost, and below a few consumers it does not pay.
  • It makes the seam rigid on purpose. Every expectation is a promise, and promises are what you are trying to have fewer of on an internal boundary.
  • It gives no evidence about performance, ordering under load, or the behaviours that only appear when both sides are under stress. It verifies agreement, not that the system works (Where a Test Must Be Real).

What can go wrong

Failure modes
  • The contract is written by the provider and describes everything it offers, which freezes the entire surface and defeats the purpose.
  • The provider-side run is not wired into the provider's pipeline, so contracts are verified after deploys rather than before, and the mechanism produces alerts instead of prevention.
  • Contracts are declared on internal seams, and a refactor that should have been free becomes a cross-team negotiation (Over-Decomposition).
  • The contract asserts on incidental detail — field order, an error message string — so it breaks constantly and gets marked as advisory, at which point it is documentation again.
Dependencies, and their direction
  • The provider now depends on the set of consumer expectations, which is a deliberate inversion: the provider's freedom is bounded by what consumers declared, rather than by what they might be doing (Dependency Inversion).
  • Both sides depend on the contract-storage mechanism — a broker, a repository, a directory of files — which is real operational infrastructure and a real failure point.
  • Nothing depends on a shared implementation type any more, which is the coupling the contract test replaces.
Misreads
  • "A schema is a contract." A schema constrains shape. Most breakages are semantic — meaning changed, optionality changed, idempotency changed — and pass every schema check (What an API Contract Actually Is).
  • "So contract-test every module boundary." Contracts freeze seams. Inside one deployable with one owner, calling the real thing is stronger evidence and costs nothing (The Modular Monolith).
  • "Contract tests replace integration tests." They replace the combinatorial explosion of pairwise integration, not the tests that verify a real dependency's semantics (Where a Test Must Be Real).
  • "The provider should write the contract." Then it describes the whole surface and freezes all of it. The value comes precisely from the contract being the consumer's small subset.

Testing it, and how it ages

What to test, and at which boundary
  • The contract suite itself, run twice: consumer-side against the double, provider-side against the real implementation (Test Doubles, Precisely).
  • A deliberate negative test: change the provider in a way that breaks a declared expectation and confirm the provider build goes red. An unverified safety mechanism is not one (Failure-Aware Feature Design).
  • Keep end-to-end tests for a small number of genuinely cross-cutting flows — contract tests replace the combinatorial middle of that suite, not all of it.
How this design ages
  • Contracts accumulate and become the real record of what a module promises — usually more accurate than its documentation, because they fail when they are wrong (Docs Close to Code).
  • Expectations that no consumer relies on any more should be deleted, and nobody ever does, so the contract slowly ossifies the provider. A periodic prune is genuine maintenance work.
  • The practice becomes necessary at exactly the point where the two sides stop deploying together. Introducing it before that point is usually premature; after it, usually late (What Changes at the Network Boundary).

Where this applies

This domain's advice is contested more than most. These labels say what each claim is specific to — and where CONTESTED appears, the note gives the strongest form of the opposing view rather than a caricature.

  • SCALE-SPECIFICBelow about three consumers, pairwise integration tests are simpler and give stronger evidence; the contract-testing machinery is not repaid. Above roughly five, pairwise integration environments become combinatorially unaffordable and contract tests are the only mechanism that scales linearly. The switchover is a real threshold, not a matter of taste.
  • GENERALThe core idea — run one suite against both the double and the real thing — applies unchanged to an in-process module and its fake, where it costs a file rather than an infrastructure decision.
  • CONTESTEDThe strongest opposing view is that consumer-driven contracts formalise a relationship that would be better removed: if two components need this much ceremony to stay compatible, the boundary is in the wrong place, and the contract infrastructure lets teams keep a bad decomposition alive comfortably. Practitioners who hold this point at the maintenance burden of contract brokers and at contracts that ossify providers for years. It is a fair criticism of the practice as commonly deployed; it is weaker where the boundary is genuinely forced — a partner API, a platform team with dozens of consumers — because then the relationship exists whether or not you formalise it.

Where the depth lives

This domain teaches the codebase-level structure and hands the rest off.

Distributed Systemsservice-boundaries
Domains that do not exist yet
  • Testing & Reliability Engineering — broker operation, contract versioning, can-i-deploy gating and how contract suites interact with a release train are that domain's mechanics. Here the question is only which seams deserve to be contracts.
  • System Design — deciding where independently-deployable boundaries should exist at all comes first; contract testing is a consequence of that decision, not a reason for it.