Invariants as Tests
An invariant on a page protects nothing after the person who wrote it leaves. Turned into a test that asserts the property after every sequence of actions — including the interleaving that found it — it becomes the only form of the rule that survives refactors, new endpoints and new engineers.
The situation, the reflex, and why it stalls
Every lesson starts where being stuck starts: someone has a problem, and the first move that comes to mind feels like progress.
You have a list of invariants and a design that holds them. How do you turn each one into something that keeps holding it after you have stopped looking?
The invariants are written, the schema has its constraints, checkout has its atomic update. I have unit tests for checkout. But a colleague adding refunds asked me "which tests would fail if I broke the total?", and I realised the answer is "the one that tests checkout, if you break it through checkout" — and refunds do not go through checkout.
Write a test per feature. Checkout has its test, refund gets one, admin edit gets one. Each asserts what that feature does, and coverage goes up. It feels like the invariants are covered because every feature that touches them has a test.
Feature tests assert behaviour, not properties. The refund test checks that a refund refunds; nothing checks that after a refund the total is still non-negative, because that was never the refund feature's job. The invariant is covered by no test in particular, which is the same as by none.
- Feature tests assert behaviour, not properties. The refund test checks that a refund refunds; nothing checks that after a refund the total is still non-negative, because that was never the refund feature's job. The invariant is covered by no test in particular, which is the same as by none.
- The interleaving that found the stock invariant is not a feature, so it has no test. The atomic update that closes it can be replaced by a "cleaner" read-then-write in a refactor and every feature test still passes.
- Coverage rises and the question "which test fails if the total goes negative?" still has no answer. That question is the one the invariant list exists to answer, and the test suite does not know the list exists.
The move
Precisely enough to apply it to a problem you have never seen — not a slogan.
- For each invariant, write a test whose assertion is the invariant itself — the property of state — and whose setup is a sequence of actions, not one feature. "After any sequence of checkout, refund and admin edit, total is non-negative" is a different test from "refund refunds", and it is the one that fails when refunds break the total (A Slice Is Testable).
- Use the disturbance that found the invariant as the first sequence. The double confirmation, the two concurrent checkouts, the price edit mid-checkout — each was a thought experiment in Finding Invariants From Examples; now it is a test that fails without the mechanism and passes with it. A test that has never failed has not proven that it checks anything.
- Where the sequence space is large, generate sequences rather than enumerate them: random actions from the store's vocabulary, assert the invariant after each. This is property-based testing (Property-Based Testing), and it is the natural form of an invariant test because the invariant is a property, not an example.
- Order the work by the cost of violation, not by ease. The double-payment test is harder to write than the negative-total one and matters more; write it first, even if it means learning how to simulate the provider's retry (Risk-First Development).
A feature test and an invariant test, side by side
The two tests below exercise the same refund code. The first asserts what refund does; the second asserts what must remain true after refund and everything else. Only the second fails when a later engineer breaks the total through a path that is not refund — and the difference is entirely in the shape of the setup and the assertion.
1// feature test: what does refund do?2test('refund reduces the total', async () => {3 const order = await checkout(cart([item('A', 2, 10)]))4 await refund(order.id, 10)5 expect((await getOrder(order.id)).total).toBe(10)6})7 8// invariant test: what must never be true, whatever happened?9test('total is never negative after any sequence', async () => {10 const order = await checkout(cart([item('A', 2, 10)]))11 for (const step of sequence(order, ['refund', 'refund', 'adminEdit', 'refund'])) {12 await step() // through the real chokepoints13 const o = await getOrder(order.id)14 expect(recomputeTotal(o)).toBeGreaterThanOrEqual(0) // the invariant, verbatim15 }16})The second test's assertion is the sentence from the invariant list. Its setup is a sequence, and the sequence will be replaced by a generator once the vocabulary settles.
Which invariant gets its test first
The order below ranks by the cost of violation and by whether the violation can be repaired, which is the opposite of the natural order — the easy tests first. It is one defensible sequence for the store; the alternative is the one a team under a launch deadline tends to choose, and the device says when that is right.
- 1Payment at most once per order — double confirmation and double click
because Violation moves real money and is not repairable without a refund and an apology; the test needs a fake provider that can retry, which is worth building first because every payment test needs it.
- 2Stock never negative — two concurrent checkouts for the last units
because Violation is a broken promise to a customer; the test must run the interleaving and be seen failing without the atomic update.
- 3Captured price never changes — edit price after checkout
because Cheap to write, and it locks in the schema decision that snapshot values are copied, which a later "normalise the schema" refactor would otherwise undo silently.
- 4Every item references an existing product — delete during checkout
because Held by a foreign key, so the test mostly proves the retired-flag behaviour on the catalog side; low cost of violation, low cost of test.
- 5Total never negative — generated sequences of checkout, refund and edit
because Repairable if violated, so it goes last — but it is the best candidate for a generator, and the generator tends to find the unlisted cases.
Three ways to hold an invariant over time
The invariant can be held by a constraint, by an example test or by a generated-sequence test, and usually by more than one. The matrix scores the three against the axes a test decision moves; the caveat is where the numbers stop being honest.
| Option | Simplicity | Reliability | Time | Maintainability | Note |
|---|---|---|---|---|---|
| Schema CHECK constraint only | Prevents the negative row; says nothing about what the code does when the write is refused. A crash on the last unit is "reliable" for the database and not for Bob. | ||||
| Hand-written interleaving test | Proves the mechanism handles the one ordering that was found; must be seen failing to be trusted; flaky in informative ways. | ||||
| Generated-sequence property test | Finds orderings nobody listed; needs a small vocabulary and a cheap oracle; failures arrive needing shrinking. |
caveat The scores compare three shapes of the same test on one invariant and mean nothing across invariants; "reliability" here is about catching violations, not about the store. In practice the store uses all three, and the question is which to write today.
How to do it
Most important first.
- Take each invariant's sentence and make it the assertion, verbatim if possible:
assert total >= 0,assert count(successful payments for order) <= 1,assert stock >= 0. - Make the setup a sequence of actions through the real chokepoints, including the ones added later. If refund cannot be called from the test, that is a finding about the chokepoint.
- Encode the interleaving that found the invariant as a concurrent test, and check that it fails when the mechanism is removed.
- Add a generated-sequence test where the action vocabulary is small and the state is checkable — inventory and totals are ideal.
- Keep the invariant list and the test list in the same place, so that a new invariant with no test is visible as a gap.
Worked on a concrete problem
The move has to produce something. This is what it produced.
- "Payment succeeds at most once per order." Test: create an order, deliver the provider's confirmation twice, and also click Pay twice before the first confirmation; assert one succeeded payment row and one charge recorded against the fake provider. First run against the version before the unique constraint: two rows. That failure is the proof that the test sees the invariant. Then, with the constraint: one row, the second insert refused and treated as "already done".
- "Stock is never negative." Test: stock 3; two checkouts of 2 started concurrently against the real database; assert exactly one succeeds, stock reads 1, and no row ever reads below zero. Removing the conditional update and replacing it with read-then-write makes the test fail — sometimes. Flakiness here is information: the race is real and the test needs to run the interleaving enough times to see it (Stress Testing: A Test That Passed Once Proves Nothing).
- "Total is never negative." Generated sequence: a random mix of checkout, partial refund, full refund and admin line edit, a few hundred steps, asserting after each that every order's recomputed total is ≥ 0. It found a case nobody had listed: two partial refunds whose sum exceeded the item price, because refund validated against the item price and not against what had already been refunded. The invariant was right; the chokepoint was missing a fact.
How you know it worked
What now exists that did not before, and what question you can now ask.
- For every invariant on the list there is a test whose assertion is the property, and the answer to "which test fails if I break this?" is a filename.
- Each such test has been seen failing — against the pre-mechanism version, or with the mechanism deliberately removed — at least once.
- A new endpoint that touches guarded state either goes through a chokepoint the tests already exercise or fails a generated-sequence test.
- At least one test found a violation that no thought experiment had listed.
The questions you can now ask
The field this whole domain exists for. After this lesson, these are the questions to put to an unfamiliar problem.
- ?If this invariant were violated tomorrow, which test would fail — and can I name the file?
- ?Has this test ever been seen failing, so that I know it observes the property and not the happy path?
- ?Is the setup a sequence of actions through the real chokepoints, including the actions added since the invariant was written?
- ?Which invariants have a small action vocabulary and a cheap oracle, and therefore want generated sequences rather than hand-picked ones?
- ?Which invariant is the most expensive to violate and the hardest to test — and is that the one I am writing first?
What can go wrong
- The invariant test is written as a feature test in disguise: one setup, one action, one assertion. It passes forever and protects one path. The sequence is the point.
- The concurrent test is made "reliable" by adding sleeps until it always passes, which usually means it no longer produces the interleaving. A concurrent invariant test that can never fail has stopped testing.
- Generated-sequence tests are applied to state with no cheap oracle — "the recommendation list is always sensible" — and the assertion becomes as complicated as the code. Property tests want properties that are easy to state and expensive to keep.
- The tests exist and the invariant list is deleted as redundant. The list is the index that tells the next engineer which tests are load-bearing; the tests alone look like all the others.
- Invariant tests run sequences and sometimes concurrency, so they are slower and occasionally flaky in ways that are informative but expensive to maintain.
- A generated-sequence test that finds a bug also finds it at an inconvenient moment, in a form that needs shrinking before it is understood.
- Tests through real chokepoints couple the suite to the design; changing where an invariant lives means changing its test's setup.
- "Constraints in the database are the test." A constraint enforces; it does not prove that the code handles the enforcement — that the second confirmation is treated as "already done" rather than as a crash. The test covers the behaviour on violation, which the constraint cannot.
- "Property-based testing replaces the thought experiment." It generates sequences you did not think of, given a property you did think of. The property still has to be found by hand first; the generator has no idea what the store must never say.
- "A prototype does not need these." A prototype that takes real money needs the payment invariant test and can skip the rest; the decision is per invariant, by the cost of violation, not per project (What Cannot Be Simplified).
Where this applies
Problem-solving advice is stated as universal far more often than it is. These labels say what each method is specific to — and where CONTESTED appears, the note gives the strongest form of the opposing view.
- GENERALAny invariant over checkable state can be a test whose assertion is the property and whose setup is a sequence; the technique carries from stores to parsers to schedulers unchanged.
- CONTESTEDSome practitioners hold that invariant and property tests are expensive noise on a small system and that a schema constraint plus a handful of feature tests is the right investment until the system has earned more. The strongest form: every test is a maintenance cost, invariant tests couple to design decisions that are still moving, and on a store with one engineer the time is better spent shipping the refund feature than generating sequences for it. The position here is that the money-and-stock invariants earn their tests on day one and the rest can wait — but that is a judgment, not a rule.
- ILLUSTRATIVEThe two partial refunds exceeding the item price and the flaky concurrent test are invented findings, chosen to show what each kind of test discovers; the "few hundred steps" is a number for shape only.
Where the depth lives
This domain asks the question and hands the answer off by name.
- — A Testing & Reliability domain would teach fakes for external providers and shrinking of generated failures; until it exists, the Design lessons above cover property tests and doubles, and /manifesto/without-ai covers writing the first invariant test before asking an assistant for one.