StateLANGUAGE-SPECIFICGENERALCONTESTED

Optional Values and Absence

Say "this may be missing" in the type where the language supports it, and never encode absence as an empty string, a zero, a sentinel date or a magic id.

Constraints & invariants Who owns what What the next change costs

The requirement, the obvious build, and why it breaks

Every lesson starts where the work starts: someone asked for something, and the first implementation that comes to mind survives until the requirement changes.

The question

This value may not be there. How do I say that so a caller cannot forget, and what do I do in a language that will not let me?

The requirement

A report of customers who have not logged in since 2020 includes every customer who has never logged in, because "never" was stored as 1970-01-01 and a comparison did the obvious thing.

The obvious build

Use a default. Zero for missing numbers, empty string for missing text, the epoch for missing dates. The column stays non-nullable, the code has no branches, and nothing can be null-dereferenced.

Why it breaks

The sentinel is a legal value of the type, so every operation happily computes with it: a sum includes the zeros, a comparison ranks the epoch first, a string concatenation produces a stray space. Nothing errors and every answer is wrong (Swallowed Errors).

How it breaks as requirements change

The sentinel is a legal value of the type, so every operation happily computes with it: a sum includes the zeros, a comparison ranks the epoch first, a string concatenation produces a stray space. Nothing errors and every answer is wrong (Swallowed Errors).
Two meanings collapse. A quantity of zero and an unknown quantity are different facts, and once both are 0 no amount of downstream code can tell them apart — the information is destroyed at the point of storage.
The convention is per-field and undocumented, so a reader has to know that -1 means "unlimited" here and "not applicable" three columns over.
It fails silently and late. The 1970 bug did not appear until someone wrote a report three years after the choice, and by then two million rows encode it.
Every new consumer re-learns the convention or does not, and the ones that do not produce plausible numbers rather than errors, which is the worst possible failure shape (Debuggability by Design).

Requirement→Constraints→Invariants→Responsibilities→Boundaries→Interfaces→State→Dependencies→Failure→Implementation→Tests→Feedback→Evolution

What limits the solution, and what must never stop being true

This domain leads with these two. A design that ignores its constraints is not a design, and an invariant nobody named is one nothing is protecting.

Constraints

The column is non-nullable and used in an index; making it nullable requires a migration on a large table.
A partner API returns "" for missing names and 0 for missing quantities, and cannot be changed.
The codebase mixes TypeScript on the server with a Python data pipeline, and the two express absence differently.

Invariants

A value that may be absent is typed as possibly-absent, or absence is checked exactly once at a boundary.
Absence is never encoded as a value from the domain of the type — no zero, no empty string, no epoch, no -1, no "N/A".
Two kinds of absence that mean different things are not represented by the same value.

Who owns what, and where the seams fall

Responsibilities decide boundaries; boundaries decide what an interface has to say.

Responsibilities

The type owns saying that a value may be absent, wherever the language can express it.
The boundary owns converting a foreign encoding of absence — the partner's "" and 0 — into the domain's representation exactly once (Boundary Adapters).
The caller owns deciding what to do when the value is missing, and the type's job is to make that decision unavoidable rather than optional.
Somebody owns distinguishing the kinds of absence: "never happened", "not applicable", "not yet loaded" and "the user declined to say" are four different facts (An Error Taxonomy That Survives Contact).

Boundaries

The parse boundary is where foreign sentinels die. Inside the model there is one representation of absence and it is the language's (Parse, Do Not Validate).
The wire and the database are outside: JSON has null, SQL has NULL with three-valued logic, and neither is the same thing as the model's optional. Mapping is the boundary's job.
The line between "absent" and "a state" matters: if absence changes which operations are legal, it is a state and belongs in the lifecycle rather than in a nullable field (Explicit State).

What a sentinel actually costs

The problem with 1970-01-01 is not that it is ugly. It is that it is a valid date, so every operation defined on dates accepts it and produces an answer. Sorting puts those customers first. Comparisons include them. A "days since login" calculation returns twenty thousand and the average is destroyed.

A sentinel does not fail; it computes. That is the entire difference between it and an optional, and it is why these bugs are found by someone noticing that a number looks odd rather than by anything in the system.

Four sentinels and the wrong answers they produce

Trigger	Symptom	Cause	Response
Missing last-login stored as `1970-01-01`	A report of customers inactive since 2020 includes everyone who never logged in at all.	The sentinel is a legal date and satisfies every comparison a real date would.	Nullable column plus an optional in the model; the report then has to say explicitly what to do with "never", which is the decision that was skipped.
Missing quantity from a partner feed stored as `0`	Stock levels read as zero and reordering triggers for products that are actually fine.	Unknown and zero are different facts collapsed into one value at the boundary.	Convert at the adapter to absent, and log the conversion count — the number will be higher than anyone expects (Anti-Corruption Layer).
Missing middle name stored as `""`	Printed names have a double space; a uniqueness check treats all no-middle-name customers as colliding.	The empty string is a valid string, so concatenation and comparison both succeed.	Optional in the model; formatting joins only the present parts.
Column made nullable, queries not revisited	`WHERE region <> 'EU'` silently drops every row with a NULL region.	SQL three-valued logic: NULL compared to anything is unknown, not true.	Explicit `OR region IS NULL` where it belongs, and a test for the case. Moving to NULL solves the arithmetic problem and introduces this one, which is worth knowing in advance.

Absence at the boundary, certainty inside it

The design that scales is not "check for null everywhere". It is to convert once, at the edge, into a type that states what is known — after which the interior of the system has nothing to check.

Note the second thing the parser does: it distinguishes kinds of absence. "Never logged in" and "we lost this during the 2019 migration" are different facts, and a report that treats them identically is answering a different question from the one it was asked.

Convert once; distinguish the kinds that matter

1// two kinds of absence, because they lead to different answers
2type LastLogin =
3  | { known: true; at: Date }
4  | { known: false; why: 'never-logged-in' | 'lost-in-migration' }
5 
6function parseLastLogin(raw: string | null): LastLogin {
7  if (raw === null) return { known: false, why: 'never-logged-in' }
8  const at = new Date(raw)
9  // the old sentinel, converted exactly once, here and nowhere else
10  if (at.getTime() === 0) return { known: false, why: 'lost-in-migration' }
11  return { known: true, at }
12}
13 
14// the report can no longer get this wrong by accident
15function inactiveSince(c: Customer, cutoff: Date): boolean {
16  return c.lastLogin.known && c.lastLogin.at < cutoff
17  //     ^ the compiler required this; the sentinel version did not
18}

The why field is the part most teams skip and later wish they had. Once absence has a reason, questions like "how many customers have we lost login history for?" become answerable, and the migration backlog stops being invisible. Note also that inactiveSince contains no null check — the optionality is handled by the shape of the type, so the branch is one the compiler insisted on rather than one a reviewer had to notice (Local Reasoning).

The smell, and where a sentinel is genuinely right

This is a smell rather than a law, and the exception is not a technicality — there are systems where a documented sentinel is the correct engineering choice, and knowing why keeps this from being a slogan.

The diagnostic is whether the sentinel can be reached by ordinary operations on the type. A value outside the representable domain is a different thing from a value inside it.

smellSentinel absence

looks like A value from the type's own domain reserved to mean "not there": 0, -1, "", 1970-01-01, 9999-12-31, "N/A", or a magic id like user_id = 0. Usually accompanied by a comment on the column and nothing enforcing it.

suggests Absence was encoded rather than expressed, so every operation defined on the type will accept the sentinel and compute with it. The failure is silent, arrives at aggregation time, and is discovered by a human noticing an implausible number rather than by any check in the system.

fix Express absence in the type where the language allows it, and convert foreign sentinels once at the adapter. For an existing sentinel in a large table, add a nullable column, backfill with reconciliation rather than blind reinterpretation — some of those epochs may be real — and deprecate the old column on a schedule (Expand and Contract).

when this is fine It is genuinely correct in two situations. First, when the sentinel is outside the representable domain of real values — a NaN for a missing float, or a timestamp of 9999-12-31 in a system where no real date can exceed a known horizon and the schema enforces it — because then no legitimate value can collide with it. Second, in columnar analytical stores where nullability has a measured storage and vectorisation cost across billions of rows, and the platform enforces the sentinel through schema contracts and validation that the pipeline actually runs. Both exceptions depend on enforcement existing somewhere; a sentinel documented only in a comment has neither property (Documentation Decay).

How to build it

Most important first.

Use the language's optional where there is one — Option<T>, T?, T | undefined with strict null checks — and let the compiler force the caller to handle it.
Where absence must be stored in a non-nullable column, keep a separate boolean or a companion column rather than reserving a value, so no arithmetic or comparison can silently include it.
Convert foreign sentinels at the adapter and record the conversion. The partner's 0 becomes absent, and the count of conversions is worth logging because it will surprise you (Logging at Boundaries).
Distinguish the kinds of absence when they lead to different behaviour. NotApplicable and Unknown as separate variants cost one line and prevent a class of report bug (Making Illegal States Unrepresentable).
Push the check to the boundary and out of the consumers: parse once into a type that says what is known, so downstream code has no optionality to forget (Parse, Do Not Validate).
For the migration, add the nullable column, backfill epochs to null, and reconcile — do not reinterpret the sentinel in place, because some of those epochs may be genuine 1970 dates (Data Migration).

What the next change costs

The field this whole domain exists for. A structure is only better if it makes the change after this one cheaper — and it is worth saying which changes it does not help.

Cost of the next change

Before: every new consumer of the field must learn the sentinel convention. The cost is one bug per consumer that does not, discovered whenever someone checks the numbers.
After: a new consumer cannot compile without handling absence. The cost per consumer is one branch, paid once, at the moment they can still ask what it should do.
The next change that becomes cheap: adding a second kind of absence. "Declined to say" becomes a variant, and the compiler lists every consumer that must now distinguish it.
The change that stays expensive: the migration itself. Two million rows, a non-nullable indexed column, and an ambiguity — some epochs may be real — that no automated backfill can resolve. That cost is why sentinel choices deserve thought on the day they are made (Data Migration).

What the recommended approach costs

Optional types add a branch at every consumer, and in code where the value is genuinely always present that branch is noise (Over-Design and Under-Design).
Nullable columns can complicate indexes and query plans, and in SQL they bring three-valued logic, which trades one silent-wrong-answer trap for another.
Distinguishing kinds of absence multiplies cases. Two kinds is usually right, four is usually a model that has confused absence with state.

What can go wrong

Failure modes

Optionality is expressed in the type and then immediately discarded with a force-unwrap or a non-null assertion at every call site, which reintroduces the crash and adds ceremony (Swallowed Errors).
A default is substituted at the boundary — missing becomes zero — so the model is optional-free and the original bug is preserved inside it.
Absence is modelled precisely in one language and flattened in another: the TypeScript service distinguishes null from undefined and the Python pipeline maps both to None, so the distinction dies at the boundary between them (Contract Tests).
The mitigation fails too: making the column nullable moves the problem into SQL's three-valued logic, where WHERE last_login <> '2020-01-01' silently excludes NULL rows — the same class of silent wrong answer in a new place.

Dependencies, and their direction

Every consumer depends on the optional type rather than on a convention, which converts an oral tradition into something the compiler participates in.
The adapter depends on the partner's encoding, deliberately and in one place, so their next change to it is a one-file change (Anti-Corruption Layer).
The database schema and the model depend on each other only through the mapper, which is what lets a non-nullable column back an optional field.

Misreads

"Null is the billion-dollar mistake, so never use null." The mistake is *unchecked* nullability — a reference that may be null with nothing forcing you to consider it. A checked optional is the fix, and in a language with strict null checks, null is that optional (Error Modeling).
"Use a default and there is no problem." A default is a sentinel with better manners. It removes the branch and keeps the wrong answer, and it is the direct cause of the 1970 report.
"Make everything optional to be safe." Then every consumer branches on cases that cannot occur, and the genuinely optional values are lost among them. Model what is actually unknown (Making Illegal States Unrepresentable).
"The database should never have NULLs." A widely-repeated claim that trades a well-understood mechanism for an invented one. NULL has real problems — three-valued logic being the main one — and reserving -1 has all of those plus arithmetic that silently succeeds (Normalization: 1NF to BCNF in Database Engineering discusses the modelling side).

Smells this explains

primitive-obsession
duplicate-knowledge

Testing it, and how it ages

What to test, and at which boundary

A test per adapter that every foreign encoding of absence maps to the domain's absence, including the ones you did not expect — the partner returning "null" as a string is a real thing.
A test that the report question is answered correctly for a customer who has never logged in, which is the specific bug and the specific regression to pin (Characterization Tests).
Property tests over aggregations: a sum over a column with absent values must equal the sum over the present ones, which catches sentinel leakage anywhere in the pipeline (Property-Based Testing).
At the SQL boundary, a test of the three-valued-logic case, because NOT IN and <> with NULL are the most reliably surprising thing in this lesson.

How this design ages

Sentinels accumulate because each one is locally convenient and nothing forces a review. A codebase acquires -1, "", 0, 1970-01-01 and "UNKNOWN" over five years, each with a different meaning.
The healthy direction is that absence gets more specific over time — "unknown" splits into "never happened" and "not applicable" as the business learns it cares about the difference.
What forces change is almost always a report. Sentinels are invisible in transactional code that only ever reads one row and catastrophic in anything that aggregates (Designing for Cost has the query-shape version).

Where this applies

This domain's advice is contested more than most. These labels say what each claim is specific to — and where CONTESTED appears, the note gives the strongest form of the opposing view rather than a caricature.

LANGUAGE-SPECIFICThis lesson's achievable strength varies more by language than almost anything else in the domain. Rust's Option<T> and Haskell's Maybe make absence a case the compiler forces you to handle, with no null at all. Kotlin and Swift give nullable types with compiler-enforced unwrapping, which is the same guarantee with different syntax. TypeScript with strictNullChecks gets there at compile time and nowhere at runtime, so a JSON parse can still produce an absent value in a field typed as present. Java has Optional for return values but references remain nullable everywhere, so the discipline is partial by construction. Python, Ruby and JavaScript without strict checks have None/nil/undefined as ordinary values with no enforcement, which is why sentinels are most common and most damaging there. And SQL is its own case: NULL exists but propagates through three-valued logic, so x <> 'a' excludes NULL rows and NOT IN with a NULL returns nothing — a trap that catches experienced engineers annually.
GENERALThat encoding absence as a value from the type's own domain destroys information is true everywhere; only the available alternatives differ.
CONTESTEDThe strongest opposing view comes from data engineering: in columnar stores and analytical pipelines, nullability costs storage and query performance, breaks vectorised execution, and NULL semantics differ subtly between engines — so a documented sentinel with a schema-level contract is both faster and more portable than nullable columns. That argument is real and measured in systems processing billions of rows, and it is why some serious data platforms mandate sentinels. It is also why those platforms invest heavily in the schema documentation and validation that application codebases never write, which is the part teams copy the least and need the most.

Where the depth lives

This domain teaches the codebase-level structure and hands the rest off.

API Designvalidation-errors enum-evolution

Backendparse-dont-validate validation-layers backend-migrations

Frontendloading-and-error-states

Databasenormalization requirements-to-schema

Domains that do not exist yet

— Programming Languages & Runtime Internals — whether absence is a type constructor, a nullable reference or an ordinary value is a language-design decision, and it determines whether this lesson describes a guarantee or a discipline.
— Testing & Reliability Engineering — sentinel leakage is caught by property tests over aggregations far more reliably than by example-based tests, because the bug only appears when values are combined.