Categorical Encoding
One-hot, ordinal and embedding encodings turn categories into numbers. Each has a vocabulary that was fitted on training data, an unseen-category policy, and a serving path that must apply both identically.
The problem, the obvious approach, and why it breaks
Every lesson starts where the work starts: someone has a problem, and the first model that comes to mind looks fine offline.
A model consumes numbers. How does a category become one, what fitted state does that create, and what happens when production sends a category training never saw?
An insurance pricer takes vehicle make and model, region and occupation as inputs. The model was strong offline. Six months in, a new manufacturer entered the market, two regions were merged administratively, and the occupation list was updated by the front-end team. Quotes for those cases became erratic and nobody noticed for a quarter.
One-hot encode every categorical column. It is lossless, it makes no ordering assumption, and every library does it. Ordinal encoding for the ones that have a natural order. Done.
A new manufacturer's vehicles arrive with a make the encoder has never seen. Depending on the library, that is an all-zeros row, an exception, or a silent mapping to whichever column is first. The model has no learned behaviour for any of these; the all-zeros case looks like "no make at all", which never occurred in training.
- A new manufacturer's vehicles arrive with a make the encoder has never seen. Depending on the library, that is an all-zeros row, an exception, or a silent mapping to whichever column is first. The model has no learned behaviour for any of these; the all-zeros case looks like "no make at all", which never occurred in training.
- Two regions merged into one new code. Every quote from those regions now looks like an unseen category, and the model loses the regional risk it had learned for both.
- The occupation list was renamed — "Software Engineer" to "Software Developer" — so a common category became unseen overnight, and the serving path's own column list, rebuilt at the next deploy from the updated reference table, no longer matched the artifact's column order.
What is being predicted, and from what data
This domain leads with these two. A target nobody defined precisely is a label nobody can trust, and a dataset nobody can describe is a model nobody can debug.
- Predict expected claim cost for a policy at quote time. The label is the total claim amount over the policy year.
- The prediction sets a price, so a wrong number is either a lost customer or an underpriced risk.
- One example is one quote with several high-cardinality categorical fields — thousands of vehicle models, hundreds of regions and occupations — and a few numeric ones.
- Categories were one-hot encoded from the values present in the training set. The serving path built its own column list from the same reference table, at deploy time.
- The front-end occupation list, the region table and the vehicle catalogue are maintained by three other teams and change without notice.
How it actually works
Precisely enough to predict its behaviour — not a framework API.
- One-hot encoding maps each category to its own indicator column. The set of columns — the vocabulary — is fitted on the training data and is state: the model's weights are indexed by it. Ordinal encoding maps categories to integers by a chosen order; the order is fitted state and, for a linear model, an assertion that the categories are equally spaced along a line. Embeddings map each category to a learned dense vector, with a row per vocabulary entry (Embeddings).
- Every encoding needs an unseen-category policy, and the policy is a modelling decision rather than an error-handling detail. Map to an explicit
__unknown__column that was present in training — by hashing rare training categories into it — and the model has learned something for that case. Map to all-zeros and the model is asked about a state it never saw. - High cardinality strains every encoding. One-hot over thousands of vehicle models produces thousands of sparse columns, most seen a handful of times, and a tree model spends its splits on them. Grouping rare categories, hashing, target encoding (Target Encoding) and embeddings are the alternatives, each trading interpretability or leakage risk for compactness.
- The vocabulary is a contract with upstream. A renamed value is not a new category to the business but is to the encoder. Drift in the category distribution — new values appearing, old ones vanishing — is a feature drift signal the encoder makes visible if it counts unknowns (Feature Drift).
A vocabulary is a contract
The one-hot encoder turned a thousand vehicle makes into a thousand columns, and the model's weights are indexed by them. That column list was fitted on the training data. It is state, and it belongs in the artifact. The serving path that rebuilt its column list from the vehicle catalogue at deploy time produced a list that was current, differently ordered, and wrong for the model.
The same encoder needs an answer for a value outside its vocabulary, and "whatever the library does" is not an answer. The policy that works is to make unknown a category the model has seen: fold rare training values into an explicit __unknown__ column so the model has learned weights for it, then send every unseen serving value there.
1def fit_vocab(train_col, min_count=20):2 counts = train_col.value_counts()3 vocab = sorted(counts[counts >= min_count].index) # stable order4 return vocab + ["__unknown__"] # unknown exists in training5 6def encode(col, vocab):7 index = {v: i for i, v in enumerate(vocab)}8 unk = index["__unknown__"]9 ids = col.map(lambda v: index.get(v, unk)) # unseen -> unknown, never zeros10 out = np.zeros((len(col), len(vocab)))11 out[np.arange(len(col)), ids] = 112 return out, (ids == unk).mean() # unknown rate is monitored13 14artifact["vocab"] = fit_vocab(train["vehicle_make"]) # shipped; serving never refitsThe min_count threshold is what puts rows into __unknown__ during training. Without it the unknown column is all zeros in training, the model learns nothing for it, and serving-time unknowns are still undefined behaviour.
What each encoding asserts
One-hot asserts nothing about the categories except that they are distinct. Ordinal encoding asserts an order and, for a linear model, equal spacing — "Gold" is one step above "Silver" which is one step above "Bronze". An embedding asserts that the categories have a geometry the model can learn from the label, which needs enough rows per category to learn it.
The choice is by cardinality, by model family and by whether the assertion is true. Ordinal-encoding region codes because the integers are compact hands a linear model a spurious trend across an alphabetical list.
| Encoding | Fitted state | Asserts | Unseen category | Fits when |
|---|---|---|---|---|
| One-hot | vocabulary, column order | categories are distinct | explicit unknown column, or undefined | low to moderate cardinality; any model |
| Ordinal | the order | a real order, and for linear models equal spacing | must map to a chosen rank | genuinely ordered categories: tiers, sizes, grades |
| Grouped / hashed | group map or hash seed | rare values behave alike | lands in a shared bucket by construction | high cardinality with trees; accepts collisions |
| Target encoding | per-category rates, prior | the category's label rate is informative | falls back to the prior | high cardinality with trees; must be out-of-fold (Target Encoding) |
| Embedding | a vector per vocabulary entry | a learnable geometry | a dedicated unknown vector | high cardinality, a network, many rows per category |
The unknown rate is the monitor
A rename upstream, a merged region, a new manufacturer: each one shows up on the first day as a step change in the unknown rate for one feature. That is the earliest and cheapest signal the encoder offers, and it exists only if the serving path counts unknowns rather than silently mapping them.
The response to a step change is a person, not a retrain. A rename needs a mapping in the serving path. A merged region needs a decision about which learned risk applies. A new manufacturer genuinely needs new data, and until it exists the honest quote is the unknown bucket's.
The categories arriving at serving are the training vocabulary plus a small, stable fraction of unknowns, encoded by the artifact's vocabulary in the artifact's column order.
holds when The encoder is loaded from the artifact; upstream reference data is stable or changes are mapped in the serving path; the unknown rate is flat.
breaks when An upstream team renames or merges values; a new entity class appears; serving rebuilds the encoder from a reference table; a retrain reorders columns and serving indexes by position.
respond Identify the upstream change; add a mapping for renames; decide explicitly how merged or new categories are priced; retrain only when there is data for the new values.
How to build it
Most important first.
- Fit the vocabulary on the training fold, store it in the artifact with the column order, and have serving load it; never rebuild it from a reference table at deploy time (Preprocessing Lives in the Artifact).
- Reserve an explicit unknown bucket that exists in training — fold categories below a frequency threshold into it — so the model has learned weights for "rare or unseen".
- Choose the encoding by cardinality and model: one-hot for low cardinality; grouped one-hot, hashing or target encoding for high cardinality with trees; embeddings for high cardinality with a network and enough data.
- Monitor the unknown rate per categorical feature at serving; a step change is an upstream rename or a new value and needs a human, not a retrain.
What to measure
Which number actually maps to the decision — and which numbers look relevant and are not.
- The unknown-category rate per feature at the serving boundary, daily. This is the number that catches a rename or a new manufacturer on the day it happens.
- Production error on the subset of quotes that contained an unknown category, once claims arrive, against the rest.
- Offline validation, which drew from the training vocabulary, cannot see any of this.
What must stay true after deployment
The field this whole domain exists for. A model is a set of assumptions with weights attached; these are the ones a monitor or a test should be checking.
- The vocabulary and column order in the serving path are those stored in the artifact, and serving never builds an encoder from a reference table.
- The unknown-category rate at serving stays close to the training rate; a step change is treated as an upstream change to be understood, not absorbed.
- The ordering used by any ordinal encoding reflects a real order in the data, and the categories with a real order are the only ones ordinally encoded.
- Offline: a contract test comparing the serving encoder's vocabulary and column order to the artifact's.
- Offline: hold out categories deliberately — train without one region — and check the model's behaviour on the unknown bucket is sane, not catastrophic.
- Online: monitor unknown rates per feature; alert on a step change; report production error stratified by whether the row contained an unknown.
What can go wrong
- An unknown bucket exists, but the rare categories folded into it in training were rare for a reason — old vehicle models — so the model learned "unknown means old and cheap", and a brand-new manufacturer is priced as if it were.
- The vocabulary is frozen in the artifact, and the serving code, written against an older artifact, indexes columns by position; a retrain that adds a category shifts every column after it.
- Ordinal encoding was applied to a category with no real order — region codes — because it was compact, and a linear model learned a spurious trend across alphabetically sorted regions.
- A frozen vocabulary means new legitimate categories are "unknown" until the next retrain, which is correct and unsatisfying for a business that wants the new manufacturer priced well from day one.
- Folding rare categories into an unknown bucket loses their individual signal, which for a long-tail category like vehicle model can be considerable in aggregate.
- Embeddings handle cardinality well and need a network and enough rows per category to learn anything; on small tabular data they underperform grouped one-hot.
- "One-hot is lossless, so it cannot cause problems." It is lossless on the training vocabulary. Its behaviour on anything outside that vocabulary is undefined unless you defined it.
- "Rebuild the encoder from the reference table at deploy so it is always current." Current is the problem. The model's weights are indexed by the training vocabulary; a current vocabulary reindexes them.
- "Unknown categories are rare, so handle them however." They are rare until the upstream team renames a common value, at which point the most common category is unknown for every request.
Where this applies
ML advice is stated as universal far more often than it is. These labels say what each claim is specific to — and where CONTESTED appears, the note gives the strongest form of the opposing view.
- MODEL-SPECIFICTree ensembles handle high-cardinality one-hot poorly and benefit from grouping or target encoding; linear models need one-hot or a carefully justified ordinal; networks can learn embeddings given enough rows per category.
- DATA-SPECIFICWith a handful of stable categories one-hot is the whole answer; with thousands of values, a long tail and upstream churn, the choice of encoding, the unknown policy and the vocabulary contract are most of the work.
Where the depth lives
This domain teaches the model and hands the rest off by name.