Evaluation Slices
An aggregate metric is a weighted average over subgroups, and the weights are the dataset's, not the business's. A model can improve on average and regress on the segment that matters, and only a sliced evaluation can see it.
The problem, the obvious approach, and why it breaks
Every lesson starts where the work starts: someone has a problem, and the first model that comes to mind looks fine offline.
The new model is better overall and the complaints are up. Which subgroup got worse, why did the aggregate hide it, and what should the evaluation have reported instead?
A lending platform replaces its credit model with one that is better on every aggregate metric. Within weeks, a partner that brings in small-business applicants from one region reports that approvals for their customers have collapsed. The overall approval rate and default rate both look fine, and the partner is threatening to leave.
Report AUC and default rate at the operating point on the full held-out set. The new model wins on both; the held-out set is representative of the applicant population, so what is good on average is good.
The aggregate is dominated by the consumer majority. The new model improved there and regressed sharply on the small-business segment, and the segment's share of rows is too small to move the average.
- The aggregate is dominated by the consumer majority. The new model improved there and regressed sharply on the small-business segment, and the segment's share of rows is too small to move the average.
- The regression is not in the metric the aggregate tracks. Default rate among approved small-business applicants is fine — because almost none are approved. The failure is in the approval rate, which the model metric never saw.
- The segment that regressed is one the business has a contract with. The dataset weights it by row count; the business weights it by revenue and by the partner's exit clause.
- A fairness review, had one been run, would have found it: the segment is correlated with region and business size, and the regression is a disparity the aggregate metric structurally cannot report (Fairness).
What is being predicted, and from what data
This domain leads with these two. A target nobody defined precisely is a label nobody can trust, and a dataset nobody can describe is a model nobody can debug.
- Predict whether an applicant will default within twelve months. The label exists only for approved applicants, and arrives a year later.
- The decision is approve, decline or refer, at a threshold set per product; the business outcome is default loss against approved volume, per partner and per region as well as in total.
- One example is one approved application with its twelve-month outcome, joined to bureau data and application fields. Most rows come from the large consumer segments; the small-business regional segment is a small fraction of rows.
- The features that predict default well in the consumer majority — bureau depth, revolving utilisation — are sparse or missing for small-business applicants, whose risk is carried by fields the model weights lightly because they matter only in the minority.
How it actually works
Precisely enough to predict its behaviour — not a framework API.
- Any metric computed over a dataset is an average over its rows, and the rows belong to subgroups in proportions the dataset happened to have. The aggregate is therefore a weighted average of per-subgroup metrics with the dataset's proportions as weights. A change that improves the large groups and damages a small one moves the aggregate in the direction of the large groups.
- The damage to the small group is usually mechanistic, not accidental. A model trained on a mixed population fits the majority's feature relationships; features that matter only in the minority receive small weights because they explain little of the total loss. A new model family or a regularisation change can shift that balance further, and the aggregate rewards it.
- Slicing computes the metric per subgroup and reports the vector. It replaces one number with many, which is the cost, and it makes visible the subgroups whose metric moved against the aggregate, which is the point. The slices worth computing are the ones the business acts on differently, the ones with different feature availability, the rare categories, and time.
The average is a weighting nobody chose
An aggregate metric is the per-slice metrics combined with the dataset's slice proportions as weights. Those proportions came from how the data was collected — which partners send volume, which products are popular — and they are not the weights the business would choose if asked. A segment that is a small fraction of rows and a large fraction of revenue is under-weighted by exactly that ratio.
The new model did not break the aggregate. It moved the aggregate in the direction the aggregate rewards: better on the many, worse on the few. The evaluation reported success because success was defined as the weighted average.
| Slice | Share of rows | Incumbent approval | Candidate approval | Contribution to aggregate change |
|---|---|---|---|---|
| Consumer, urban | large | baseline | slightly up | positive, dominates |
| Consumer, rural | medium | baseline | flat | near zero |
| Small business, partner region | small | baseline | sharply down | negative, too small to register |
| New product, launched this quarter | tiny | no history | no history | no slice — invisible |
Why the small slice regresses
A single model fitted to a mixed population learns the majority's relationships. The features that carry risk for small-business applicants — trading history, sector — explain little of the total loss because there are few such rows, so they receive small weights; the bureau features that carry risk for consumers receive large ones, and are sparse or missing for small businesses. The model scores the minority with the majority's rulebook.
A new model family, a stronger regulariser or a different feature set can shift that balance and the aggregate will reward the shift. The segment's regression is not noise. It is the model doing what the loss asked, on a population the loss barely counted.
Better AUC and lower default rate at the operating point on the full held-out set; the report carried no per-partner or per-region breakdown.
Partner's regional small-business approval rate collapsed within weeks; overall approval and default rates unchanged; a contractual escalation.
- 1The segment is a small share of rows and its regression was too small to move the aggregate, which was dominated by a gain on the consumer majority.
- 2The model metric was default rate among approved applicants, which stays fine when the segment is no longer approved; approval rate was not a model metric and was not reported.
- 3Bureau features are sparse for the segment, and the new model weighted them more heavily than the incumbent, so the segment scored as high-risk by default.
The slice list is a living assumption
Slicing protects the subgroups that have slices. A partner signed after the slice list was written, a product launched last quarter, a region entered this year — each is an unprotected subgroup until someone adds it. The evaluation's coverage of the business is a thing that decays.
So the list is maintained like the schema: reviewed when the business changes, extended when a category the model treats as rare appears, and checked against the population the model is actually scoring, not the one it was evaluated on.
The set of slices in the evaluation covers every segment, partner, region, product and rare category on which the model's behaviour is separately consequential, so a regression in any of them is visible before release.
holds when The slice list is regenerated from the current partner, product and region tables before each evaluation, and includes the rare categories of every feature the model leans on.
breaks when A new partner or product arrives without a slice; a slice's population shifts so that its metric describes a different group; feature availability changes for one segment and not others.
respond Add the slice, re-evaluate the shipped model on it, and treat a regression there as a release defect even though the release passed — the evaluation was incomplete, not the model correct.
How to build it
Most important first.
- Define the slices before evaluation: by segment, region, device, partner, product, time period, and by each rare category of any feature the model leans on. Write them down with the aggregate metric so they are computed for every candidate (Model Regression Tests).
- Report the per-slice metric with its uncertainty, because a small slice has a wide interval and a regression inside the interval is not yet evidence (Metric Uncertainty).
- Set a release rule that a candidate may not regress a named slice beyond a stated tolerance, whatever the aggregate does. The aggregate improving is necessary, not sufficient.
- Where a slice has systematically different feature availability, consider whether it needs its own model or its own features, rather than a share of a model tuned for the majority (Missing Data).
What to measure
Which number actually maps to the decision — and which numbers look relevant and are not.
- Approval rate and default rate per partner and per region at the operating point, against the previous model on the same period. This is the vector the release decision reads.
- The worst slice's change, as a single headline number alongside the aggregate — the aggregate says whether the model improved, the worst slice says whether anyone was hurt.
- Do not read the aggregate default rate as evidence that a segment is fine when the model has stopped approving that segment.
What must stay true after deployment
The field this whole domain exists for. A model is a set of assumptions with weights attached; these are the ones a monitor or a test should be checking.
- The slices defined at evaluation are the subgroups the business cares about in production, including any added since; a segment with no slice is a segment with no evaluation.
- The proportion of each slice in production is close enough to its proportion in the held-out set that the aggregate means the same thing — or, better, the release decision does not depend on the aggregate.
- Feature availability per slice is stable: a data source that starts or stops covering a segment changes that segment's model behaviour without any change to the model.
- Offline: compute the metric per named slice for the candidate and the incumbent on the same held-out period; list every slice where the candidate is worse beyond its interval. Zero such slices is the release condition.
- Online: per-slice approval rate on day one, because approval rate needs no delayed label; per-slice default rate when the labels arrive.
- Over time: re-derive the slice list quarterly from the current partner, product and region lists, and add a slice for any category the model treats as rare.
What can go wrong
- The slices are computed but there are hundreds, and every candidate regresses some slice by chance; the release rule becomes impossible and is quietly dropped. Slices need a tolerance and an interval, and the named slices need to be few.
- A slice is defined on a feature the model uses, and the slice metric is fine, but the population inside the slice shifted — the partner started sending a different kind of applicant — and the per-slice number describes the old population (Data Drift).
- The evaluation slices are the business's segments as of last year; a new partner, a new product or a new region has no slice and regresses invisibly until it complains.
- Slices multiply the evaluation and the report; the release decision becomes a vector comparison that someone has to own, and the temptation is to collapse it back to one number.
- Protecting a small slice from regression can block a candidate that is much better for the majority, which is a business decision the evaluation surfaces and cannot make.
- Small slices have wide intervals; a rule strict enough to catch a real regression will also flag noise, and the tolerance has to be argued per slice.
- "The model is better on the held-out set, so it is better." It is better on the held-out set's weighted average. Whether it is better for the business depends on the business's weights, which are not the row counts.
- "The default rate for that segment is fine." The default rate among approved applicants is fine because the model has stopped approving them. The metric that would have shown the problem is the approval rate, and it was not a model metric.
- "We checked for fairness by looking at the protected attributes." The slice that regressed was a partner's regional small-business segment, not a protected class. Slicing is for every subgroup the business acts on; fairness is one reason among several.
Where this applies
ML advice is stated as universal far more often than it is. These labels say what each claim is specific to — and where CONTESTED appears, the note gives the strongest form of the opposing view.
- GENERALThat an aggregate is a row-weighted average over subgroups, and that the business's weights differ from the dataset's, holds for every metric on every task.
- DOMAIN-SPECIFICLending makes the segment structure explicit through partners, products and regulation; a content ranking system has the same structure across languages, devices and new-user cohorts, with the difference that nobody has a contract naming the slice, so it regresses more quietly.
- SIMPLIFIEDThe segment proportions and the size of the regression are chosen for the shape of the argument, not taken from any real lending portfolio.
Where the depth lives
This domain teaches the model and hands the rest off by name.
- — Regulatory compliance — fair-lending rules define some of the slices by law and the disparity tests that must be run on them; this lesson treats fairness as one reason to slice among several and defers the legal requirements to that field.