R² (Coefficient of Determination)
R² is the fraction of variance the model explains relative to predicting the mean. It can be negative out of sample, it is not comparable across datasets, and a high value can describe a model that is useless for the decision.
The problem, the obvious approach, and why it breaks
Every lesson starts where the work starts: someone has a problem, and the first model that comes to mind looks fine offline.
A vendor reports an R² of 0.9 for a demand model and your own model scores 0.6 on your data. Is theirs better, and what would it mean if yours went negative on next month's data?
A retail planning team is comparing an in-house demand model against a vendor's. The vendor quotes a much higher R² on their benchmark. The in-house team's number looks poor by comparison, and on a recent month it came out below zero, which the planning lead reads as "worse than random".
R² is the standard regression score. It is bounded by one, higher is better, and it lets you compare models with a single number the way accuracy does for classification. A model at 0.9 explains ninety percent of what is going on; one at 0.6 explains sixty.
The vendor's benchmark includes a product mix with huge between-product variance, most of which any model explains by knowing which product it is. The in-house data is per-store-per-product residual variance that is genuinely hard. The two numbers are not measuring the same difficulty.
- The vendor's benchmark includes a product mix with huge between-product variance, most of which any model explains by knowing which product it is. The in-house data is per-store-per-product residual variance that is genuinely hard. The two numbers are not measuring the same difficulty.
- The month where R² went negative was a promotional month where the mean of the training period was a bad predictor and the model was worse still. Negative is not "worse than random"; it is "worse than predicting last quarter's average", which on a month unlike the training period is a low bar the model still failed to clear.
- The planning decision is a replenishment quantity, and the cost of being wrong is asymmetric. R² says nothing about the sign of the errors, so a model with a good R² that systematically under-orders promoted items looks fine on the report and empties the shelf.
What is being predicted, and from what data
This domain leads with these two. A target nobody defined precisely is a label nobody can trust, and a dataset nobody can describe is a model nobody can debug.
- Predict next week's unit sales per store and product. The label is the recorded sales count, which is censored by stock-outs — a product that sold out on Wednesday records the sales it managed, not the demand it had.
- The prediction feeds a replenishment order, so the decision downstream is a quantity, and the cost of a miss depends on whether it is an over-order (holding cost) or an under-order (lost sales).
- One example is one store-product-week: last eight weeks of sales, price, promotion flags, and the same week last year. The vendor's benchmark uses a different set of stores and a different product mix.
- Variance in the target differs enormously between subsets. Weekly sales across a whole chain vary by orders of magnitude between products; within one store and one product they vary by a handful of units.
How it actually works
Precisely enough to predict its behaviour — not a framework API.
- R² is one minus the ratio of the model's sum of squared residuals to the sum of squared residuals of the mean predictor:
1 − SS_res / SS_tot. It is squared error normalised by the squared error of the simplest possible baseline, evaluated on the same data. - Because the denominator is the variance of the target *in this dataset*, R² depends on how much variance the dataset happens to contain. The same model, with the same absolute errors, scores higher on a dataset with more spread — which is why it is meaningless to compare across datasets, and why aggregating many products lifts it.
- On training data with an intercept, R² is bounded below by zero, because the fitted model can always do at least as well as the mean. On held-out data the "mean" in the denominator is the held-out mean, which the model never saw; a model can lose to it, and then R² is negative. That is not a bug in the metric. It is the metric reporting that the baseline won.
A ratio of two squared errors
R² compares the model's squared residuals to those of a model that predicts the mean of the evaluation set for every row. Both sums are computed on the same data, so the score is relative to how spread out that data is. A model with an absolute error of ten units scores well when the target ranges over thousands and badly when it ranges over tens.
The negative case falls out of the same formula. On held-out data the denominator uses the held-out mean, which the model did not have. If the held-out period is unlike training — a promotion, a shock — the model can do worse than that mean, and the ratio exceeds one.
1import numpy as np2 3def r2(y, yhat):4 ss_res = np.sum((y - yhat) ** 2)5 ss_tot = np.sum((y - np.mean(y)) ** 2) # the mean of *this* set6 return 1 - ss_res / ss_tot7 8# the same absolute error on two evaluation sets with different spread9y_wide = np.array([10, 200, 45, 900, 30, 600])10y_narrow = np.array([40, 45, 42, 48, 44, 41])11err = np.array([8, -8, 8, -8, 8, -8]) # identical residuals12 13print(r2(y_wide, y_wide + err)) # close to 1: the spread hides the error14print(r2(y_narrow, y_narrow + err)) # negative: the mean of the set does betterSame model error, two very different scores. Neither is wrong. R² is answering "how much of this set's spread did you account for", and the sets have different spread.
Which baseline the denominator should be
The standard denominator is the mean, because it is the simplest predictor that needs no features. But the planners do not use the mean; they use the same week last year, and that baseline already explains most of the seasonal variance. A model that beats the mean comfortably can lose to the planners' rule, and R² will not say so.
The useful comparison is the improvement over whatever the business does today, in the units the decision uses. R² against the mean is a diagnostic that the model learned anything at all; it is not the evaluation.
A high value on the chain-wide evaluation set, driven by between-product variance the model explains by knowing the product id.
MAE in units against last-year-same-week on the slice the replenishment decision uses, reported with the R² out of sample as a sanity check.
The decision is made per store-product against an existing rule, so the evaluation has to be made on that slice against that rule, or it measures a difficulty the decision never faces.
What must stay true for the score to mean anything
A quoted R² carries an implicit claim that the production target has the variance the evaluation set had. Narrow the slice, or enter a low-variance season, and the same model earns a lower score for the same error. Widen it — aggregate to region — and the score rises for nothing.
It also carries the claim that the mean was a fair baseline. In a period the training window never contained, the mean is a bad baseline, R² against it is uninformative, and a negative value is a prompt to check whether the period is unlike training before concluding anything about the model.
The variance of the target on the slice the decision uses stays close to the variance of the evaluation set, so the score keeps meaning what it meant at launch.
holds when The product mix, store set and aggregation level of the production evaluation match the offline evaluation, and the period resembles the training window.
breaks when A report aggregates to a higher level; a low-variance season arrives; a promotion produces a period the mean predictor and the model both handle badly.
respond Recompute on the decision's slice and against the business baseline before reading the number. If the period is genuinely new, the answer is more data from it, not a different model.
How to build it
Most important first.
- Use R² for what it is: a normalised comparison against the mean baseline on one fixed dataset. It tells you whether the model beat the simplest predictor, and by how much of the available variance (Majority Class and Mean Predictor).
- Compute it out of sample and against the baseline the business actually uses. If the planners currently order last year's same week, that is the denominator that matters, not the global mean.
- For the decision, report a metric in units — MAE or a quantile loss in units of stock — alongside R², so that "explains sixty percent of variance" is accompanied by "is typically eleven units off" (MSE, RMSE and MAE).
- When comparing models across datasets or vendors, insist on the same evaluation set. If that is impossible, compare against the same baseline computed on each dataset, and report the improvement over baseline rather than R² itself (Beating the Baseline).
What to measure
Which number actually maps to the decision — and which numbers look relevant and are not.
- Improvement over the planner's current rule in units of stock, on a held-out period with the same product mix as production. This is the number that maps to a replenishment decision.
- R² out of sample as a sanity check that the model beats the mean — a negative value is a signal to stop, not a value to explain away.
- Do not compare R² between datasets, product subsets, or time periods. The denominator changed, so the number did.
What must stay true after deployment
The field this whole domain exists for. A model is a set of assumptions with weights attached; these are the ones a monitor or a test should be checking.
- The variance of the target in production is similar to the variance in the evaluation set. R² computed on a high-variance evaluation set overstates what the model explains on a narrower production slice.
- The mean predictor remains a meaningful baseline. In a period unlike the training window — a promotion, a season — the mean is a bad baseline and R² against it is uninformative in both directions.
- The relationship between R² and the decision cost has been checked once, on real orders, and the metric is not standing in for a cost it does not measure.
- Offline: compute R² on the training set and on a held-out period; a large gap is overfitting, a negative held-out value is a baseline that won. Then compute the same on the per-store, per-product slice the decision uses.
- Online: when the week's sales arrive, compute MAE in units and the improvement over the planners' rule; treat R² as secondary.
- Over time: plot R² alongside the target variance per week. A falling R² with falling variance is the denominator shrinking, not the model failing.
What can go wrong
- A model with a high in-sample R² and a low out-of-sample R² has memorised the training set; the gap is the overfitting signal, and the high number is the misleading one (Overfitting).
- The evaluation set is aggregated to chain level for a nicer chart. R² jumps because between-product variance is easy; the per-store decision it was supposed to inform is unchanged.
- The target is censored by stock-outs, so the model learns to predict recorded sales rather than demand. R² looks fine against recorded sales and the model perpetuates the stock-outs it was built to prevent (Selection Bias).
- Refusing to compare R² across datasets removes the one number vendors and papers are happy to quote, and forces an evaluation on your own data, which costs time and sometimes a procurement argument.
- Reporting a metric in units instead of a bounded score makes it harder to say "good" or "bad" without context, which is the point but is also why people reach for R².
- A quantile or cost-weighted loss that matches the replenishment decision is harder to explain to a planning lead than "percent of variance explained".
- "An R² of 0.9 means the model is ninety percent accurate." It means squared residuals are a tenth of the target variance on that dataset. It says nothing about accuracy in units, nothing about the sign of errors, and nothing about the decision.
- "Negative R² means the model is worse than random." It means the model is worse than predicting the held-out mean. That is a specific, informative failure — usually a period unlike the training window — and "random" is not the comparison.
- "Their R² is higher, so their model is better." On a different dataset, with a different variance in the denominator, the comparison is meaningless. Demand the same evaluation set or the same baseline.
Where this applies
ML advice is stated as universal far more often than it is. These labels say what each claim is specific to — and where CONTESTED appears, the note gives the strongest form of the opposing view.
- GENERALThat R² is normalised by the evaluation set's own variance, and can go negative out of sample, follows from its definition and holds for any model family.
- DATA-SPECIFICOn a dataset with large between-group variance — many products, many stores — R² is easy to inflate by including the group identity; on a narrow, single-entity series the same model earns a far lower value for the same absolute error.
- SIMPLIFIEDThe R² values quoted here are for the shape of the argument, not measurements; the point is the comparison, not the numbers.
Where the depth lives
This domain teaches the model and hands the rest off by name.
- — Statistics — adjusted R², partial R² and the relationship to the F-test are where the statistical treatment continues; this lesson stops at the point where the number is used to make a decision.