Baselines

Rule, mean predictor, majority class, linear model, simple tree. Mandatory before anything complex, because a model that does not beat a useful baseline has not earned its cost.

Baselines Are Mandatory

A metric with nothing to compare it to is a number, not a result. Before anything complex: a rule, a constant predictor, a linear model, a shallow tree — and the question of whether the proposed model beats a useful one by enough to pay for itself.

Q · The model reports a strong validation metric. Compared to what — and what is the simplest thing that would have scored nearly as well?
The Rule Baseline

The heuristic the business already uses is the strongest baseline most models face, and the honest reason a model has to win by a margin: the rule is free to run, already trusted, and already in production.

Q · What does the rule the business runs today score on our holdout — and if we cannot say, what is the model being compared to?
Majority Class and Mean Predictor
▶ lab

The constant predictor is the floor of every metric. Under imbalance it wins accuracy without looking at a single feature, and for regression it defines R² = 0 — which is why scoring it first is how you find out whether the metric means anything.

Q · What does a predictor that ignores every feature score on our metric — and if that number looks good, what does it say about the metric?
The Linear Baseline

Logistic or linear regression as the first real model: cheap to fit, cheap to serve, readable, and the reference for everything after it. If the complex model cannot beat it clearly, the complexity is not earning anything.

Q · What does a regularised linear model on the same features score, and what does its distance from the complex model tell us about where the signal is?
Beating the Baseline
▶ lab

A comparison is only a comparison on the same split, the same metric, the same threshold policy, with an interval — and the margin has to be read against what the winner costs to serve. A small gain that costs a GPU and eighty milliseconds is a loss.

Q · The candidate beats the baseline. On what split, by which metric, at which threshold, with what uncertainty — and after the serving cost, is it still a win?