Baselines
Rule, mean predictor, majority class, linear model, simple tree. Mandatory before anything complex, because a model that does not beat a useful baseline has not earned its cost.
A metric with nothing to compare it to is a number, not a result. Before anything complex: a rule, a constant predictor, a linear model, a shallow tree — and the question of whether the proposed model beats a useful one by enough to pay for itself.
The heuristic the business already uses is the strongest baseline most models face, and the honest reason a model has to win by a margin: the rule is free to run, already trusted, and already in production.
The constant predictor is the floor of every metric. Under imbalance it wins accuracy without looking at a single feature, and for regression it defines R² = 0 — which is why scoring it first is how you find out whether the metric means anything.
Logistic or linear regression as the first real model: cheap to fit, cheap to serve, readable, and the reference for everything after it. If the complex model cannot beat it clearly, the complexity is not earning anything.
A comparison is only a comparison on the same split, the same metric, the same threshold policy, with an interval — and the margin has to be read against what the winner costs to serve. A small gain that costs a GPU and eighty milliseconds is a loss.