SIMULATED

ML Failure Simulator

Break a serving system ten different ways and read the nine signals a monitoring stack would show. Then do it blind: open signals one at a time, name the cause, and find out how many causes were still consistent with what you looked at.

Problem→Target→Data→Representation→Split→Model→Training→Evaluation→Validation→Deployment→Inference→Monitoring→Drift→Retraining

Every failure here has a wrong fix that looks right, and for most of them it is the same one: retrain. Retraining on corrupted labels bakes the corruption in; retraining on a feature that went null teaches the model to ignore it; retraining because someone moved the threshold changes nothing the threshold did. The order of checks that transfers is business metric first, then whether quality is unknown or down — the difference is the whole diagnosis — then serving, then features, and the model last.

Add leakage

A new feature — "days since last support ticket" — is added to the training set with a point-in-time join that is off by one: it sees the ticket the user opens after the outcome. The model retrains and ships.

Offline AUC▲ up

Training pipeline — the evaluation step of the most recent training run

baseline 0.86 on the held-out future period

Online quality▼ down

Model monitoring — precision and recall joined against labels as they arrive

baseline precision 0.71, recall 0.64, measured with a 7-day label delay

p99 latencynormal

Serving dashboard — request latency at the model API, tail percentile

baseline 48 ms

Error ratenormal

Serving dashboard — 5xx and timeout responses at the model API

baseline 0.1%

Prediction meannormal

Prediction logs — mean score over the last hour

baseline 0.21

Feature null ratenormal

Feature store — fraction of served rows with a null in any model feature

baseline 0.4%

Feature distributionnormal

Feature monitoring — PSI of each served feature against the training set

baseline PSI < 0.05 on every feature

Business metric▼ down

Product analytics — the outcome the model was deployed to move

baseline flat, seven-day average

Label lagnormal

Label pipeline — age of the newest label joined to a prediction

baseline 7 days

How to read this page honestly

What the model is, and what it deliberately refuses to be.

SIMULATEDA declarative model of a serving system: ten failures, nine signals, one reading per pair. The readings are what each signal would show — including *unknown* (the labels have not arrived, so quality is unknowable rather than bad) and *missing* (the thing producing the signal is gone). Nothing is measured on a real system.
SIMULATEDSome partial views are genuinely ambiguous. Feature PSI up, prediction mean up, quality down reads identically for real drift and for a serving-side preprocessing change; candidates() returns both, and the page says so instead of pretending the first three signals were enough. Over all nine signals every failure is distinct — which is the argument for the signals that feel redundant.
SIMPLIFIEDReal incidents have several causes at once, readings that are noisy rather than clean directions, and a dashboard that was not built to show the signal you need. The habit that transfers is the order of checks: business metric first, then whether quality is *unknown* or *down*, then serving, then features, then the model — and never the model first.

The lessons behind it

ML Incident Debugging →Model Monitoring →Train / Serve Skew →