ML Observability
A healthy model server can serve a wrong model indefinitely. Observability for a model means tracing Request → Features → Model Version → Prediction → Decision → Outcome, and watching signals service health does not have.
The problem, the obvious approach, and why it breaks
Every lesson starts where the work starts: someone has a problem, and the first model that comes to mind looks fine offline.
The serving dashboards are green and the model is wrong. What signals does a model need beyond service health, and how does one prediction get traced to its outcome?
Our fraud model runs behind an API with full observability — latency, errors, saturation, traces. Last month it flagged almost nothing for nine days and every dashboard was green. We found out from the chargeback report.
A model is a service, so observe it like one: golden signals, traces, SLOs. If the API is fast, error-free and within capacity, the model is healthy. Quality is a separate concern for the data science team's monthly report.
The nine-day incident produced no error, no latency change and no saturation. The model returned a valid score for every request; the score was wrong. Service telemetry is blind to a wrong-but-well-formed response by construction.
- The nine-day incident produced no error, no latency change and no saturation. The model returned a valid score for every request; the score was wrong. Service telemetry is blind to a wrong-but-well-formed response by construction.
- When the chargeback report arrived, the question was "what did the model see for these transactions?" — and nobody could answer, because the feature fetch, the score and the outcome had no shared key.
- The prediction mean had halved on the first day. Nobody had a panel for it, because prediction mean is not a service metric.
- The quality panel, when someone built it, was flat for the nine days, because the chargebacks that would have shown the damage had not arrived yet. Flat meant unmeasured, and it read as fine (Ground-Truth Delay).
What is being predicted, and from what data
This domain leads with these two. A target nobody defined precisely is a label nobody can trust, and a dataset nobody can describe is a model nobody can debug.
- Predict whether a transaction will be charged back; the decision is approve, review or decline; the outcome is a chargeback that arrives up to 90 days later.
- The observability target is not the API. It is the chain from the request to the outcome, with every hop recorded so that a wrong decision can be traced to the hop that made it wrong.
- The API emits request logs, latency histograms, error counts and distributed traces — the standard service telemetry, all of which describe whether a response was returned and how fast.
- The model server logs a score per request. The feature service logs a fetch per request. The decision service logs an action. The chargeback pipeline writes outcomes to a warehouse table. None of these shares an id.
- During the nine days, a feature was served as null after an upstream column rename; the model imputed the training-set mode and scored everything low; the threshold turned "low" into "approve".
How it actually works
Precisely enough to predict its behaviour — not a framework API.
- A model can fail in ways the service around it cannot see: its inputs can move (feature drift, null spikes, skew), its outputs can move (prediction drift), its decisions can move (a threshold change) and its correctness can move (concept drift, decay) — each without a single error or slow response. Each needs its own signal, at the boundary where it happens (Model Monitoring).
- The signals form a chain that mirrors the request's path. Request telemetry says a request arrived. Feature telemetry says what it was scored on. Model-version telemetry says which weights scored it. Prediction telemetry says what score came out. Decision telemetry says what the score became. Outcome telemetry — late — says whether that was right. A break anywhere in the chain is a hop that cannot be checked.
- The chain is joined by a correlation id carried from the request through every hop and attached to the outcome when it arrives. Without it, the signals are six separate time series that can only be compared by eye; with it, one prediction can be reconstructed and the population can be sliced by any hop — this model version, this feature value, this decision — against outcomes.
What green cannot see
The golden signals — latency, traffic, errors, saturation — describe whether the service is doing its job as a service: returning responses, on time, without failing, within capacity. A model server that scores every transaction low because one feature is null is doing that job perfectly. Its response is valid JSON, fast, and wrong.
The model's failures live one level down, in the values. The input values moved (a null where a number should be); the output value moved (a halved prediction mean); the decision moved (the flag rate collapsed); and eventually the correctness moved (chargebacks on approved transactions). Each is a signal, and none of them is a service signal.
| Signal | Where it comes from | What it catches | Delay |
|---|---|---|---|
| Latency, errors, saturation | API and model server | Outages, capacity, timeouts, a slow feature store | Seconds |
| Feature null rate, feature distribution | Feature boundary, per feature | A broken feature, skew, a population shift | Minutes to hours |
| Prediction mean and distribution | Prediction log | Anything that moved the inputs or the weights | Minutes to hours |
| Flag rate per decision | Decision service | A threshold or policy change the scores cannot show | Minutes to hours |
| Fallback rate | Serving path | Degraded decisions hidden inside healthy responses | Minutes |
| Label lag | Outcome pipeline | Whether quality is measurable at all | Immediate |
| Quality against labels | Prediction log joined to outcomes | Whether the model is right | The label delay |
The chain
The nine-day incident was undiagnosable for a simple reason: the question "what did the model see for this transaction?" could not be answered. The feature fetch was logged in one service, the score in another, the decision in a third, and the chargeback landed in a warehouse table keyed on something else. Six records of one event with no way to join them.
The fix is one id, minted at the edge and carried through every hop, attached to the outcome when it arrives. Then one prediction can be reconstructed — and, more usefully, the whole population can be sliced by any hop against outcomes: quality per model version, per feature bucket, per decision, per fallback rung.
What must hold for the model to be observable
Observability for a model is a property of the chain, not of any dashboard. If the id is minted late, the early hops are invisible; if the outcome uses a different key, quality is computed on whatever happens to join; if label lag is not shown, a flat quality line is misread as health. Each of those is an assumption the team is making without noticing, and each can be checked by trying to reconstruct one prediction.
The assumption device below is the one the domain rests on: that a wrong decision can be traced to the hop that made it wrong. When it holds, an incident is an afternoon. When it does not, it is a chargeback report and a guess.
For any request id, the features it was scored on, the model version, the score, the decision and — once it exists — the outcome can be retrieved and joined.
holds when The id is minted at the edge and propagated through every service; each hop writes its record keyed on it; the outcome pipeline carries it or a deterministic mapping to it; retention on the prediction log outlasts the label delay.
breaks when A service in the chain drops the header; the outcome is keyed on a different entity; the prediction log is sampled or expires before labels arrive; a fallback path serves a decision without writing a record.
respond Fix the hop that is missing before anything else; a model incident with a broken chain cannot be diagnosed, only guessed at, and the guess is usually "retrain".
1select r.request_id, r.received_at,2 f.feature_snapshot_ref, -- a reference into the feature log, not raw values3 p.model_version, p.score,4 d.action, d.threshold_policy, d.fallback_rung,5 o.chargeback, o.observed_at -- null until the label arrives6from requests r7join feature_fetches f using (request_id)8join predictions p using (request_id)9join decisions d using (request_id)10left join outcomes o using (request_id)11where r.request_id = :id;The left join on outcomes is the honest part: for a recent request the outcome is null, and a quality metric that inner-joins here silently computes itself on old requests only. The fallback_rung column is what tells you a healthy-looking score came from the cache.
How to build it
Most important first.
- Instrument the chain, not the service: emit a record at each hop — request, features (as references), model version, score, decision, and later outcome — all keyed on one request id (Tracing a Prediction, Prediction Logging).
- Add the model signals to the dashboard next to the service signals: prediction distribution and mean, flag rate (the fraction of decisions that are positive), per-feature null rate and distribution distance, fallback rate, label lag, and quality against arrived labels. The failure simulator at
/ml/failuresuses nine of these and shows which failures each one can and cannot see. - Alert on the leading signals — null rate, prediction mean, flag rate — with thresholds set from the model's own history, because these move on day one while quality moves after the label delay.
- Show label lag on the quality panel itself, so a flat line reads as "unmeasured since day N" rather than as "fine". Reuse the observability domain's tracing, structured logging and SLO practice for the mechanics; the model adds hops and signals, not a new stack.
What to measure
Which number actually maps to the decision — and which numbers look relevant and are not.
- Per-feature null rate and distribution distance at the serving boundary; prediction mean and distribution; flag rate per decision; fallback rate. These are the signals that fired on day one of the nine days, had they existed.
- Quality against arrived labels, plotted with label lag, per model version. This is the number the business cares about and the last one to move.
- API latency, errors and saturation are necessary and say nothing about the model. Green is not a model signal.
What must stay true after deployment
The field this whole domain exists for. A model is a set of assumptions with weights attached; these are the ones a monitor or a test should be checking.
- Every hop in the chain — edge, feature service, model server, decision service, outcome pipeline — carries and records the same request id, so that one prediction can be reconstructed end to end.
- The leading signals (null rate, prediction mean, flag rate) have thresholds set from this model's history, and a breach reaches someone who can act within the day.
- Label lag is visible on the quality panel and a flat quality line is read as unmeasured, not as healthy.
- Offline: pick a request id from last week and reconstruct the prediction from logs alone — features, version, score, decision, outcome. If any hop is missing, the chain is broken there.
- Online: inject a known failure in staging — null a feature, move the threshold — and confirm the corresponding signal fires within its expected delay, while service telemetry stays green.
- Over time: once labels arrive, slice quality by every hop — model version, feature bucket, decision, fallback rung — and confirm the slicing is possible; that it is possible is the test of the chain.
What can go wrong
- The request id is minted at the model server rather than at the edge, so the trace begins after the feature fetch and the hop that failed is the one not traced.
- The prediction log and the outcome table use different keys — transaction id in one, order id in the other — and the join that would compute quality is approximate, so quality is computed on the subset that joins, which is not random.
- The dashboard has all the model signals and forty alerts; the null-rate alert fires daily on a feature that is legitimately sparse, and the one that mattered was muted with it (Alert Fatigue: The Page Nobody Reads in the observability domain describes the mechanism).
- Six signals per model, each with a threshold, is an alerting policy per model to author and tune; the alternative is a model that fails silently for nine days.
- Carrying a request id through five services owned by three teams is an organisational agreement, not a code change, and the outcome pipeline is usually the team that never agreed to it.
- Logging the feature vector per request is the storage and privacy cost that makes the chain diagnosable; references instead of raw values reduce it and lose some diagnosability.
- "The dashboards are green, so the model is fine." The dashboards say the service returned responses quickly. A wrong score is a fast, well-formed response.
- "Quality is flat, so nothing changed." Quality is computed on labels that have arrived; if none have, the line is flat because it is empty. Label lag has to be on the same panel.
- "We have traces, so we have observability." A trace shows the request's path through services. It does not show the feature values, the model version or the outcome unless someone put them there, and it ends when the response is returned — weeks before the outcome.
Where this applies
ML advice is stated as universal far more often than it is. These labels say what each claim is specific to — and where CONTESTED appears, the note gives the strongest form of the opposing view.
- GENERALThat service telemetry cannot see a wrong-but-well-formed response, and that the model therefore needs input, output, decision and outcome signals of its own, holds for every deployed model regardless of family or task.
- SCALE-SPECIFICFor a batch model scoring nightly, the chain is a handful of tables and the request id is a row key; the tracing problem is trivial and the signals are computed per batch. The hard version is online inference across services owned by several teams, where the id has to be propagated and the outcome joined back weeks later.
- SIMULATEDThe failure simulator this lesson refers to runs a declarative model of a serving system; its nine signals and their baseline values are constructed to make the diagnostic argument, not measured on any real service.
Where the depth lives
This domain teaches the model and hands the rest off by name.