The Model Lives in a Notebook

Decide what you would do from the brief alone, including whether you would change anything at all. Everything below it is available, but the exercise stops working if you open it first.

The brief you were given

A ticket from support engineering: "The lead-scoring numbers in the CRM have not updated in nine days. The data scientist who ran it is on leave. Can someone re-run the model?" There is a notebook, a CSV on a laptop, and a pickle file with no date.

The trap — the fix that moves the metric and is not the fix

Re-running the notebook top to bottom so the CRM updates, and closing the ticket. The numbers refresh, support engineering is satisfied, and the on-leave engineer returns to find the model retrained on a differently exported CSV with different rows, producing scores that shifted for reasons nobody can explain. The re-run looked like the fix because the symptom was "numbers are stale"; the actual problem was that the scores had no lineage, and re-running made a second untraceable batch.

Read this even if you are confident. It is here rather than behind a button because it is the answer most teams actually ship, it passes review, and its cost arrives weeks later when the labels do.