What is model drift?
Model drift is the loss of accuracy a deployed model suffers as the world moves away from its training data. The model itself does not change. Its parameters were fixed when training ended, while customers, prices, products, and processes keep moving. Predictions that were reliable at launch become steadily less so, and the model gives no warning, because it reports wrong answers with the same confidence as right ones. The same problem is also called model decay.
Model drift has two main causes. Data drift is a change in the inputs, such as a new customer segment or a product line the model never saw. Concept drift is a change in the relationship between inputs and outcomes, so the same inputs now lead to a different result. A third cause produces the same symptoms: an upstream system changes a field, a unit, or a default value, and the model quietly receives different data.
Types of model drift
- Sudden drift follows a discrete event, such as a new regulation, an acquisition, or a change in pricing policy.
- Gradual drift builds slowly as customer preferences, equipment, and markets change, and it is the hardest kind to notice.
- Recurring drift follows seasons and business cycles, and it hurts only models whose training data did not cover a full cycle.
Model drift in practice
An auto insurer uses a model to estimate the final cost of each new collision claim so adjusters can set reserves. Over two years the estimates fall further and further below the settled amounts. Two changes are responsible. Parts and labor have become more expensive for the same damage, which is concept drift. More vehicles now carry cameras and radar sensors in bumpers and windshields, which is data drift. Neither change looks alarming in any single month. The insurer now compares estimates with settled amounts every month and retrains when the gap crosses a limit agreed with the claims department.
What to watch for
- Agree before launch on how drift will be measured, which threshold triggers action, and who responds.
- Expect outcome data to lag, because loan defaults and claim settlements can take months to arrive.
- Rule out pipeline changes before blaming the model, since a renamed code or a new unit often looks like drift.
- Treat retraining as recurring work, and use champion-challenger testing to confirm that a retrained model beats the current one before it takes over.