What is shadow mode?
Shadow mode is a deployment stage in which a new model receives the same live inputs as the system it is meant to replace, but its outputs are only recorded. The existing system, which may be an older model, a rules engine, or a human process, keeps making every decision. Afterward the team compares the shadow model’s predictions with what actually happened and with what the current system decided.
Shadow mode differs from a canary deployment, where the new version does act on a share of real traffic, and it usually comes first. It is also the only honest test of a model trained on historical data, because production inputs always differ from the training set.
Shadow mode in practice
A credit union builds a fraud detection model to replace hand-written rules on debit card transactions. The model runs in shadow mode for six weeks. Every transaction is scored, but only the rules engine can block one. Analysts then compare the model’s flagged transactions with confirmed fraud reports. The model would have caught more fraud, but it would also have blocked a cluster of legitimate purchases at one grocery chain, which the team fixes with more training data before a canary deployment.
What to watch for
- Shadow mode doubles inference load, so plan compute for two systems.
- Define the comparison before starting, including how long the period lasts and what result justifies the switch.
- For language model applications, shadow outputs often need human review rather than automatic scoring, which takes staff time.