What is model risk management?
Model risk management (MRM) is the discipline of inventorying, validating, and controlling the models a company relies on for decisions. Model risk is the chance that a model is wrong, or is used wrongly, and that the error causes financial loss, bad decisions, or regulatory trouble. The practice comes from banking. In 2011 the US Federal Reserve and the Office of the Comptroller of the Currency issued supervisory guidance SR 11-7, which told banks how to govern the models behind credit, pricing, capital, and fraud decisions.
The same structure now applies to AI broadly. A language model that triages complaints or a classifier that flags suspicious transactions is a model in the SR 11-7 sense, and supervisors in banking and insurance have said as much.
How model risk management works
SR 11-7 rests on three pillars. The first is sound development, implementation, and use: documented design, tested code, and a clear statement of what the model is for and where it should not be used. The second is independent validation, often called effective challenge, performed by people who did not build the model and who have the standing to reject it. Validation covers conceptual soundness, ongoing monitoring, and outcomes analysis such as backtesting predictions against what actually happened. The third is governance: policies, a model inventory, risk tiering so that scrutiny matches materiality, and board-level reporting.
AI systems strain the framework in known places. A generative model’s output is not reproducible, so backtesting needs sampling and human grading. Vendor models change under the user’s feet, so the inventory must record versions and validation must be repeated on each change. The prompt, the retrieval index, and the model together form the model, and validators have to treat the whole assembly as one unit.
Model risk management in practice
A regional bank wants a language model to summarize customer complaints and route them to the right team. MRM classifies it as a medium-tier model because routing errors delay regulatory response deadlines but do not deny anyone credit. Validation builds a labeled set of five hundred historical complaints, measures routing accuracy, and reads a sample of summaries for omitted facts. The model goes live with monthly accuracy reports and a rule that any version change from the vendor triggers a re-run of the test set.
What to watch for
- Validators need independence and authority, since a review that cannot stop a launch is paperwork.
- Tier models by what they can cause, and spend validation effort on the top tier.
- Write down the intended use so that a model built for one purpose is not quietly reused for another.
- Outside banking, borrow the structure without the examiner’s vocabulary, or business units will ignore it.