SR 11-7 and AI Governance: What the Fed Expects From Your Model Risk Management

Federal Reserve Supervisory Letter SR 11-7 ("Guidance on Model Risk Management"), issued April 4, 2011, defines the Federal Reserve's expectations for model risk management at banking organizations. The OCC issued a parallel document (OCC 2011-12) the same day. SR 11-7 defines a "model" as a quantitative method, system, or approach that applies statistical, economic, financial, or mathematical theories, techniques, and assumptions to process input data into quantitative estimates. Under this definition, every machine learning model deployed in financial services is a model under SR 11-7 — including large language models used for customer service, credit underwriting assistance, or regulatory compliance functions.

What SR 11-7 Actually Requires

SR 11-7's model risk management framework has three core requirements: model development, implementation, and use must be subject to rigorous model validation; there must be a model inventory that captures all models in use; and there must be ongoing monitoring of model performance. For traditional statistical models (logistic regression, linear regression, decision trees), these requirements are well-understood and documented. For machine learning models — particularly foundation models and LLMs — the SR 11-7 requirements create interpretive challenges that examiners are currently resolving through examination findings rather than updated guidance.

The model validation requirement under SR 11-7 has three components: evaluation of conceptual soundness, ongoing monitoring, and outcomes analysis. Conceptual soundness requires documentation of the model's theoretical underpinning, the assumptions that drive its behaviour, and the limitations of those assumptions. For a logistic regression credit model, this is straightforward: the model assumes a linear relationship between input variables and log-odds of default. For an LLM used in credit underwriting assistance, "conceptual soundness" documentation must address transformer architecture assumptions, training data biases, and the relationship between token prediction and the financial decision the model is supporting.

The Stochastic Model Problem

SR 11-7's validation requirement assumes deterministic models: given the same inputs, the model produces the same outputs. This assumption underlies the replication testing approach in SR 11-7 — an independent validator can rerun the model with the same inputs and verify the outputs match. LLMs are stochastic: the same prompt produces different outputs on different runs (at temperature > 0). This breaks the replication testing approach. The SR 11-7-compliant validation methodology for stochastic models requires: distributional testing (does the output distribution match expected characteristics across many runs?), adversarial testing (does the model behave predictably under adversarial prompts?), and bounded output validation (do all outputs fall within acceptable financial decision ranges?).

Ongoing Monitoring Requirements

1. Model performance tracking: for each LLM deployed in a financial decision function, define and track performance metrics that map to the financial outcome — not just accuracy on a test set, but downstream decision quality metrics
2. Data drift monitoring: LLMs are sensitive to input distribution changes; monitor the distribution of inputs to production LLMs and trigger revalidation when the input distribution shifts materially from the training/validation distribution
3. Output distribution monitoring: track the distribution of LLM outputs (decision classes, confidence scores, output lengths) and alert on anomalous shifts
4. Fairness and disparate impact monitoring: SR 11-7 does not use the term "fairness," but the Equal Credit Opportunity Act and Fair Housing Act require that models used in credit and housing decisions not produce disparate impact — monitor for disparate impact continuously, not just at model deployment
5. Revalidation triggers: define specific conditions that trigger model revalidation — input distribution shift beyond a threshold, output distribution shift, adverse outcomes rate change, or material update to the foundation model