Model risk management applies model-specific controls across inventory, validation, limitations, change management, monitoring, incidents, and retirement. The depth of control should reflect the model's business impact, rate of change, data sensitivity, reversibility, and regulatory exposure.
First, keep reproducible validation evidence, documented limits, material-change triggers, named owners, and clear revalidation conditions. Meanwhile, enterprise governance sets the broad policy and risk framework.
In contrast, model risk management supplies model-specific challenge and evidence. Therefore, set review timing from risk and events, not from a universal external standard.
Document limitations and permitted use
This model card summarizes known limitations, intended uses, and guardrails for safe deployment. Next, use the checklist to confirm whether the model fits your context and to document remaining risks before approval.
Please complete each checklist item with concrete information; leave no items blank for production deployment.
- Intended useDescribe primary tasks, target user groups, and success criteria (what correct behavior looks like).
- Out-of-scope usesList prohibited or discouraged applications and explain why they are inappropriate.
- Performance limitsProvide performance metrics, known failure modes, and scenarios, populations, or data types where accuracy degrades.
- Data limitations and biasSummarize training data sources, known gaps, and potential biases or affected subgroups.
- Safety and privacyIdentify risks to safety, confidentiality, or privacy and describe mitigation measures and residual risks.
- Human oversightDefine required review steps, decision gates, escalation paths, and roles responsible for model outputs.
- Deployment constraintsNote environmental, regulatory, security, or infrastructure requirements and any operational limits (latency, throughput).
- Versioning and contactsRecord model version, release date, owner, and a point of contact for questions or incident response.
Not Every AI Model Creates The Same Level Of Risk
Start with the decision: which systems need strict controls and which need light guardrails. Consider what the model does, who it affects, the data it uses, and how easy it is to recover from failure.
For example, a chat assistant that writes customer-facing contract text needs different controls from an internal document recommender.
Therefore, enterprise programs should map models to business processes, not just technical categories. A predictive model used to approve loans touches compliance and financial risk, while the same algorithm used for internal prioritization does not.
As a result, capture process impact in the inventory so risk tiering reflects organizational exposure, not developer intent.
Moreover, proportionate governance reduces review bottlenecks and preserves engineering agility while ensuring critical systems are validated and monitored.
Define explicit thresholds for when a model enters a higher control band so product teams can design to meet or avoid those thresholds depending on strategy.
- Assess risk by harm to customers, finances, operations and reputation.
- Include recoverability: how quickly can the business contain a failure?
- Differentiate customer-facing outputs from internal decision support.
- Make tier thresholds explicit to guide design and procurement choices.
Classify models by business consequence first, technology second.
For the security and privacy controls that support this delivery plan, review data security and privacy in enterprise AI.
What Model Risk Means For Enterprise AI
Model risk management addresses whether a particular model or model-enabled system remains valid, understood, controlled, and safe to operate. Portfolio policy, approval forums, and broader values are handled by the governance framework, operating model, and responsible-AI practice.
Model risk arises from design choices, training and operational data quality, integration errors, user interaction patterns and changes after deployment.
Consequently, each source maps to different controls: design and data require validation and lineage; integration needs interface testing and access controls; operational data problems need monitoring and retraining policies.
In addition, dependence on external models and APIs creates supply-chain risk. Document third-party components, their versioning, vendor SLAs and whether the provider offers model cards or security attestations.
Therefore, where external models are material to outcomes, treat them like internal models for validation and incident response planning.
However, distinguish model risk from broader responsible AI concepts such as fairness policy or ethics statements. Responsible AI guides values and policy choices; model risk management operationalizes exposure reduction and resilience through tests, thresholds and approvals.
Four sources of model risk
| Risk source | What it covers |
|---|---|
| Design risk | Architecture, loss functions and objective misalignment issues. |
| Data risk | Sample bias, leakage, label quality and training drift. |
| Integration risk | Input sanitation, API misbehavior and dependency failure. |
| Change risk | Untracked retraining, version swaps and silent rollouts. |
Model risk combines technical faults and operational exposure that produce real business harm.
For the lineage controls that connect data evidence to AI decisions, review AI data lineage.
Create A Complete AI Model Inventory
A usable inventory records model name, owner, version, deployment state, primary purpose and principal users. Capture business process mappings such as downstream systems, customer touchpoints and regulatory domains.
Also, include the location of code, training data references and the compute environment. This information helps operational teams locate artifacts when an incident occurs.
Next, record dependencies: upstream data sources, feature stores, third-party models and service endpoints.
Link to data lineage artifacts described in the enterprise data and AI modernization pages so teams can trace inputs back to authoritative sources and evaluate data sensitivity and quality for each model.
Furthermore, assign clear owners for model lifecycle stages: development, validation, deployment and support. Owners must update inventory entries after a version release, major retrain, architecture change or decommissioning.
Finally, automate population where possible using CI/CD hooks or model registry integration to avoid stale records.
- Minimum fields: name, owner, purpose, version, status and deployment date.
- Include data lineage links to upstream datasets and feature stores.
- Track third-party components and provider version identifiers.
- Automate inventory updates from CI/CD and model registries where possible.
An accurate inventory is the single most effective control for scalable model governance.
Classify Models By Impact And Risk
Set Clear Risk Tiers And Approval Paths
Use a simple risk-tier matrix. Score business importance, autonomy, effect on customers, data sensitivity, and ability to recover. Use three or four tiers: low, medium, high, and critical.
Then, give each tier its own controls and approval path. Teams will know what checks and monitoring are needed before launch.
For example, practical cases help standardize classification. A critical tier includes automated credit decisions with financial impact. In contrast, a high tier includes customer-facing chat agents that influence contract language.
Meanwhile, medium risk covers internal prediction services, while low risk covers non-decision internal helpers.
Moreover, anchor tiers with examples relevant to the enterprise to reduce subjective assessments.
Next, store tier decisions in the model inventory and require risk-owner sign-off for high and critical models.
However, business context changes over time. Therefore, conduct periodic tier reviews. For external models, include a vendor risk grade and treat vendor upgrades as material changes that require reclassification.
Match Controls To Each Risk Tier
Risk tiers, required controls and approval paths
| Risk tier | Typical impact | Example | Required controls | Approval |
|---|---|---|---|---|
| Low | Limited internal impact; outputs are advisory and easy to reverse. | Internal document tagging or search support. | Inventory entry, owner, baseline testing, access control and basic logging. | Model owner or product owner. |
| Medium | Influences internal work or operational priorities but does not make final decisions. | Demand forecast or internal prioritization model. | Documented validation, monitoring thresholds, change log and human review. | Model owner and business process owner. |
| High | Can affect customers, contracts, sensitive data or regulated processes. | Customer-facing assistant that drafts contract terms. | Independent validation, bias and security tests, human oversight, staged release and incident plan. | Business risk owner, compliance and model validation lead. |
| Critical | Can create major financial, legal, safety or rights-related harm. | Automated credit decision or safety-critical control. | Full independent challenge, explainability, shadow testing, continuous monitoring, rollback and executive escalation. | Executive risk authority with legal or regulatory sign-off. |
Tie tiering to concrete business examples so classification is repeatable and auditable.
Validate Models Before Approval
Test The Model Against Its Stated Limits
Validation covers performance, robustness, limitations, bias and security. For each model tier define required tests: holdout performance metrics, stress tests for edge cases, adversarial resilience checks and bias audits against relevant protected attributes.
In addition, require documentation of known limitations and expected operating ranges before sign-off.
Similarly, explainability and human oversight requirements scale with tier. Therefore, high and critical systems need interpretable explanations for decisions and defined human-in-the-loop protocols.
Furthermore, validate that logging and traceability support post hoc analysis. Store inputs, model version, outputs and decision rationale where feasible and legally permitted.
Likewise, test security for models that handle sensitive data or control critical actions. Test APIs, check training artifacts for data leaks, and confirm access controls and encryption meet company standards.
Finally, link the evidence to the governance framework and data-security guidance for audit.
Document Validation Evidence And Regulatory Context
Validation tests and evidence required before approval
| Validation area | Test method | Evidence | Acceptance criteria | Owner |
|---|---|---|---|---|
| Performance | Evaluate holdout data, calibration and relevant business scenarios. | Metric report, confidence intervals and benchmark comparison. | Meets the approved metric thresholds across priority use cases. | Independent validator with model owner support. |
| Robustness | Run edge-case, stress, distribution-shift and adversarial tests. | Scenario results, failure analysis and residual-risk record. | Failure rates remain within limits and recovery controls work. | Validation and engineering leads. |
| Limitations | Test stated operating boundaries and prohibited-use scenarios. | Model card, limitation register and permitted-use statement. | Limits are reproducible, documented and enforced through guardrails. | Model owner and business process owner. |
| Bias and fairness | Compare relevant performance and outcome metrics across affected groups. | Segmented metrics, gap analysis and mitigation results. | Results meet the approved fairness thresholds or have an accepted treatment plan. | Responsible AI, compliance and validation leads. |
| Security | Assess APIs, access controls, prompt attacks and data leakage paths. | Security test report, vulnerability record and remediation proof. | No unresolved critical findings; required controls are verified. | Security lead and system owner. |
Validation must produce reproducible artifacts and a clear statement of limitations before approval.
For current US supervisory guidance on a tailored, risk-based model risk approach, see the Federal Reserve's SR 26-2 letter, which supersedes SR 11-7.
Applicability note: SR 26-2 is US banking supervisory guidance. It is most relevant to Federal Reserve-regulated banks with more than $30 billion in assets.
The guidance says that generative and agentic AI models are outside its scope. Use it as a banking model-risk reference, not as a universal AI standard.
For organizations outside banking, use the NIST AI Risk Management Framework (AI RMF) as the broader risk-management reference.
Similarly, apply it to generative or agentic AI systems. Its Govern, Map, Measure and Manage functions help teams identify, assess and control AI risks across industries and use cases.
Control Versions, Changes And Releases
Define material changes that require revalidation: data schema shifts, new training data sources, architecture changes, major hyperparameter retuning, or new third-party model versions.
For material changes, require the same validation artifacts as initial approval. In addition, document approval from the model owner and a business risk approver.
Next, use a release policy that enforces staged rollouts, canary traffic splitting and rollback plans for higher tiers. For critical models, require pre-production shadow runs and acceptance criteria tied to operational metrics.
Furthermore, automate version tagging in the model registry and ensure production systems reference immutable model identifiers to avoid accidental swaps.
Moreover, maintain change logs that record who approved each release, test artifacts, and the planned rollback procedure.
For external models treat provider upgrades as a release event and include contractual requirements for notification windows and reproducible testing in procurement agreements where possible.
- Define material change categories that trigger revalidation and approval.
- Use staged deployment, canaries and traffic controls for higher tiers.
- Require immutable model identifiers and versioned artifacts in production.
- Keep a signed change log with approval, tests and rollback steps.
Treat model releases like software releases with clear gating and rollback capabilities.
Monitor Risk After Deployment
Connect observability signals to risk thresholds and escalation rules. Signals include input distribution drift, output distribution changes, confidence score shifts, operational latencies and key business KPIs.
Then, map each signal to alert thresholds and define who is responsible for triage at each severity level so detection leads to action.
Moreover, monitoring must include business-side telemetry as well as technical metrics. For example, a drop in customer retention immediately after a model change is as important as a performance metric.
In addition, integrate model observability with existing monitoring platforms and operational dashboards so business owners can see model impact in context.
Therefore, set review and revalidation timing from risk, material changes, and incidents. Do not use one universal minimum. Low-impact models may need a lighter schedule.
In contrast, high-impact models need closer monitoring, clear change triggers, and formal revalidation after a material change. Also, keep monitoring results, incidents, and revalidation decisions in the model record for audit and improvement.
Monitoring signals and ownership
| Signal area | What to track |
|---|---|
| Observability | Input and output distribution drift sensors with thresholds. |
| Business telemetry | Customer complaints, KPI shifts and SLA breaches. |
| Alerting and triage | Roles, response time targets and escalation paths. |
| Scheduled reviews | Cadence tied to risk tier and recent changes. |
Monitoring connects technical signals to business outcomes and triggers revalidation when thresholds are exceeded.
Manage Incidents, Exceptions And Retirement
An incident flow should cover containment, impact assessment, remediation, disclosure and decommissioning. Containment options include circuit breakers, traffic throttles and temporary disables. Define who can enact containment for each model tier and document expected timelines for initial mitigation actions.
Next, the impact assessment must capture scope, affected populations, regulatory or contractual exposures, and required remediation steps.
For customer-facing incidents, include communications templates and an escalated legal and compliance review. Subsequently, remediation may require model retraining, data correction, or a rollback to a validated version.
Eventually, models reach the end of life and need retirement plans. At that point, archive model artifacts and permitted training data snapshots. Finally, update the inventory.
Finally, capture lessons learned in a post-incident review. Feed them back into validation criteria and tiering guidance to reduce recurrence.
- ContainmentCircuit breakers, disable endpoints and traffic throttling options.
- AssessmentScope, affected users, contractual and regulatory exposure checklist.
- RemediationRollback, retrain, data fixes and verification tests.
- RetirementArchive artifacts, update inventory and document lessons learned.
Prepare incident playbooks that specify containment and remediation authority by tier.
Frequently Asked Questions
Model risk management focuses on measurable exposures that can cause business harm, such as financial loss, customer impact or operational disruption.
Responsible AI covers policy, ethics and values. The two overlap, but model risk operationalizes controls like validation, monitoring and incident response so policy commitments become enforceable practices.
Treat a third-party model like an internal model when it materially affects decisions, customer outcomes or regulatory exposure. Require vendor versioning, testing artifacts, service guarantees and inclusion in your inventory.
For critical reliance, include contractual notification windows for upgrades and the right to audit or reproduce validation tests.
Low-tier models still need an inventory entry, baseline performance evidence, logging, access control, and a named owner.
Use a review cadence that fits the model's impact, rate of change, and operational context, and define clear triggers to escalate if performance degrades or the model is repurposed for higher-impact use.
High-risk models need closer monitoring and documented challenge and validation evidence. Revalidate after a material change, an incident, or a meaningful shift in data or use.
Teams may review these models often, but the cadence must be a documented risk-based control, not a universal minimum. Put the triggers in change control so revalidation happens consistently.
