SR 11-7 Compliance & Model Risk Management

In 2011, the Federal Reserve and Office of the Comptroller of the Currency (OCC) released a Supervisory Guidance on Model Risk Management for use by banking organisations and supervisors to evaluate organisations' management of model risk.

Similar guidance has been introduced by other regulators, such as:

1. The Prudential Regulation Authority at the Bank of England introduced the Model risk management principles for stress testing in 2018
2. The European Central Bank (ECB) released the Guide to Internal Models in October 2019
3. The Hong Kong Monetary Authority (HKMA) released the Supervisory Review Process (CA-G-5) in 2020.

Managing model risk is important because of the potential for adverse consequences resulting from decisions based on incorrect or misused model outputs and reports. These include:

• Financial loss.
• Poor business or strategic decision-making
• Damage to a banking institution's reputation.

What is sr11 7?

The SR11-7 paper outlines supervisory guidance for:

• Model Risk Management
• Model Development, Implementation, and Use
• Model Validation
• Governance, Policies, and Controls

The guidance was intended to be applied to all banking organisations that fall under the Federal Reserves supervision,

What is Model Risk Management (MRM)?

Model risk management involves managing, measuring, and mitigating the risks of adverse consequences resulting from decisions made based on incorrect or misused models.

What is a model?

A "model" refers to a quantitative approach that uses statistical, economic, financial, or mathematical theories, techniques, and assumptions to convert input data into quantitative estimates.

Models are typically quantitative, rely on expert judgement, and can be created from various technologies, including C++, Excel spreadsheet, Matlab, Python, R, SAS, and SQL.

How Financial institutions use models

Models are simplified representations of real-world scenarios used extensively within financial institutions.

Financial institutions might use them for a variety of purposes, including:

• Valuing positions
• Identifying and measuring risks
• Developing investment/trading strategies
• Assessing capital adequacy (incl. Basel III)
• Conducting stress testing (e.g., DFAST and CCAR)
• Meeting financial/regulatory reporting requirements (e.g., SOX)

Reasons for model risk

As models become increasingly complex, there is higher uncertainty about a model's inputs; it's assumptions, broader use and a more significant potential impact on business decisions. The regulatory guidance outlines two primary reasons for model risk:

• A model may have fundamental errors and produce inaccurate outputs when viewed against its design objective and intended business uses.
• For example, this could be related to incorrect data inputs, broken or inaccurate formulas, typos, or other variables that could impact a model's forecasts or output.
• A model may be used incorrectly or inappropriately, or there may be misunderstandings about its limitations and assumptions.
• For example, a model intended to forecast growth rates does not account for volatility and current market conditions, or a model designed for pricing short-term debt instruments is used for calculating other securities that may require a fundamentally different approach.

Establishing a Model Risk Management framework

To identify, assess, and mitigate model risk, a MRM framework should oversee the design, development, validation, implementation, controls, and ongoing monitoring of models.

A robust model risk management framework that aligns with the organisation's broader risk management framework should be developed by a bank's senior management and board as part of their overall responsibilities.

Financial institutions are expected to maintain a model risk management framework approved by the Board and managed by senior management, which should provide regular reporting and oversee the execution to ensure ongoing compliance.

Model Risk Management Governance

The regulator has advised that firms should manage model risk through an active model risk management framework. The model risk oversight process should be independent, and governance will require ongoing maintenance to reflect changes in how models are used and the potential for new model risk exposure. For Example:

• Increasingly, the use of models has become more widespread and complex, and the focus of MRM has shifted from individual model management to enterprise-wide model risk management.
• More recently, artificial intelligence and machine learning techniques are increasingly being adopted, introducing specific risks such as bias, interpretability and the validation of machine learning models.

Lifecycle Management

Model risk management requires that firms manage risk throughout each model's lifecycle, from development to retirement. There is no prescribed way to achieve this. However, effective MRM lifecycle management ensures that the model risk is appropriately managed and mitigated and aligns with an organisation's overall risk management framework.

An example lifecycle management programme might include:

Model Development

Model Risk Management controls should ensure a disciplined approach to model development and implement processes consistent with model users' and business objectives.

As models are developed, documentation should be created to ensure that the model use is appropriate and that users understand how to utilise models correctly.

Model inventory

Identify and build a model inventory which outlines what models exist, how complex models are, and where and how each model is used.

Model validation

Model development relies heavily on the experience and judgement of its creators. Therefore, it is essential to conduct comprehensive validation to ensure that models perform as intended. To identify model limitations, tests should be independent and conducted to assess each model's performance, stability, and data relationships.

Classification & Monitoring

Classify and approve models based on their risk levels, such as complexity and materiality. Quantitative models and techniques can be used to measure risk based on data inputs, variables, or potential impact from errors.

Model follow-up

Continuous review from internal risk management functions to oversee that model developers maintain disciplined model development practises and that actively used models are registered in an up-to-date model inventory.

As market conditions necessitate adjustment, risk management functions should provide guidance on model risk to ensure that integrity is maintained and models remain fit for purpose.

Retirement

The active decommissioning and retirement of models when they are no longer required or no longer suitable for their intended purpose to ensure the

Why financial institutions should care about model risk

If you don't establish a robust governance model for managing risk, when things go wrong, the consequences can be severe. For example:

• Due to the highly aggressive misuse of a risk-hedging tool, a global bank exceeded its value-at-risk limits for almost a week. Although the bank eventually identified the risk, it failed to change its investment strategy because the risk model was inadequately governed and validated. Instead, the bank only adjusted control parameters. This led to a significant loss that amounted to billions of dollars.

Consequences of inadequate model risk management

The impact of inadequate controls can be severe and can include the following:

• financial loss
• misinformed decision making
• regulatory fines
• cease-and-desist orders
• damage to a financial institution's reputation

How to identify model risk

Regulators advise the model risk should be effectively challenged and managed like any other type of risk. For example, this requires risk management stakeholders to assess the following:

• sources of risk
• model complexity
• potential magnitude of risk
• model limitations and assumptions

Furthermore, regulators advise that model risk should be assessed in the aggregate. This means there should be oversight across the inputs and dependencies across the model inventory. For example, models may depend upon data from other files or sources, so broader visibility over data lineage is also important.

How platforms like Workscope support Model Risk Managment

Users rely on a wide-ranging use of technologies to build and deploy models across financial institutions. The trend has recently shifted towards utilising AI and ML within python models. However, end-user computing applications such as spreadsheets are still widely used as modelling tools.

The SR 11-7 guidance has highlighted that "user-developed applications, such as spreadsheets or ad hoc database applications used to generate quantitative estimates, are particularly prone to model risk."

The flexibility that spreadsheets offer is often a double-edged sword in managing risk. Identifying spreadsheet models is challenging because it is easy for models to be created and shared. The ongoing audit of spreadsheet models is difficult because models are so flexible and easy to manipulate, which creates the potential for human error or intentional tweaking of variables.

Utilising platforms like Workscope can reduce the time and resources it takes to establish model risk management controls for spreadsheets and other EUC models. This is achieved through automated discovery, risk assessment and the subsequent classification of risk.

Interested in deploying Workscope for Model Risk Management?

For more information, please get in touch via info@workscope.com

‍