Regulators are expecting firms to invest more in governance and controls to ensure Model Risk Management standards are maintained.
In 2011, the Federal Reserve and Office of the Comptroller of the Currency (OCC) released a Supervisory Guidance on Model Risk Management for use by banking organisations and supervisors to evaluate organisations' management of model risk.
Similar guidance has been introduced by other regulators, such as:
Managing model risk is important because of the potential for adverse consequences resulting from decisions based on incorrect or misused model outputs and reports. These include:
The SR11-7 paper outlines supervisory guidance for:
The guidance was intended to be applied to all banking organisations that fall under the Federal Reserves supervision,
Model risk management involves managing, measuring, and mitigating the risks of adverse consequences resulting from decisions made based on incorrect or misused models.
A "model" refers to a quantitative approach that uses statistical, economic, financial, or mathematical theories, techniques, and assumptions to convert input data into quantitative estimates.
Models are typically quantitative, rely on expert judgement, and can be created from various technologies, including C++, Excel spreadsheet, Matlab, Python, R, SAS, and SQL.
Models are simplified representations of real-world scenarios used extensively within financial institutions.
Financial institutions might use them for a variety of purposes, including:
As models become increasingly complex, there is higher uncertainty about a model's inputs; it's assumptions, broader use and a more significant potential impact on business decisions. The regulatory guidance outlines two primary reasons for model risk:
To identify, assess, and mitigate model risk, a MRM framework should oversee the design, development, validation, implementation, controls, and ongoing monitoring of models.
A robust model risk management framework that aligns with the organisation's broader risk management framework should be developed by a bank's senior management and board as part of their overall responsibilities.
Financial institutions are expected to maintain a model risk management framework approved by the Board and managed by senior management, which should provide regular reporting and oversee the execution to ensure ongoing compliance.
The regulator has advised that firms should manage model risk through an active model risk management framework. The model risk oversight process should be independent, and governance will require ongoing maintenance to reflect changes in how models are used and the potential for new model risk exposure. For Example:
Model risk management requires that firms manage risk throughout each model's lifecycle, from development to retirement. There is no prescribed way to achieve this. However, effective MRM lifecycle management ensures that the model risk is appropriately managed and mitigated and aligns with an organisation's overall risk management framework.
An example lifecycle management programme might include:
Model Risk Management controls should ensure a disciplined approach to model development and implement processes consistent with model users' and business objectives.
As models are developed, documentation should be created to ensure that the model use is appropriate and that users understand how to utilise models correctly.
Identify and build a model inventory which outlines what models exist, how complex models are, and where and how each model is used.
Model development relies heavily on the experience and judgement of its creators. Therefore, it is essential to conduct comprehensive validation to ensure that models perform as intended. To identify model limitations, tests should be independent and conducted to assess each model's performance, stability, and data relationships.
Classify and approve models based on their risk levels, such as complexity and materiality. Quantitative models and techniques can be used to measure risk based on data inputs, variables, or potential impact from errors.
Continuous review from internal risk management functions to oversee that model developers maintain disciplined model development practises and that actively used models are registered in an up-to-date model inventory.
As market conditions necessitate adjustment, risk management functions should provide guidance on model risk to ensure that integrity is maintained and models remain fit for purpose.
The active decommissioning and retirement of models when they are no longer required or no longer suitable for their intended purpose to ensure the
If you don't establish a robust governance model for managing risk, when things go wrong, the consequences can be severe. For example:
The impact of inadequate controls can be severe and can include the following:
Regulators advise the model risk should be effectively challenged and managed like any other type of risk. For example, this requires risk management stakeholders to assess the following:
Furthermore, regulators advise that model risk should be assessed in the aggregate. This means there should be oversight across the inputs and dependencies across the model inventory. For example, models may depend upon data from other files or sources, so broader visibility over data lineage is also important.
Users rely on a wide-ranging use of technologies to build and deploy models across financial institutions. The trend has recently shifted towards utilising AI and ML within python models. However, end-user computing applications such as spreadsheets are still widely used as modelling tools.
The SR 11-7 guidance has highlighted that "user-developed applications, such as spreadsheets or ad hoc database applications used to generate quantitative estimates, are particularly prone to model risk."
The flexibility that spreadsheets offer is often a double-edged sword in managing risk. Identifying spreadsheet models is challenging because it is easy for models to be created and shared. The ongoing audit of spreadsheet models is difficult because models are so flexible and easy to manipulate, which creates the potential for human error or intentional tweaking of variables.
Utilising platforms like Workscope can reduce the time and resources it takes to establish model risk management controls for spreadsheets and other EUC models. This is achieved through automated discovery, risk assessment and the subsequent classification of risk.
For more information, please get in touch via info@workscope.com