Photo Credit: Victor Habbick

On April 4, 2011, the Office of the Comptroller of the Currency published supervisory guidance on Model Risk Management. The purpose was to provide comprehensive guidance for banks on effective model risk management. The use of models invariably presents model risk, which is the potential for adverse consequences from decisions based on incorrect or misused model outputs and reports. The essence of this guidance is that “garbage in results in garbage out”.

Financial Institutions that utilize AML monitoring systems utilize models in various ways. Examples of AML Models include transaction monitoring, enterprise risk assessment, customer risk rating, and alert/ case scoring. Sound model risk management processes include governance, inventory, development, implementation, and use, tuning, and validation.  

          It is important to ensure your financial institution policies and procedures outline certain aspects of model risk management. The use of surveillance monitoring software presents a risk if the system is not working as designed or is configured incorrectly. Ensuring that the system is configured correctly is a “must” prior to placing any reliance for combating money laundering and/or terrorist financing. A key component in which institutions can confirm that the system is configured correctly is through an outsourced system validation.

Prior to engaging a firm to complete the validation, the institution should complete a thorough review of the firm’s scope of review to determine if it is adequate. As the scope of review of third party vendor validations may differ, institutions should ensure that the scope is comprehensive to satisfy regulatory expectations. At a minimum, the scope of review for a system validation should consist of the following:

1. Transaction Code Mapping Verification: An essential step in this process consists of a “transaction code mapping verification”. This process verifies that ALL transaction codes from the core banking system are properly mapped and coded to the AML system.
2. Transaction Coverage Verification: This process involves the extraction of sample of core transaction data and automated monitoring system transaction. The data from the core system is reconciled to corresponding period automated monitoring system data to determine if there are any significant gaps. The Bank should ensure that the sample selection is comprehensive in size. This is an issue that has been raised by regulators in evaluating the effectiveness of the validation process.
3. Alert Processing Validation:  This process involves an analysis of the configuration of the automated systems alert engine. A sample of alerts should be selected in which the firm will replicate the parameters of the alerts and compare the results against the system determine if output is accurate.

Before the system validation is outsourced to a 3rd party vendor or performed in-house, a best practice for any institution is to conduct parallel monitoring. For example, if the institution was previously using manual transaction monitoring processes (e.g., monthly review of core system generated large cash reports, wire transactions, etc.) and recently acquired a monitoring software application, both monitoring processes should be carried out until the software attestation is complete.

More recently, the regulators have been emphasizing the validation and adequacy of the alerts and thresholds being used by an institution.  They have noted that in many cases these thresholds and the effectiveness of the alerts are not being reviewed periodically. Quite often, it has been noted that there are perhaps too many alerts which result in diluting the overall effectiveness of the monitoring process. Depending upon the size and complexity of an institution, regulators are requesting that Bank review the adequacy of the alerts and threshold on a periodic basis (i.e., annually).  The overall intent of such review is to validate and support the use of the threshold and assumption being used in the monitoring process to ensure that they remain consistent of the bank’s transactional population. Thus, if the average wire transfer transaction is $25,000 and the system alert is set a flag for every transfer in excess of $90,000, this may indicate that the threshold is not perhaps set properly. This review is often referred to the BSA/AML Alerts Optimization.

For more information on Supervisory Guidance on Model Risk Management, click the link here.

Comments