Model Review Approach

The Prescio model review approach includes the following 10 items.

Introduction

Reliance on quantitative models in banking and finance has dramatically increased over the preceding several years. The literature continues to grow, producing ever more complex modeling approaches. Increased complexity naturally leads to higher levels of model risk. Identification of model risk is one key goal of the model validation process. A marriage of sound business knowledge and advanced mathematical and statistical techniques, combined with a great deal of imagination, is required to develop sound and accurate models. The Prescio model review approach combines a wealth of business knowledge with mathematical and statistical expertise to provide a complete assessment of a model’s development and implementation.
.
Our model review teams are comprised of leading experts in the banking and finance industries who work alongside mathematicians and statisticians.Our goal is to provide sound judgment in all facets of model development from theoretical derivation of the model to data analysis, customized testing of model fit and output, and regulatory compliance.

Validation vs. Verification

The Prescio model review approach focuses on three main areas– validation, verification, and regulatory compliance. Model validation is an assessment of the choice and output of the modeling exercise. This portion of the review considers all facets of the modeling process from data inputs to final testing of the model output, including an assessment of model choice. Model verification is focused on the mathematical and statistical soundness of the model implementation. This portion of the review considers the mathematical and statistical derivation of the model, the accuracy of the computations, the correct implementation of the model in an appropriate platform, and more. In a complete model review it is necessary to assess the choice and output of the model as well as the implementation of the model.

Model Validation

Our review team conducts a series of comprehensive tests to validate the model’s output at all stages. This begins with a review of the model’s data for completeness, appropriateness, and accuracy. In conjunction with the model verification stage, a line-by-line review and testing of the model code is performed. Prescio provides a series of generally accepted tests for model training and model output. Prescio also designs customized testing procedures specific to the models under review.

Model Verification

Our mathematical and statistical experts use a combination of documentation review, line-by-line review, and testing of the model implementation to verify computational accuracy. In many cases, Prescio has internal mathematical and statistical models for benchmarking purposes during the verification process.

Regulatory Compliance

The Prescio model review process follows all applicable regulatory guidelines. Many of our clients develop Basel II compliant models and also fall under the regulatory umbrella of the OCC. Our model validation team is experienced in Basel and OCC compliance requirements.

Documentation Review

Prescio provides a comprehensive review of the model documentation. Our business experts provide feedback on applicability of the modeling approach and documentation related to current industry best practices. Our mathematical and statistical team provides a review of the mathematical and statistical soundness of the model derivation and its presentation and our team also provides feedback on current mathematical and statistical methodologies. The combination of business and technical documentation review provides valuable insight into the model potential.

Back Testing

The Prescio review team custom designs back testing procedures to verify stability of the model with respect to the data inputs. Stability of the model coefficients is necessary for a model to function appropriately.

Scenario Analysis

The Prescio review team custom designs test scenarios to verify the stability of model outputs. A robust model exhibits stable behavior under different scenarios. Scenario analysis also gives insight into the model potential and its future performance in varying economic conditions.

Stress Testing

The Prescio review team custom designs stress tests in conjunction with the scenario analyses. This pushes the limits of the model and allows us to observe its behavior under extreme economic (or other) circumstances.

Model Review Report

Prescio provides a comprehensive report of all findings during the review process. The report includes a comprehensive documentation review report, a line-by-line code review, and model testing results. The report also includes complete listings of all back tests, scenario analyses, and stress testing output and analyses. A review of the governance and reporting structure of the modeling approach is also provided. A list of suggested improvements and a literature review rounds out our typical model review report.

Related Links