Prescio completed a model update, model redevelopment, and model sensitivity analysis of a set of Pre-Provision Net Revenue (PPNR) models for a major U.S. bank. These models, developed in the ARIMA framework, forecast business line balances for commercial loan, deposit, non-interest income, and non-interest expense segments of the bank’s business.

Existing models developed for the CCAR process were updated with current macroeconomic variables. The updated models were tested for statistical soundness. Out-of-time test was conducted on the updated models, and model forecasts under normal and stressed conditions were examined. Models which were found to be no longer statistically sound after the update, or updated models whose out-of-time test or forecasts were poor, were adjusted or redeveloped.

Models were adjusted or developed in a multi-phase process – after each phase of the process, the statistical soundness of the new model, along with the out-of-time tests and forecasts were reassessed. Models that continued to perform poorly proceeded to the next phase in the adjustment process. In the first phase, the model structure was adjusted and the statistical soundness of the adjusted model was reassessed. In the second phase of adjustments, the variables of the model were altered by either dropping variables and/or adding new variables. In the third phase, the modeling methodology was altered to a new framework, and the model was re-estimated.

Once models were successfully updated, adjusted, or redeveloped, the sensitivity of the models was examined. Two aspects of model sensitivity were examined: parameter sensitivity and data input sensitivity. In the parameter sensitivity the model parameters were varied in a systematic way and the output of the models was examined. In the data input sensitivity the independent variables were systematically varied and the output of the model was examined. The variation of the model output with the change in parameters, or the changes in the input data, was quantified and taken as a measure of sensitivity of the model to changes, or misestimation, of the model parameters, or changes in the input variables respectively.