12 C.F.R. § 252 app B to Part 252
This Policy Statement describes the principles, policies, and procedures that guide the development, implementation, and validation of models used in the Federal Reserve's supervisory stress test.
1. Principles of Supervisory Stress Testing
The system of models used in the supervisory stress test is designed to result in projections that are (i) from an independent supervisory perspective; (ii) forward-looking; (iii) consistent and comparable across covered companies; (iv) generated from simpler and more transparent approaches, where appropriate; (v) robust and stable; (vi) conservative; and (vii) able to capture the impact of economic stress. These principles are further explained below.
1.1. Independence
(a) In the supervisory stress test, the Federal Reserve uses supervisory models that are developed internally and independently (i.e., separate from models used by covered companies). The supervisory models rely on detailed portfolio data provided by covered companies but do not rely on models or estimates provided by covered companies to the greatest extent possible.
(b) The Federal Reserve's stress testing framework is unique among regulators in its use of independent estimates of losses and revenues under stress. These estimates provide a perspective that is not formed in consultation with covered companies or influenced by firm-provided estimates and that is useful to the public in its evaluation of covered companies' capital adequacy. This perspective is also valuable to covered companies, who may benefit from external assessments of their own losses and revenues under stress, and from the degree of credibility that independence confers upon supervisory stress test results.
(c) The independence of the supervisory stress test allows stress test projections to adhere to the other key principles described in the Policy Statement. The use of independent models allows for consistent treatment across firms. Losses and revenues under stress are estimated using the same modeling assumptions for all covered companies, enabling comparisons across supervisory stress test results. Differences in covered companies' results reflect differences in firm-specific risks and input data instead of differences in modeling assumptions. The use of independent models also ensures that stress test results are produced by stress-focused models, designed to project the performance of covered companies in adverse economic conditions.
(d) In instances in which it is not possible or appropriate to create a supervisory model for use in the stress test, including when supervisory data are insufficient to support a modeled estimate of losses or revenues, the Federal Reserve may use firm-provided estimates or third-party models or data. For example, in order to project trading and counterparty losses, sensitivities to risk factors and other information generated by covered companies' internal models are used. In the cases where firm-provided or third-party model estimates are used, the Federal Reserve monitors the quality and performance of the estimates through targeted examination, additional data collection, or benchmarking. The Board releases a list of the providers of third-party models or data used in the stress test exercise in the annual disclosure of quantitative results.
1.2. Forward-Looking
(a) The Federal Reserve has designed the supervisory stress test to be forward-looking. Supervisory models are tools for producing projections of potential losses and revenue effects based on each covered company's portfolio and circumstances.
(b) While supervisory models are specified using historical data, they should generally avoid relying solely on extrapolation of past trends in order to make projections, and instead should be able to incorporate events or outcomes that have not occurred. As described in Section 2.4 , the Federal Reserve implements several supervisory modeling policies to limit reliance on past outcomes in its projections of losses and revenues. The incorporation of the macroeconomic scenario and global market shock component also introduces elements outside of the realm of historical experience into the supervisory stress test.
1.3. Consistency and Comparability
The Federal Reserve uses the same set of models and assumptions to produce loss projections for all covered companies participating in the supervisory stress test. A standard set of scenarios, assumptions, and models promotes equitable treatment of firms participating in the supervisory stress test and comparability of results, supporting cross-firm analysis and providing valuable information to supervisors and to the public. Adhering to a consistent modeling approach across covered companies means that differences in projected results are due to differences in input data, such as instrument type or portfolio risk characteristics, rather than differences in firm-specific assumptions made by the Federal Reserve.
1.4. Simplicity
The Federal Reserve uses simple approaches in supervisory modeling, where possible. Given a range of modeling approaches that are equally conceptually sound, the Federal Reserve will select the least complex modeling approach. In assessing simplicity, the Federal Reserve favors those modeling approaches that allow for a more straightforward interpretation of the drivers of model results and that minimize operational challenges for model implementation.
1.5. Robustness and Stability
The Federal Reserve maintains supervisory models that aim to be robust and stable, such that changes in model projections over time reflect underlying risk factors, scenarios, and model enhancements, rather than transitory factors. The estimates of post-stress capital produced by the supervisory stress test provide information regarding a covered company's capital adequacy to market participants, covered companies, and the public. Adherence to this principle helps to ensure that changes in these model projections over time are not driven by temporary variations in model performance or inputs. Supervisory models are recalibrated with newly available input data each year. These data affect supervisory model projections, particularly in times of evolving risks. However, these changes generally should not be the principal driver of a change in results, year over year.
1.6. Conservatism
Given a reasonable set of assumptions or approaches, all else equal, the Federal Reserve will opt to use those that result in larger losses or lower revenue. For example, given a lack of information about the true risk of a portfolio, the Federal Reserve will compensate for the lack of data by using a high percentile loss rate.
1.7. Focus on the Ability To Evaluate the Impact of Severe Economic Stress
In evaluating whether supervisory models are appropriate for use in a stress testing exercise, the Federal Reserve places particular emphasis on supervisory models' abilities to project outcomes in stressed economic environments. In the supervisory stress test, the Federal Reserve also seeks to capture risks to capital that arise specifically in times of economic stress, and that would not be prevalent in more typical economic environments. For example, the Federal Reserve includes losses stemming from the default of a covered company's largest counterparty in its projections of post-stress capital for firms with substantial trading or processing and custodian operations. The default of a company's largest counterparty is more likely to occur in times of severe economic stress than in normal economic conditions.
2. Supervisory Stress Test Model Policies
To be consistent with the seven principles outlined in Section 1, the Federal Reserve has established policies and procedures to guide the development, implementation, and use of all models used in supervisory stress test projections, described in more detail below. Each policy facilitates adherence to at least one of the modeling principles that govern the supervisory stress test, and in most cases facilitates adherence to several modeling principles.
2.1. Soundness in Model Design
(a) During development, the Federal Reserve
(i) subjects supervisory models to extensive review of model theory and logic and general conceptual soundness;
(ii) examines and evaluates justifications for modeling assumptions; and
(iii) tests models to establish the accuracy and stability of the estimates and forecasts that they produce.
(b) After development, the Federal Reserve continues to subject supervisory models to scrutiny during implementation to ensure that the models remain appropriate for use in the stress test exercise. The Federal Reserve monitors changes in the economic environment, the structure of covered companies and their portfolios, and the structure of the stress testing exercise, if applicable, to verify that a model in use continues to serve the purposes for which it was designed. Generally, the same principles, rigor, and standards for evaluating the suitability of supervisory models that apply in model development and design will apply in ongoing monitoring of supervisory models.
2.2. Disclosure of Information Related to the Supervisory Stress Test
(a) In general, the Board does not disclose information related to the supervisory stress test or firm-specific results to covered companies if that information is not also publicly disclosed.
(b) The Board has increased the breadth of its public disclosure since the inception of the supervisory stress test to include more information about model changes and key risk drivers, in addition to more detail on different components of projected net revenues and losses. Increasing public disclosure can help the public understand and interpret the results of the supervisory stress test, particularly with respect to the condition and capital adequacy of participating firms. Providing additional information about the supervisory stress test allows the public to make an evaluation of the quality of the Board's assessment. This policy also promotes consistent and equitable treatment of covered companies by ensuring that institutions do not have access to information about the supervisory stress test that is not also accessible publicly, corresponding to the principle of consistency and comparability.
2.3. Phasing in of Highly Material Model Changes
(a) The Federal Reserve may revise its supervisory stress test models to include advances in modeling techniques, enhancements in response to model validation findings, incorporation of richer and more detailed data, public comment, and identification of models with improved performance, particularly under adverse economic conditions. Revisions to supervisory stress models may at times have material impact on modeled outcomes.
(b) In order to mitigate sudden and unexpected changes to the supervisory stress test results, the Federal Reserve follows a general policy of phasing highly material model changes into the supervisory stress test over two years. The Federal Reserve assesses whether a model change would have a highly significant impact on the projections of losses, components of revenue, or post-stress capital ratios for covered companies. In these instances, in the first year when the model change is first implemented, estimates produced by the enhanced model are averaged with estimates produced by the model used in the previous stress test exercise. In the second and subsequent years, the supervisory stress test exercise will reflect only estimates produced by the enhanced model. This policy contributes to the stability of the results of the supervisory stress test. By implementing highly material model changes over the course of two stress test cycles, the Federal Reserve seeks to ensure that changes in model projections primarily reflect changes in underlying risk factors and scenarios, year over year.
(c) In general, phase-in thresholds for highly material model changes apply only to conceptual changes to models. Model changes related to changes in accounting or regulatory capital rules and model parameter re-estimation based on newly available data are implemented with immediate effect.
(d) In assessing the materiality of a model change, the Federal Reserve calculates the impact of using an enhanced model on post-stress capital ratios using data and scenarios from prior years' supervisory stress test exercises. The use of an enhanced model is considered a highly material change if its use results in a change in the CET1 ratio of 50 basis points or more for one or more firms, relative to the model used in prior years' supervisory exercises.
2.4. Limiting Reliance on Past Outcomes
(a) Models should not place undue emphasis on historical outcomes in predicting future outcomes. The Federal Reserve aims to produce supervisory stress test results that reflect likely outcomes under the supervisory scenarios. The supervisory scenarios may potentially incorporate events that have not occurred historically. It is not necessarily consistent with the purpose of a stress testing exercise to assume that the future will be like the past.
(b) In order to model potential outcomes outside the realm of historical experience, the Federal Reserve generally does not include variables that would capture unobserved historical patterns in supervisory models. The use of industry-level models, restricted use of firm-specific fixed effects (described below), and minimized use of dummy variables indicating a loan vintage or a specific year, ensure that the outcomes of the supervisory models are forward-looking, consistent and comparable across firms, and robust and stable.
(c) Firm-specific fixed effects are variables that identify a specific firm and capture unobserved differences in the revenues, expenses or losses between firms. Firm-specific fixed effects are generally not incorporated in supervisory models in order to avoid the assumption that unobserved firm-specific historical patterns will continue in the future. Exceptions to this policy are made where appropriate. For example, if granular portfolio-level data on key drivers of a covered company's performance are limited or unavailable, and firm-specific fixed effects are more predictive of a covered company's future performance than are industry-level variables, then supervisory models may be specified with firm-specific fixed effects.
(d) Models used in the supervisory stress test are developed according to an industry-level approach, calibrated using data from many institutions. In adhering to an industry-level approach, the Federal Reserve models the response of specific portfolios and instruments to variations in macroeconomic and financial scenario variables. In this way, the Federal Reserve ensures that differences across firms are driven by differences in firm-specific input data, as opposed to differences in model parameters or specifications. The industry approach to modeling is also forward-looking, as the Federal Reserve does not assume that historical patterns will necessarily continue into the future for individual firms. By modeling a portfolio or instrument's response to changes in economic or financial conditions at the industry level, the Federal Reserve ensures that projected future losses are a function of that portfolio or instrument's own characteristics, rather than the historical experience of the covered company. This policy helps to ensure that two firms with the same portfolio receive the same results for that portfolio in the supervisory stress test.
(e) The Federal Reserve minimizes the use of vintage or year-specific fixed effects when estimating models and producing supervisory projections. In general, these types of variables are employed only when there are significant structural market shifts or other unusual factors for which supervisory models cannot otherwise account. Similar to the firm-specific fixed effects policy, and consistent with the forward-looking principle, this vintage indicator policy is in place so that projections of future performance under stress do not incorporate assumptions that patterns in unmeasured factors from brief historical time periods persist. For example, the loans originated in a particular year should not be assumed to continue to default at a higher rate in the future because they did so in the past.
2.5. Treatment of Global Market Shock and Counterparty Default Component
(a) Both the global market shock and counterparty default components are exogenous components of the supervisory stress scenarios that are independent of the macroeconomic and financial market environment specified in those scenarios, and do not affect projections of risk-weighted assets or balances. The global market shock, which specifies movements in numerous market factors, applies only to covered companies with significant trading exposure. The counterparty default scenario component applies only to covered companies with substantial trading or processing and custodian operations. Though these stress factors may not be directly correlated to macroeconomic or financial assumptions, they can materially affect covered companies' risks. Losses from both components are therefore considered in addition to the estimates of losses under the macroeconomic scenario.
(b) Counterparty credit risk on derivatives and repo-style activities is incorporated in supervisory modeling in part by assuming the default of the single counterparty to which the covered firm would be most exposed in the global market shock event. Requiring covered companies subject to the large counterparty default component to estimate and report the potential losses and effects on capital associated with such an instantaneous default is a simple method for capturing an important risk to capital for firms with large trading and custodian or processing activities. Engagement in substantial trading or custodial operations makes the covered companies subject to the counterparty default scenario component particularly vulnerable to the default of their major counterparty or their clients' counterparty, in transactions for which the covered companies act as agents. The large counterparty default component is consistent with the purpose of a stress testing exercise, as discussed in the principle about the focus on the ability to evaluate the impact of severe economic stress. The default of a covered company's largest counterparty is a salient risk in a macroeconomic and financial crisis, and generally less likely to occur in times of economic stability. This approach seeks to ensure that covered companies can absorb losses associated with the default of any counterparty, in addition to losses associated with adverse economic conditions, in an environment of economic uncertainty.
(c) The full effect of the global market shock and counterparty default components is realized in net income in the first quarter of the projection horizon in the supervisory stress test. The Board expects covered companies with material trading and counterparty exposures to be sufficiently capitalized to absorb losses stemming from these exposures that could occur during times of general macroeconomic stress.
2.6. [Reserved]
2.7. Credit Supply Maintenance
(a) The supervisory stress test incorporates the assumption that aggregate credit supply does not contract during the stress period. The aim of supervisory stress testing is to assess whether firms are sufficiently capitalized to absorb losses during times of economic stress, while also meeting obligations and continuing to lend to households and businesses. The assumption that a balance sheet of consistent magnitude is maintained allows supervisors to evaluate the health of the banking sector assuming firms continue to lend during times of stress.
(b) In order to implement this policy, the Federal Reserve must make assumptions about new loan balances. To predict losses on new originations over the planning horizon, newly originated loans are assumed to have the same risk characteristics as the existing portfolio, where applicable, with the exception of loan age and delinquency status. These newly originated loans would be part of a covered company's normal business, even in a stressed economic environment. While an individual firm may assume that it reacts to rising losses by sharply restricting its lending (e.g., by exiting a particular business line), the banking industry as a whole cannot do so without creating a "credit crunch" and substantially increasing the severity and duration of an economic downturn. The assumption that the magnitude of firm balance sheets will be fixed in the supervisory stress test ensures that covered companies cannot assume they will "shrink to health," and serves the Federal Reserve's goal of helping to ensure that major financial firms remain sufficiently capitalized to accommodate credit demand in a severe downturn. In addition, by precluding the need to make assumptions about how underwriting standards might tighten or loosen during times of economic stress, the Federal Reserve follows the principle of consistency and comparability and promotes consistency across covered companies.
(c) In projecting the denominator for the calculation of the leverage ratio, the Federal Reserve will account for the effect of changes associated with the calculation of regulatory capital or changes to the Board's regulations.
2.8. Firm-Specific Overlays and Additional Firm-Provided Data
(a) The Federal Reserve does not make firm-specific overlays to model results used in the supervisory stress test. This policy ensures that the supervisory stress test results are determined solely by the industry-level supervisory models and by firm-specific input data. The Federal Reserve has instituted a policy of not using additional input data submitted by one or some of the covered companies unless comparable data can be collected from all the firms that have material exposure in a given area. Input data necessary to produce supervisory stress test estimates is collected via the FR Y-14 information collection. The Federal Reserve may request additional information from covered companies, but otherwise will not incorporate additional information provided as part of a firm's CCAR submission or obtained through other channels into stress test projections.
(b) This policy curbs the use of data only from firms that have incentives to provide it, as in cases in which additional data would support the estimation of a lower loss rate or a higher revenue rate, and promotes consistency across the stress test results of covered companies.
2.9. Treatment of Missing or Erroneous Data
(a) Missing data, or data with deficiencies significant enough to preclude the use of supervisory models, create uncertainty around estimates of losses or components of revenue. If data that are direct inputs to supervisory models are not provided as required by the FR Y-14 information collection or are reported erroneously, then a conservative value will be assigned to the specific data based on all available data reported by covered companies, depending on the extent of data deficiency. If the data deficiency is severe enough that a modeled estimate cannot be produced for a portfolio segment or portfolio, then the Federal Reserve may assign a conservative rate (e.g., 10th or 90th percentile PPNR or loss rate, respectively) to that segment or portfolio.
(b) This policy promotes the principle of conservatism, given a lack of information sufficient to produce a risk-sensitive estimate of losses or revenue components using information on the true characteristics of certain positions. This policy ensures consistent treatment for all covered companies that report data deemed insufficient to produce a modeled estimate. Finally, this policy is simple and transparent.
2.10. Treatment of Immaterial Portfolio Data
(a) The Federal Reserve makes a distinction between insufficient data reported by covered companies for material portfolios and immaterial portfolios. To limit regulatory burden, the Federal Reserve allows covered companies not to report detailed loan-level or portfolio-level data for loan types that are not material as defined in the FR Y-14 reporting instructions. In these cases, a loss rate representing the median rates among covered companies for whom the rate is calculated will be applied to the immaterial portfolio. This approach is consistent across covered companies, simple, and transparent, and promotes the principles of consistency and comparability and simplicity.
3. Principles and Policies of Supervisory Model Validation
(a) Independent and comprehensive model validation is key to the credibility of supervisory stress tests. An independent unit of validation staff within the Federal Reserve, with input from an advisory council of academic experts not affiliated with the Federal Reserve, ensures that stress test models are subject to effective challenge, defined as critical analysis by objective, informed parties that can identify model limitations and recommend appropriate changes.
(b) The Federal Reserve's supervisory model validation program, built upon the principles of independence, technical competence, and stature, is able to subject models to effective challenge, expanding upon efforts made by supervisory modeling teams to manage model risk and confirming that supervisory models are appropriate for their intended uses. The supervisory model validation program produces reviews that are consistent, thorough, and comprehensive. Its structure ensures independence from the Federal Reserve's model development function, and its prominent role in communicating the state of model risk to the Board of Governors assures its stature within the Federal Reserve.
3.1. Structural Independence
(a) The management and staff of the internal model validation program are structurally independent from the model development teams. Validators do not report to model developers, and vice versa. This ensures that model validation is conducted and overseen by objective parties. Validation staff's performance criteria include an ability to review all aspects of the models rigorously, thoroughly, and objectively, and to provide meaningful and clear feedback to model developers and users.
(b) In addition, the Model Validation Council, a council of external academic experts, provides independent advice on the Federal Reserve's process to assess models used in the supervisory stress test. In biannual meetings with Federal Reserve officials, members of the council discuss selective supervisory models, after being provided with detailed model documentation for and non-public information about those models. The documentation and discussions enable the council to assess the effectiveness of the models used in the supervisory stress tests and of the overarching model validation program.
3.2. Technical Competence of Validation Staff
(a) The model validation program is designed to provide thorough, high-quality reviews that are consistent across supervisory models.
(b) First, the model validation program employs technically expert staff with knowledge across model types. Second, reviews for every supervisory model follow the same set of review guidelines, and take place on an ongoing basis. The model validation program is comprehensive, in the sense that validators assess all models currently in use, expand the scope of validation beyond basic model use, and cover both model soundness and performance.
(c) The model validation program covers three main areas of validation:
(1) Conceptual soundness;
(2) ongoing monitoring; and
(3) outcomes analysis. Validation staff evaluates all aspects of model development, implementation, and use, including but not limited to theory, design, methodology, input data, testing, performance, documentation standards, implementation controls (including access and change controls), and code verification.
3.3. Stature of Validation Function
(a) The validation program informs the Board of Governors about the state of model risk in the overall stress testing program, along with ongoing practices to control and mitigate model risk.
(b) The model validation program communicates its findings and recommendations regarding model risk to relevant parties within the Federal Reserve System. Validators provide detailed feedback to model developers and provide thematic feedback or observations on the overall system of models to the management of the modeling teams. Model validation feedback is also communicated to the users of supervisory model output for use in their deliberations and decisions about supervisory stress testing. In addition, the Director of the Division of Supervision and Regulation approves all models used in the supervisory stress test in advance of each exercise, based on validators' recommendations, development responses, and suggestions for risk mitigants. In several cases, models have been modified or implemented differently based on validators' feedback. The Model Validation Council also contributes to the stature of the Federal Reserve's validation program, by providing an external point of view on modifications to supervisory models and on validation program governance.
3.4. Simple approach for projecting risk-weighted assets
(a) In projecting risk-weighted assets, the Federal Reserve will generally assume that a covered company's risk-weighted assets remain unchanged over the planning horizon. This assumption allows the Federal Reserve to independently project the risk-weighted assets of covered companies in line with the goal of simplicity (Principle 1.4). In addition, this approach is forward-looking (Principle 1.2), as this assumption removes reliance on historical data and past outcomes from the projection of risk-weighted assets.
(b) In projecting a firm's risk-weighted assets, the Federal Reserve will account for the effect of changes associated with the calculation of regulatory capital or changes to the Board's regulations in the calculation of risk-weighted assets.
12 C.F.R. 252 app B to Part 252