Review of Bank of Russia – NES Workshop ‘Identification and Measurement of Macroprudential Policies Effects’

In the first week of June 2021, the Bank of Russia and the New Economic School hosted a joint international online workshop titled ‘Identification and Measurement of Macroprudential Policies Effects’. Participants’ presentations suggest that macroprudential policy measures against high-risk lending produce their intended effects, but also, as a rule, bring about side effects. These effects may include a reduction in low-risk loan disbursements, if such measures are disincentivising in nature (as in Russia), or, vice versa, significant growth in the portfolio of low-risk loans, if the macroprudential tools are of a restrictive nature (as in Switzerland and Ireland).


Introduction
With macroprudential policy measures having in recent years become increasingly common central bank tools, focus is now turning to the goals which macroprudential regimes can achieve and the optimal method of conducting macroprudential policy. These issues were discussed at an international online workshop held jointly by the Bank of Russia and the New Economic School in early June 2021.
The subjects of workshop participants' presentations included analysis of the impact of macroprudential tools on lending in Russia (Penikas, 2021), Switzerland (Behncke, 2020), and Ireland (Acharya et al., 2020) and insight into the effect of macroprudential policy measures on industrial output in a large sample of countries, based on data from international organisations (Madeira, 2020). vol. 80 no. 3 This workshop summary is structured as follows. Section 2 compares the research objectives of the four key presentations. Section 3 describes the data that has been available to the researchers. Section 4 outlines the methods the authors used or could potentially use in their studies. Section 5 presents a comparison of the presenters' findings. Findings are finalised in Section 6.

Research objectives
For the same action of a central bank, at micro-and macro-level its consequences may differ and even be opposite. To cite one example, let us consider the capital adequacy ratio, that is, the ratio of capital 1 to the amount of riskweighted assets of a bank. In 1988, the Basel Committee on Banking Supervision established the use of this indicator in the first Basel Capital Accord (Basel I), while a number of advanced economies such as the US, Switzerland, and Japan (Goodhart, 2011) had previously applied the ratio.
From the perspective of microprudential regulation, the more capital a bank has, the more losses it can sustain at the expense of its own funds, rather than its depositors' funds. Sustaining such losses is certainly not without consequence. Assume that a bank has squandered all its accumulated profit: its shareholders will demand that the bank's management compensate for it. This will lead to the bank's attempt to accept excessive risk, raising loan rates, reducing deposit rates, and so on, and all of this time, it will have to remain compliant with the capital adequacy ratio. However, as Peter Cook, one of the most famous Basel Committee chairmen, has pointed out in discussions about the format of the capital adequacy ratio, the requirement that the minimal capital adequacy level always be maintained is absurd. In his opinion, this is similar to the introduction of a rule that there must always be at least one car at a taxi stand: imagine a passenger seeing the last taxi at the stand and its driver refusing to pick him up so as not to break the rule (Goodhart, 2011, p. 555). This leads to a gridlock situation. When there is a crises in an economy, and production chains need to be launched to overcome it, this is more quickly done with borrowed money and rather than by waiting for free balances to accumulate from profit. However, strict rules prevent banks from extending loans, since they are constrained by the capital adequacy ratio requirements. Ultimately, the no fast recovery from the crisis will occur, notwithstanding the banks will, microprudentially, demonstrate stability.
In this example, the economy may benefit at the macro level from a lower capital adequacy ratio. This is how the countercyclical capital buffer (CCyB), one of several macroprudential tools, operates. These tools came to be used more frequently after the 2007-2009 crisis, and studies of their impact, including in connection with microprudential policies, became a relevant topic in the following years. september 2021 In her opening remarks, Anna Obizhaeva (NES) outlined the fundamental research issue related to macroprudential policies: we do not know how they affect the economy.
It is important to note that this problem has several important aspects. First, research findings are often limited to the narrow segment of lending targeted by the macroprudential policy. Meanwhile, dynamics (e.g. decline) observed in the segment under the study may be offset by the dynamics (growth) of another segment that remained outside the scope of the study.
Second, impact is a complex category. Many papers, including one by the Bank for International Settlements (Bank for International Settlements, 2020; hereinafter, BIS, 2020), state that they study the effectiveness of the macroprudential policies, but are in fact evaluating the impact of the measures.
Third, specific papers on macroprudential policy tend to lose their relevance relatively quickly: in issues related to financial bubbles and economic overheating, the central banks cannot afford relying on research based on data that are five or even ten years old.
Fourth, when assessing the impact of macroprudential policies, researchers may see that the macroprudential actions taken change emerging trends so that it is almost impossible to see the effect of the policy measure itself. A series of random experiments could be of help here (Enikolopov, 2020), but they can only be carried out in application to the narrow segment of micro loans, not to the overall lending sector.
Elizaveta Danilova, Director of the Financial Stability Department of the Bank of Russia, noted in her opening remarks that the Bank of Russia has extensively used macroprudential instruments to limit the risks to financial stability and enhance financial resilience in the banking sector. Specifically, a range of measures has been implemented since 2013 to prevent bubbles in retail lending. Research into the effects of measures is essential for the timely adjustment of macroprudential policy. For example, the Bank of Russia's intention to introduce direct quantitative restrictions on lending is based on research findings that show the effects of previously introduced macroprudential policy measures to be time lagged and of small scale (see, in particular, Penikas, 2021). It is expected that the tool of direct quantitative restrictions will become available to the regulator from 1 January 2022 (RBC, 2021).
The studies discussed at the workshop are similar in that they all assess the effects of macroprudential policies. They differ in the levels at which the research is conducted (bank level or country level), the macroprudential toolkits under study and the outcome indicators reflecting the research subject (growth in lending, either in one segment or across all loan categories at the bank level in the sample, impact on output, which may vary subject to the availability of loans).
For example, the papers presented by Henry Penikas (Bank of Russia) and Carlos Madeira (Central Bank of Chile) outline the multiple policy measures that vol. 80 no. 3 were enacted during the periods under study. Penikas assesses the effect on bank lending in Russia (Penikas, 2021) and Madeira -the effect on industrial production in some countries (Madeira, 2020). Stefanie Behncke (Swiss National Bank) and Katharina Bergant (IMF) explore the impact of loan-to-value (LTV) restrictions in Switzerland (Behncke, 2020) and Ireland (Acharya et al., 2020) respectively. Behncke (2020) additionally takes into account the introduction of a CCyB in Switzerland in the same period. The situations being viewed retrospectively, all the authors are limited in their choice of policies to study. None of the researchers had the opportunity to pilot any particular measure, even through a survey, as is being undertaken, for example, by the Basel Committee through the Quantitative Impact Study (QIS).

Data
To study the effects of the measures chosen, the authors largely used microeconomic data at the bank level, except for Carlos Madeira, who used macroeconomic data at the country level.
When compared, the studies show the dilemma the authors seek to solve: the choice of shorter time periods with more homogeneous data or longer periods to obtain robust estimates of the parameters (the data may contain structural shifts in the second case). For example, Penikas (2021) and Acharya et al. (2020) analyse the most homogeneous data, for Russia from 2015 to 2019 and for Ireland from 2013 to 2016 respectively. The other two papers presented at the workshop use data which have greater variation in time, indicators, and countries, but which are potentially more heterogeneous. For example, Madeira (2020) uses data for 93 countries from 1990 to 2016 -during this period, two major global crises occurred in 1998 and 2007-2008, which gave rise to significant structural shifts. Behncke (2020) considers data from Switzerland for the period from 2011 to 2017. However, it was during this time that the Swiss National Bank abandoned its currency peg against the euro (2015), a decision that triggered an approximate 30% strengthening of the Swiss franc (from 1.20 francs to about one franc per euro).
We note the granularity of the data used in Behncke (2020) and Acharya et al. (2020). These data are granular to the level of specific retail loans and allow the authors to study the LTV effects for individual loans instead to being limited to averaged indicators. Highlighting the conditions in which the decision was made to introduce macroprudential tools in Ireland, Katharina Bergant, in her presentation of the study by Acharya et al. (2020), demonstrated the conditions under which the macroprudential measures had been introduced in Ireland, by citing the data indicating that the mortgage lending expansion in the country had clearly contributed to boosting of the housing prices. Figure 1 shows that real estate prices (solid green line) rise in parallel with rising mortgage lending (dotted blue line) and drop as mortgage lending drops, with approximately one-year lag. september 2021 Source: Acharya et al. (2020)

Methods
The work by Madeira (2020) is the only one of those discussed at the workshop in which the author constructs a panel regression. The paper is similar in its method to the BIS study (2020). In Madeira (2020), the dependent variable in the regression is the growth rate of industrial production, and the set of independent variables include macroprudential policy indicators. In addition, the author uses the External Financial Dependence Index (the Rajan-Zingales index) proposed in Rajan and Zingales (1998), which makes it possible to identify the greater magnitude for the macroprudential policies impact for industries with higher external finance dependence. At the same time, as discussant Mikhail Stolbov (MGIMO) pointed out, the Rajan-Zingales index has its limitations, and in the case of some countries, it might be advisable to use adjusted versions of the index, for example, those based on the data of the Bruegel-UniCredit project (for details, see Stolbov, 2021). Andrey Sinyakov (Bank of Russia) proposed that it might be advisable for the study to look into the cross-sectoral relations based on the interindustry balance data (input-output tables), considering that these relations may enhance or weaken macroprudential policy effects. Carlos Madeira explained in response that he had also considered the option of using such tables, available in databases at the University of Groningen. On the one hand, the use of such data could enrich the study, but on the other hand, it would have reduced the number of countries in the sample from 93 to 40.
The remaining three papers discussed (Penikas, 2021;Behncke, 2020;Acharya et al., 2020) use the difference-in-difference (DID) method to assess the effect of macroprudential policies. Importantly, Acharya et al. (2020) consider the introduction of LTV and LTI (loan-to-income) restrictions essentially as a oneoff event that took place in Ireland in 2014, whereas the other two papers consider several events. The research subject of Behncke (2020) is the introduction in 2012 of an LTV cap on mortgage loans and the subsequent introduction of a CCyB within the span of a year, while Penikas (2021) considers the successive introduction of a series of risk weight add-ons for unsecured consumer loans applied in the calculations of the statutory capital adequacy ratio in Russia.
The work of Behncke (2020) offers an exemplary approach to the assessment of policy measures, or the treatment effects. 2 Before applying the DID method, the author checks whether there is a general trend in the data and demonstrates the need to control for factors that may be affecting it. For example, as follows from Figure 2, data on mortgage lending growth rates in the pilot (solid line) and control (dotted line) groups indicate an overall trend only over a certain interval (until mid-2012). The pilot group is banks with low capital reserves (relative to the statutory capital adequacy ratio): low reserves leave these banks with few options to respond to new macroprudential policies. The control group is made up of banks with large reserves. Unlike the pilot group banks, they can carry on issuing high-risk loans while remaining compliant with statutory capital adequacy requirements by drawing on their accumulated reserves. It can be seen that the group of banks most exposed to macroprudential policy effects had scaled back lending even before the new measures were implemented in 2012. Therefore, the author emphasises, it is necessary to at least control for the overall trend for bank and time effects. Source: Behncke (2020) Discussant Carsten Sprenger (NES) pointed out that further value might be added to the paper by the use of the ratio of risk-weighted mortgage assets to all assets, along with data on capital ratio, and the comparison of the results obtained. He also suggested the possible parallel assessment of the impact of the two measures under study. However, Stefanie Behncke explained that only one bank in her sample came under this dual impact, and that any reasonable conclusions would necessarily involve more than one experimental observation.
Although DID is the most common method to assess the treatment effect, its use may be limited or produce distorted results. Penikas (2021) discusses this issue in application to macroprudential restrictions on consumer lending in Russia. The starting point for the study was the BIS approach (2020). However, it also has two major drawbacks. First, it involves the analysis of changes in a group of regulated banks without comparison of their indicators to those of banks less exposed to regulation. Therefore, there is no comparison between the pilot and control groups. Second, double, and triple interactions of the policy tools with bank performance indicators (size, liquidity, deposit base, and capital adequacy) proposed for inclusion in the model may distort the assessment of regulatory effects.
The assessment of a macroprudential policy effect is based on an econometric regression of the type: (1) where Y t is the dependent variable (the share of consumer loans in the assets of bank in time period t); M P t is indicating whether the macroprudential policy is in place at time t; X t is the control variable -a bank characteristic; ε t is the stochastic error term. The policy effect is the value of the coefficient before the product M P t × X t .
Let us consider the two approaches to assessing the policy effect, which are based on this regression specification: BIS (2020) and DID. The key difference between the two approaches is how we define the control variable and, consequently, the product variable M P t × X t . The BIS (2020) approach involves the use of the lags of the bank characteristics. In the DID approach, the bank characteristic is what distinguishes the control group from the pilot group. The DID approach is different in one critical way from BIS (2020): DID assumes that the characteristic is time independent. Penikas (2021) shows that it is possible to construct an example in which the use of the lag values of the control variable in the regression under the BIS method (2020) produces an incorrect estimation of the macroprudential policy effect on the dependent variable (the share of consumer loans in bank portfolios).
The application of the traditional difference-in-difference method also has its particularities. For example, Penikas (2021) points out that the period between the introduction of two policies is methodologically undetermined for application of this method. When several policies are successively introduced, banks may fail to respond within the time between the dates of introduction of the policies, which distorts the estimates obtained. Adequate estimation involves the creation of fullfledged pairs of pilot observations (before and after) for each policy introduced and a copy of observations corresponding to the intermediate period. The intermediate vol. 80 no. 3 period is considered the after period in the case of earlier measures and the before period in the case of later measures.
In determining the pilot and control groups, Penikas (2021) draws on the work of Behncke (2020), considering capital adequacy ratio as a possible indicator for a bank's exposure to macroprudential policy measures (a similar approach is used in Kozlovtseva et al., 2020); another such indicator is the share of consumer loans in bank assets. Stefanie Behncke, discussant for this paper, supported the evidence of the author's example indicating that use of the BIS (2020) approach in this case may fail to produce convincing results. Still, she proposed, it makes sense, rather than creating a repeat sample, to try and estimate the effects of various policies for each quarter in one model, in order to see the development of the situation over time, as she had done in the study she presented. This recognition of several policies in one model is also known as a spatial-difference-in-difference method (Horn et al., 2019). It was applied in the paper (Merika et al., 2020) to study the effects of the transition to internal models (Basel II internal ratings-based approach).

Findings
In all their works, the authors conclude that the macroprudential policies they considered produced the intended effect and no unforeseen effects (opposite to those intended) were observed. Penikas (2021) shows that after the announcement of macroprudential policies the banks with the largest portfolios of consumer loans relative to assets tend to reduce them earlier and to a greater extent. Madeira (2020) posits that the application of macroprudential policies reduces output, especially in countries with high debt burdens. This reduction can hardly be seen as favourable: rather, it is expected, according to the speaker. After all, if banks cannot benefit from high-risk loans, their disbursements drop, as do overall loan disbursements (notwithstanding the regulator's expectation that a reduction in high-risk loans will lead to growth in low-risk ones, which may ultimately result in an expansion of overall lending and possibly of output). As a consequence, output drops. Behncke (2020) points out that, following the introduction of the LTV cap in Switzerland, the share of corresponding loans started to shrink. In Ireland, LTV measures drastically reduced the disbursement of such loans (Acharya et al., 2020).
At the same time, the studies find that, beyond the intended impact, in all of the cases the macroprudential measures had certain side effects.
For example, according to Acharya et al. (2020), reduced disbursement of highrisk mortgage loans in Ireland came with a multiple increase in the issuance of lower-risk loans (those not subject to the new measures). As a result, the overall dynamics of mortgage lending were virtually unchanged (see Figure 3). In addition, the authors found that banks exposed to macroprudential policies increased lending to high-risk corporate borrowers. september 2021 Source: Acharya et al. (2020) According to discussant Konstantin Styrin (Bank of Russia, NES), some effects outlined in Acharya et al. (2020) seem unexpected and should be reverified or have their reasons explained. For example, it follows from the coefficients of the estimated regression that, after the introduction of the regulatory measures, banks with different business models (different risk appetites) started to offer loans on the same terms, with differences only in the loan amounts. According to Styrin, the study shows that the regulatory measures indeed produced the intended effect but does not answer the question of their implications for the overall financial stability.
In fact, as Katharina Bergant said during the discussion, the analysis of the probability of default at the level of individual loans showed that, following the introduction of the regulation, this indicator increased, and increased more strongly in the banks affected by the measures. Therefore, although the findings suggest a decrease in risks to the banking system from the housing market, the question of the 'net effect' on financial stability is impossible to answer. Behncke (2020) notes that, as LTV caps were introduced, the banks in the control group unaffected by the measure showed a rise in the issuance of loans with higher-risk LTI ratios, which remained unregulated. At the same time, as reported by Stefanie Behncke in the discussion, she failed to identify other effects of banks' adjustment to the restrictions, in particular, she did not find any significant changes in the geography of lending (the data do not enable a rural/urban breakdown of loans, though comparisons at the canton level are possible) or in the size of corporate borrowers. Penikas (2021) shows that a small (albeit statistically significant) reduction in the share of consumer loans in Russian banks' assets comes with a still greater reduction in general loans. That is, despite regulatory measures aimed at forcing banks to abandon high-risk loans, banks prefer to keep high-risk loans in their portfolios, at the cost of a much stronger reduction in low-risk loans.
Therefore, all studies find a sort of 'communicating vessels' effect. Macroprudential measures intended to reduce the issuance of high-risk loans either lead to a decrease in the amount of lower-risk loans under disincentivising regulation (as in Russia) or, vice versa, to an increase under restrictive regulation (as in Switzerland and Ireland). In both cases, banks' total exposure does not shrink, with changes seen only in the structures of their loan portfolios and lending policy priorities. The effects identified present real challenges for the regulatory practice and make policy makers look for new solutions. For example, Russia is considering the introduction, tentatively from 1 January 2022 (RBC, 2021), of quantitative restrictions which will force Russian banks to reduce the high-risk loans that banks currently try to retain in their portfolios above all.

Concluding remarks
Summing up the workshop, Alexander Morozov, Director of the Research and Forecasting Department, stressed that financial stability and macroprudential policy are priority issues in the Bank of Russia's research agenda. 3 He talked about the existence of redistributive effects from macroprudential policy measures, not only on the structure of lending, but also on the overall lending market, that is, redistribution takes place not only within bank portfolios, but also between banks. When market leaders reduce their portfolios of high-risk loans, competing banks seek to benefit from this and take their places in the ranking by size of portfolio of such loans. As a result, at the level of the entire banking system the risk may show no downward trend. The identification of such effects by the researchers should enable well-targeted regulatory influence on the banking system and help create the lending structure that best supports the achievement of the financial stability agenda.