After the Crash, Risk Management Planning
- 10 Comment
The downward slide of financial markets and the cratering of the financial services companies involved has been greeted with regulatory imperatives ranging from calls for a college of supervisors and systemic risk regulators to demands for a new financial architecture. The terms and arguments will vary, but it is clear that change is in the air. Following are some initial thoughts that may assist in planning for next year’s risk management efforts, please share your additional thoughts.
1. Risk Management is not only about Modeling
One of the major events in the recent news was that models can, and often do break down. For those who have lived through prior market cycles, we have seen that it is in fact a common occurrence. Yes, models break, or they fail to capture the risk; so what? Do we stop modeling? Of course we do not stop modeling. Rather than asking for fewer models, we will be asked for more and more models in the future! With that trend in mind, this is a good time to reinforce in the risk management policies and guidance that risk management encompasses a full life-cycle of understanding, modeling, forecasting, and refinement.
Modeling is one element of the risk management process and in recent years the models took on an aura of invincibility, examples include black-box quant trading and CDO capital structuring. Lesson learned - we will have to model even more, but the models will still break. In times of market stress, it makes more sense to align resources to understanding and challenging model assumptions so that the R&D docket can be finalized for tomorrow’s new model challenges.
As a planning item, make sure your inventory of models, model assumptions, model weaknesses, etc. are all in order. Auditors and management will want to know about it sooner or later.
2. Data Data Data
Garbage in – Garbage out. I have seen dozens of presentations over the years where a person casually explains that the stock market crash of 1987 was removed from the data set, or some other simplifying assumptions about market movements.
Recently, it was accepted wisdom that static correlations in a Gaussian Copula approach could capture CDO credit correlations (oops). Risk managers need to focus on the ugly data and ensure we utilize proper methods for collecting and employing data. Simplifying assumptions about distributions should be challenged (e.g., market movements may not be random normal). We should also look at the outliers, acknowledge that correlations are not static, put thought into forecasting based on more than just past volatility and apply the same discipline to collection of data that we apply for modeling markets.
We need to invest resources in obtaining and analyzing time series data, reference data, econometric forecasts and model input data. All this should be in a professionally managed database – hint – these activities can be outsourced to specialty firms so your analysts can focus on the context, not the content.
Also, remember that to truly model the Mortgage Securities and CDOs, you need to have accurate data about the underlying loans and credits (these drive the cash flows). It seems like a simple statement, but I have seen many cases where firms are not investing in getting into loan-level analysis to model mortgage backed risks. In my humble opinion, that is a huge gap that needs to be closed and an area where front and middle offices should be spending a lot of resources. Most executives have no clue how much this can cost, and now is a good time to start laying the foundation for the required investments – maybe with the PPIPs, there will be new front office support for data collection and analysis budgets.
3. Reporting Investments
Demands for reporting will continue to grow. With the rise in “stress tests” by the regulators and increased national ownership of banks, the level of reporting will only continue to increase. We will be asked to provide more dashboards, more drill-down, more scenarios, slicing and dicing, etc. Now is a good time to ensure your reporting data models and BI tools are set up because there is no doubt that the needs are only going to increase this year and next. Over the past few years, there have been some good advances in domain modeling and BI planning which you can use to ensure that the data models are able to change to meet your business needs. Typically these types of engagements are done by consulting firms because the internal IT staff are a precious resource.
ONE LAST NOTE ON STRESS TESTS
Since everyone will be asking to submit portfolios to huge gains and losses, remember that a stress test should also include the test where nothing moves. The “dull” test will capture the losses from optionality which may have been taken into positions to show lower risk in extreme market moves. Remember the payoff profiles as illustrated below:
Just some helpful hints! Please add your other planning items for the benefit of readers.
More model control than modeling itself
There is an increasing focus in imposing controls to model performance and ensure a proper governance of models.
Modelers are under a huge pressure by supervisors, auditors and internal validators. The most natural hypothesis is questioned and documentation is required more than ever before. This pressure prevent modelers to improve current models, moving the focus to explain why current models are as the are and adding new evidence to prove that the commonly accepted hypothesis of the past still apply or do not. Banks need help here.
Also new ideas are treated with enormous caution, again preventing banks from a sane and natural evolution in the modeling techniques.
On the governance side Senior Management will never have a proper understanding of models neither it is their function. However, there is an increasing pressure to get models approved by the SM. A more realistic approval process is still to be established that gives confidence to the SM that models are robust and are used accordingly to its aim. Final approval needs to be delegated to forums with a more specific expertise.
The FSA in the report “The Turner Review. A regulatory response to the global banking crisis” of March 2009 states that “the FSA will therefore in future play a more active role in assessing the technical competence of senior risk managers.” See to believe, but we will hear of training programs and strategic management consulting for a new more technically-focused governance. Appart from this it is an excellent report!!!
Models are not necessarily black boxes because the math is hard, rather, practitioners tend to become complacent in monitoring them. Models are usually built for the current environment or ‘regime’. When the environment changes, e.g., as it did starting last Aug ‘07, models need to be recalibrated at best or completely respecified at worst. Users need to fully understand the limitations (weak spots) of these models, manage the risk more conservatively when stretching their capabilities, and adjust the parameters regularly (see “The Financial Modelers’ Manifesto”).
CDO models are a prime example. The traditional ‘Normal Copula’ model (the Black Scholes of the CDO world) and other securitization models incorporate assumptions about correlation. Leverage is derived from the level of correlation or diversification (think beta). Since we live in a fat-tailed world, correlation goes to 1 very quickly in a downturn, which as we saw, causes losses to plow right through the mezzanine and even senior tranches of these securities. Certainly, a stress test with correlation = 1 would keep this risk on the radar screen.
For the most part, traders and firms are able to effectively monitor their day-to-day market risk. VaR is also useful but not sufficient. Stress testing should also be part of the risk management process. Counterparty credit risk is another matter. I highly doubt anyone thought banks/prime brokers would become more credit risky than the hedge funds they service. Hopefully the new regulatory architecture will address too-big-to-fail and systemic risk to help manage the complexity of this problem. Although the ‘reserve’ model still dominates, I would like to see counterparty credit risk move toward a ‘market’ model, where counterparty credit risk is more actively managed/hedged.
I think Paco’s comments about the need to improve understanding models and Dave’s observations about moving towards more sophisticated models for counterparty risk measurement are indicators of upcoming priorities:
- Increased Model Sophistication;
- Increased Emphasis on Understanding/Documenting Models; and (in my opinion)
- Increased sophistication for gathering and analyzing model data.
I also believe that there might be an opportunity to increase the emphasis on “nonstandard” approaches for forecasting market variables as part of risk measurement. Areas for research include bringing concepts of econometrics and subjective inputs (e.g., Dave Kelly’s suggestion to move correlations to 1) as complements to the current approach of relatively simple statistics (historical moving averages, limited time-span historical simulation - even the GARCH methods are apparently inadequate compared to what we have seen in the past 12 months). Another subjective adjustment that can be implemented is variable liquidity assumptions. I remember that when we first started modeling emerging market risks we shifted the liquidity periods (ie., time to unwind a position) out to as long as 68 days, maybe even more. However, we did not take into account the prospective market variability when liquidity dried up as the effort was perceived to be too esoteric.
Paco, I recall we had a PhD do some work on Jump-Diffusion approaches for forecasting market variables - not sure if those documents ended up getting published, but those kind of models might come back into vogue.
Anyone have an opinion about whether the current banking environment will lead to acceptance of modeling advances or whether executive management caution will stall innovation?
The current financial crisis has wreaked havoc on the markets worldwide and the entire economy. Will the TARP funds, which have been pumped into the banks, be enough to rescue this situation? Not really.
One of the solutions would be to go back to the basics and follow the life cycle of risk management. Market Risk, Credit Risk, Operational Risk (as specified by Basel II) and Liquidity Risk should be the major key components measured and has to be followed rigorously. Most of the banks had implemented Basel II but who were looking at the numbers coming out from these implementations? Market was on a roll, everyone in the street was talking about CDO, CDO2 even without understanding the risks involved in these products where the underlings’ are highly correlated which can wipe out the portfolio in a hurry. So it follows that the financial institutions should understand what the models are capable of. Models nowadays have become quite complex and rightfully so understanding the nature of the products and have been implemented quite easily with the advances in technology. But at the same time the drawbacks of these models should not be underestimated as they are based on assumptions and this is very important.
It won’t be surprising to see experienced independent (third party) risk management boutiques coming up in the near future to help the financial institutions around the globe to implement and ensure that at least some of the risk management responsibilities like Stress Testing, Back Testing VaR, Risk Limits etc.are carried out.
Is Risk Management the only solution
a) to come out of this current situation we are in today or
b) can it prevent such a situation in future
Absolutely not, but it is a well-proven process/tool, which works very well as long as we understand it’s limitations and keep on enhancing the process.
Pretty much everything that I have read omits the one key thing that was proven in 2007-2008 - which is that the markets need a new governance scheme for assigning credit ratings. Once governance has been repaired, it will probably then make sense to consider some methodological changes.
Rating agencies based in the U.S. assigned ratings to several financial companies that were much too high. Similarly, the ratings assigned to hundreds (thousands?) of tranches of CDOs were too high. One might argue that these errors call for a methodology change. However, I think that what is necessary is a new regime for assigning credit ratings that is separate from profit-seeking entities. There have been several calls to create a consortium on the buy side that would fund a service bureau that would assign ratings. I think something along these lines would be the best approach.
The goal has to be to get the credit rating more accurate. If we reach this goal, then I think we will likely conclude that the current models for obligor and counterparty risk are actually pretty good. We should also remember that given the structures of the prime brokers, it was always very unlikely that hedge funds would be the locus of catastrophic disruptions in the financial markets - especially after the scare in 1998. Hedge funds are very deliberate and very conservative in structuring their balance sheets and managing their liquidity risk - if any begins to stray, its prime broker will quickly intervene to maintain the hedge fund’s balance sheet within prudent bounds. The key difference between hedge funds and financial institutions is that managers of hedge funds (almost) always have a complete understanding of the dynamics of their balance sheets, while the managers of many financial institutions (especially large ones) do not.
Roy (nice to meet you again!!) pointed out that we can expect “to see experienced independent (third party) risk management boutiques”. Indeed. Large banks shed more that 125,000 jobs. Many of those who were invited to abandon they jobs will try to continue to do what they used to do. Roy’s vision could be a feasible, logical solution for many of them. The question is to identify what you can do better that the best ones among those 125,000.
David comments that he “…would like to see counterparty credit risk move toward a ‘market model’”, and Eric criticizes credit rating and claims that “The goal has to be to get the credit rating more accurate.” Consortiums are already in place aimed to share PD data, which is a more objective measure that rating, then PDs will be mapped into ratings. Anyhow rating models need to be worked out hard and eventually migrate to a more direct rating/PD relationship.
Ed recalled some work we did in the past. We hired one of my colleagues from the University to design a stochastic diffusion model with jumps. That guy eventually became famous in the academy because the discovered a statistical paradox that was named after him, the Parrondo’s Paradox.
These previous comments have a common point, which happen to be another model, the Incremental Risk Charge (IRC). IRC is aimed to cover (a) default risk and (b) migration risk. Additionally there is some concern about (c) spread risk (jumps in spread with no changes in rating) and (d) event risk for equity position (more jumps).
Credit data consortiums are prerequisites for (a) and (b) but the solution is David’s market-like model. The answer for (c) and (d) above requires adding jump diffusion to such a model to account for credit risk (not only for trading portfolios). Can any of us do this better than any one of those 125,000? Maybe not better but, for sure, not worse.
First of all, congratulations to Ed for raising an excellent issue and creating a discussion, it is a pleasure to find old and qualified friends on the web!
My last years moved me from a more modeling approach to senior management implications, experiencing that Roy’s and Paco’s comments on the need for new models and new Risk Management tools are quite true. The problem is to translate a EVT or new VaR into a “reasonable” measure that is comparable with the existing bank capital (quite scarce these days) otherwise will be considered from Senior Management as another quant model that is going nowhere, and this is the challenge – make it understand as to a 5yr kid.
Super-regulation is something that seems to be moving away, but there is a clear need for Basle III, you also need to consider that mid and smaller financial institutions do not have the “muscle” that a large bank does and even savings bank associations have not succeeded to implement jointly models.
Eric’s idea on joint approaches is excellent, but in Europe firms are more reluctant to share intellectual capital, and this leads me to support the idea to create risk management boutiques that will assist these mid-sized banks and funds in
- Developing risk measurement models
- Assess on investments and asset allocation
- Assisting on risk reporting, limit setting, exception analysis
- …
Moreover, I have witnessed FI that still hold Lehman structured products booked at par… there are still plenty of skeletons in the closet
First of all, this group of writers has really contributed to the discussion and I am grateful for your insight, points of view, and the challenges you have set out. Two areas where I think we can delve into discussion are:
1. Stupid Simple Measures
2. Recommended Next Steps
1. Stupid Simple Measures
The 2 simple measures that led to chaos, in my opinion, are the VaR and the Credit Rating. I think that each of these is really aiming to provide relative risk measures rather than absolute risk, but Executive Management and Regulators took it too far.
Anyone who has had the honor of working with Eric Gronningsater knows that he was an innovator in field of market risk measurement, and that he did it kicking and screaming. We were always challenged not to put much faith in a single VaR measure and not to rely on the limited information that it provided. Every portfolio was subjected not only to a VaR calculator, but a series of risk measures that aimed to capture the overall risk profile of the book. Of course, this would create consternation because management wanted simplistic, consistent measures; but for those who really wanted a full understanding of the market risk profile, it was required to actually think about multi-dimensional aspects of market risk. Lets face it, banks were forced to have a one-page VaR summary of risk because JP Morgan did it, we had to “keep up”. I thought that most risk managers understood that VaR was not the full risk measure, but the link between VaR and Capital modeling soon changed all that.
Similarly, credit ratings have a usefulness, but not in an absolute sense. For example, a bond with a bad credit rating has a higher probability of default than a bond with a good rating. I believe that relative risk is the most one can glean from the credit rating. I do not believe that a AAA really means that probability of default is .002% (or whatever the agencies claim it to be these days). Anyone who really believed that the rating could predict the probability of default amount had been seduced by a desire for short-cut thinking. But again, the capital modeling regime used credit ratings as a foundation item.
So if simple measures are really only good for evaluating relative risk size, then why did we let ourselves create capital models based on these concepts? Basing capital adequacy on simple measures implies that these measures are actually good at predicting economic losses. I think that the transition from using simple risk measures for relative risk measurement to using them for capital adequacy was a leap of faith (out of the pan, into the fire).
Also, the emphasis that was placed on backtesting was really misguided because it implied that we actually believed that the models should have functioned. In reality, the statistical methods were flawed and we could have devoted time to identifying where the models break by using a more multi-disciplinary approach than simple backtesting.
2. Recommended Next Steps
Some of you have suggested that we need some kind of consortium to control the rating agencies, but I am not sure what that really means and your comments/clarifications would be appreciated.
I believe that the ratings agency concept can be disbanded and that the comfortable oligopoly can be disbanded. Investors who want credit advice should either buy it or do it themselves. Since the whole rating agency concept arose from the Depression, it is clear that investors required some form of protection, so I am not claiming that credit analysis should be disbanded, but rather we should have healthy competition in the space. Imagine what may have changed if the ratings agencies were liable in the event that actual credit performance varied from their predictions - maybe then you would have seen more carefully constructed ratings models. I do not believe that the ratings agencies could be adequately capitalized to perform this function, so that idea seems like a dead-end.
On the topic of capital adequacy, I think that the approach to improve risk measurement involves a movement away from singular, simple measures towards a more robust estimate of capital employed to cover risk. Perhaps we can define an approach where multiple metrics of expected and unexpected loss are analyzed and a robust mechanism for aggregating these risks into capital measures is deployed. We can then subject these measures to systemic scenarios (i.e., subjective shocks) to identify where the models break down, and then reinforce the capital reserves based on model weaknesses. Of course, this approach will require tons of computer power, good data and mathematicians/statisticians.
Yes, risk management is dead, long live risk management!
It’s been a long time since I had such a interesting exchange of ideas. Ok, let’s continue. Since I am a forward-looking guy I’ll focus in Ed’s next steps
Data Consortium
Several initiatives are in place where banks share their experience of defaults and recovery by populating a common database. No client information is disclosed but just what it is necessary to calibrate your models and benefit from bigger populations (for example http://www.pecdc.org. There is also an American version). However, I guess Eric was not exactly speaking of this.
New measurements
Imagine a multiple metrics-multiple risks model. It is built in separated modules so that market risk guys execute their VaR and stress testing; Credit risk guys run their control processes and compute credit risk capital as well as credit stress test, and so on with all the risks (operational, business, reputational…). Finally, you have an engine that knows how to aggregate consistently all these specific purpose measures into a common framework. Additionally you have a set of controls that help you to compute add-ons for all model weaknesses and, in the case of market risk, estimate Fair Value Adjustments (FVA) for all mtm instruments.
Believe me that you do not need a lot of computing power to get that system, neither a lot of money. Yes, you need data and good databases but key, the most important thing, is a sound policy and a robust governance of the model so that all parties do that they have to do when they have to do it. By the way, this is the kind of governance that the regulators are so keen to see.
Today I had a Big SCAP Attack. I just had to dig into the Supervisory Capital Assessment Program - SCAP - which was released in a whitepaper by the Fed this weekend. The document makes for good reading and reinforces my belief that the entire world runs on Excel, but that is a different blog entirely.
There were 2 aspects of the introduction to the SCAP which struck me.
First, there was a concise (albeit scary) definition of capital. “The traditional role of capital, especially common equity, is to absorb unexpected losses and thus to protect depositors and other creditors.” This definition is interesting to think about now that the US Government will own a substantial quantity of the Common Equity for certain banks. An unanswered but relevant follow-up question is “What happens if this is insufficient?” In a normal world, the financial entity goes bankrupt, but these are not normal banks anymore (they are too big to fail). And if they are too big to fail, why do they really need capital? Perhaps this stress test is the first step at coming to reality? In other words, this test can separate the good banks from the bad banks and we can begin to consider letting the bad banks fail.
Second, there was a clear explanation of the test. The basic logic is as follows:
1. Start with 2 economic scenarios (one is bad, other is worse)
2. Combine “projected losses, revenues, and changes in reserves …to evaluate the amount and quality of capital that each firm should have at the end of 2010.”
The 3 big drivers - house prices, GDP and Unemployment.
Sounds good, now we model all our assets and liabilities under adverse scenarios of these and get a capital estimate, nice!
Wait a minute…
Last I heard, banks made money of the Net Interest Income - what happens to NII under these scenarios? And what about changes in the overall level of interest rates? And the dollar? and option volatility? inflation? credit spreads? Yeah, what about all those nasties? Do we hold them constant?
Good News! When I read into the details, there was more linkage.
Revenue (PPNR) was defined as “net interest income plus non‐interest income minus non‐interest expense. It is therefore the income after non‐credit‐related expenses that would flow into the firms before they take provisions or other write‐downs or losses”…(Bank Holding Company) “BHCs were instructed to project the main components of PPNR under each of the macroeconomic scenarios.”
Also, for the trading books, (if you had something bigger than $100 billion), then an additional test:
“the firms conducted a stress test of their trading book positions and counterparty exposures as of market close on February 20, 2009, based on an instantaneous re‐pricing of trading positions equal to the changes in market pricing variables that occurred over the period of June 30, 2008 to December 31, 2008. Aside from the dollar loss estimates, BHCs were asked to disclose the positions that were included in this analysis as well as the risk factors that were stressed and the changes in variables employed (for example, changes in rates and spreads, and percentage changes in equities, foreign exchange, and commodities). Firms were also asked to provide the results of the stress tests conducted in the usual course of business from January 2009 or the most recent dates available.”
The Fed was pretty good at putting all the caveats, and even created a nice definition of the stress test
“a stress test provides a systematic, disciplined framework for gauging the magnitude of capital buffers that might be needed by different firms to absorb losses under plausible “what if” scenarios.”
After the definition of the requirements, there was an overview of the role of the regulators in reviewing, testing and benchmarking these estimates.
For example, “A key aspect of this analysis was to understand the particular parameters and assumptions employed and their consistency with the macroeconomic scenarios provided, as well as the models and methodologies used to generate the loss and resource estimates..”
The reviewers looked for internal consistency and also did their own estimating. “Aside from a direct review of the assumptions and models used in loss and resource projections submitted by the participating BHCs, the agencies developed independent benchmarks against which to evaluate the submissions.”
Also,the Fed “developed specific portfolio risk profiles in order to make cross‐firm comparisons to gauge the reasonableness of the loss estimates submitted by the firms”
In a nutshell, the document is good reading (B+), and provides a summary exposition on principles of stress testing and on bank capital analysis. It would have been an A paper if the juicy details about the linkage of Economic Scenarios to market factors were produced - and also if we got to see the analysis of variances between the Fed benchmarks and Bank models. I guess that will come in the sequel. The link for the whitepaper is below:
http://federalreserve.gov/newsevents/press/bcreg/bcreg20090424a1.pdf
Enjoy!
Ed