FRTB Model Validation Framework
One of the big areas of change in the FRTB regulations is that the regulators will be far more prescriptive about model validation. What this means is that banks will be required to carry out quite stringent backtesting of the their models as well as be accountable for the assumptions made in the modelling.
- It will have to be shown that models do not underestimate risk owing to their assumptions e.g normal distribution.
- Hypothetical PnL – this is an important piece as it involves calculating whats also sometimes known as the Clean PnL – that is to say the P&L arising from holding the positions constant and applying the market moves to them. There will be no other elements such as fees and commissions.
- Full distribution backtesting – this involves calculating the p-value which is defined as the probability of observing a profit that is less than, or a loss that is greater than the amount reported according to the model used to calculate ES.
- Backtesting against VaR at 99% and 97.5% and an ES at 97.5%
FRTB – Risk Theoretical P&L
Another crucial aspect is the requirement to calculate what is known as the “Risk Theoretical P&L”. This is the P&L that is calculated by the sensitivities / risk factors that the pricing models from the trading desk produce.
Once this Risk Theoretical P&L is calculated then it can be compared to the hypothetical P&L to serve as a benchmark of accuracy / tolerance. The aim is to ensure that all risk factors that are used in the risk management models capture the material drivers from the pricing models that are used to calculate the P&L. The equation below is used to calculate a measure of how the Risk P&L moves in relation to the Hypothetical P&L.
Backtesting is the other crucial part of this framework and as noted above backtesting will be carried out against 97.5% and 99% VaR on a 1 day horizon. The backtesting will be performed against the hypothetical or clean P&L as noted above, however the BIS also notes that it would be desirable to also perform the backtesting against the dirty (or actual) P&L. It is also important to note that the backtesting framework must be up and running when the internal models framework is up and running. From this date a years worth of data should be gathered before statistical inferences can be made and reported to the regulator.
An outlier is identified whenever the hypothetical P&L is greater than the VaR at 99% confidence interval. In a year with 260 trading days, at 99% C.I we would expect to have no more than 2.6 outliers (round up to 3). If there are more than this number of outliers then the model is not valid and there will be remediation required. The number of outliers can be broken up into zones that are green , amber and red – with green being within the expectation of a reasonable model and red being problematic. The number of outliers and their zones is summarized below :
1 – 4 Outliers – Green Zone and the multiplier is 1.5
5 – 9 Outliers – Amber Zone and the multiplier ranges from 1.7 to 1.92
Over 10 – Red Zone and the multiplier is 2
It is important that the bank maintains the data from backtesting for regulatory oversight and that there is also commentary and explanation of when outliers occur. There are various reasons for why outliers happen from model not being resilient (modelling assumptions incorrect , data is incorrect etc) to model accuracy is not granular enough (need more buckets) to market moves markedly different to model assumptions (extreme events) to finally intraday trading skewing the P&L.
A crucial point to consider is that the testing will be done at desk level and also enforced at desk level. This testing will have to be done on a regular basis and reported to the regulator. The results of the test will also govern is the desk can use the Internal model or if it will have to move off to use the standardised model.