In the previous articles we learned a lot about how VaR is calculated using various methodologies. We also learned about stress testing our portfolios. But can we really rely on these VaR methods and accept the results they throw at us? In other words, how accurate are these models in doing their job, and estimating the risk accurately.

This question can be answered by back testing a VaR model.  Back testing helps us test the accuracy of a VaR model. Under this technique, the losses forecasted using VaR are compared with the actual losses at the end of the time horizon. The time horizon could be anything such a 1-day, 10-days, 1-month or more.

VaR calculates losses with a certain confidence interval. Suppose we have a one-month VaR of $1million at a confidence interval of 99%. This means that there is a 1% chance of having losses that exceed $1million by the end of the month.

If at the end of the month, the actual losses have not exceeded the VaR calculated losses, then we can say that the VaR model is appropriate. However, if the actual losses exceed the VaR calculated losses, in this case above $1 million, then we can say that the VaR model may not be accurate. This is called reconciling ex ante VaR estimates with ex post P&L values.

Every time the actual loss is above the VaR estimated loss, it’s called a breach or violation of VaR. However, just because the actual loss is above the estimated loss one time, we cannot say that the model has failed. We need to look at the frequency of such breaches.

Let’s say that we have the daily VaR figures at 99% confidence for 100 days. At 99% confidence, we can expect the losses to breach the VaR on one out of 100 days. The model can be said to have a problem only if the number of breaches exceeds the desired number based on confidence level.

If we reconcile the daily VaR values calculated at 99% confidence level over a period of one year, then 2.5 violations (5% of 250 trading days) are normal. If the number of breaches exceed 2.5, then we need to reevaluate our VaR model.

Under the Basel II Market risk Amendment, banks are required to perform back tests by comparing the ex-ante VaR estimates with the actual P/L. This is called dirty back testing is is modt commonly used.

Another method is to compare VaR estimates with the hypothetical P/L values of the portfolio. This is called clean back testing.

However, back testing results can have issues because the actual losses can exceed the ex-ante VaR estimates for many reasons beyond the applicability of the model. For example, poor historical data, inaccurate mappings, incorrect volatility estimates, among others.

So, when back testing tells us about the problem with VaR model, it is important that we identify the real issues, before taking any action.