RISK MODEL BACKTESTING

Abstract

Within this paper we shall research the validation methods of the risk model and we shall provide an overview of the existing literature which deals with validation and performance assessment of the VaR (Value at Risk) model. The importance of backtesting of the risk model stems from the fact that credit institutions have been allowed by regulatory bodies to use internal model-based approach to market risk measurement and based on such assessments, to determine the capital adequacy. Therefore, the regulator has developed a "traffic light" approach for the backtesting model. However, the last financial crisis has shown that this approach of model validation has failed to provide valid assessments of the VaR model, which has further led to significant underestimation of the risk and a collapse of many banks worldwide. For this very reason, the academic literature has been focused more than ever before, on proposal and development of new techniques for the verification of the risk model. Therefore, the aim of this paper is to provide a comparative overview of the validation methods of the market risk model, which have evolved during the past several years.

Headnote

Abstract

Within this paper we shall research the validation methods of the risk model and we shall provide an overview of the existing literature which deals with validation and performance assessment of the VaR (Value at Risk) model. The importance of backtesting of the risk model stems from the fact that credit institutions have been allowed by regulatory bodies to use internal model-based approach to market risk measurement and based on such assessments, to determine the capital adequacy. Therefore, the regulator has developed a "traffic light" approach for the backtesting model. However, the last financial crisis has shown that this approach of model validation has failed to provide valid assessments of the VaR model, which has further led to significant underestimation of the risk and a collapse of many banks worldwide. For this very reason, the academic literature has been focused more than ever before, on proposal and development of new techniques for the verification of the risk model. Therefore, the aim of this paper is to provide a comparative overview of the validation methods of the market risk model, which have evolved during the past several years.

Key words: Value at Risk, market risk, backtesting, bank

JEL classification: C15, C22, C52, G21, G32

...

(ProQuest: ... denotes non-US-ASCII text omitted.)

(ProQuest: ... denotes formulae omitted.)

Introduction

It is important for a financial institution to monitor the volatilities of the market variables (interest rates, exchange rates, equity prices, commodity prices, etc.) on which the value of its portfolio depends. Financial institutions are faced with various forms of risk. In order to control, manage and measure risks, they have been actively engaged in the financial risk management process. The risk management function contributes to better risk management within banks, through continuous measurement of risk of the current portfolio of financial assets and other exposures, as well as by taking steps, either directly or in cooperation with other functions of the bank, in order to reduce the possibility of loss. From the regulator's perspective, the size and the risk of the bank's assets are one of the most important determinants for defining the amount of the necessary capital of the bank.

The globalization of the financial market has led to the need for the globalization of the supervision system of the financial sector. Regulatory bodies are in charge of protection of the financial system against catastrophic events, which could be source of systematic risk. In the last couple of years, the central issue of risk management is to determine the capital adequacy for financial institutions, in order to protect themselves against the market risk. The assessment of the market risk is a complex and an extremely important task for each and every credit institution. This increased focus on risk management has led to the development of various methods and tools for the risk measurement, which financial institutions are faced with.

Financial risk management has truly undergone a revolution in the last couple of years, which has been intiated by the introduction of Value at Risk (hereinafter: VaR), a new method for measuring one aspect of the financial risk, the market risk. In the light of the recent financial crisis, measuring of the market risk is drawing even more attention and is becoming more and more significant. The last global, financial crisis has shown that systems for management and calculation of exposure to such risks, have significantly failed, and has therefore forced banks to take certain steps for the purpose of forming efficient internal approaches and methods for market risk management. Risk managers are attempting to revise the previous methods, given that they feel that one of the most important causes of the recent crisis is the actual poor risk management.

Market risk arises primarily from the bank's trading operations. It is the risk relating to the possibility that instruments in the bank's trading book will decline in value. (Hull, 2012). The trading book marks the positions within the business books of the bank which refer to the financial instruments and the real assets. These are intended for trade or hedging of other elements of the trading book and for such there are no restrictions in regard to their trade, nor restrictions for these positions to be protected against risk (Base Committee on Banking Supervision, 2009).

The VaR models measure the market and the price risk of securities portfolio, that is, the risk of decrease in portfolio market values, as a result of changes in the movement of interest rates, foreign currencies, prices of securities and the price of commodities. The VaR models encompass several components of the market risk into one quantitative measure of potential losses within a given time horizon. So, the model for assessment of the market risk is the model that envisages the value at risk of the portfolio for one or several confidence levels, during a specific time horizon. In practice, horizon is most often defined as one day (trading day). However, the calculation of VaR is a complex task, which includes numerous mathematical and statistical assumptions. Given that they cannot be always fulfilled, VaR models must be subjected to the backtesting process, with the help of various statistical tools.

Backtesting is an important part of the VaR system. It examines how well the VaR model would have performed in the past. Both literature and practice have developed two most frequent manners of model validation, which could refer to potential weaknesses of the VaR model (Kupiec, 1995; Christoffersen; 1998). One refers to the number of outliers, i.e. the number of times when the realized loss exceeds the VaR value. The other manner refers to the extent, in which outliers have been grouped, i.e. whether in time, they have become independent. There are statistical tests which can help determine the accuracy of the VaR model and suggest whether a model should be rejected due to excessive or insufficient number of outliers or due to their frequent grouping.

One of the major disadvantages of adequate measuring of exposure to market risk, by the Serbian banks, in regard to the securities trading activities or calculation of the currency and interest rate risk, primarily lies in the insufficient use of internal models for risk measurement, which are based on VaR methodology. In that sense, the contribution of this paper is to support and suggest the local banks the introduction of sophisticated internal risk models, as well as reliable techniques for validation of their accuracy and reliability when assessing market risks.

The remainder of this paper is organized in the following way. The second part of the paper elaborates on basic concept and the objectives of the backtesting process, as a critical process in financial risk management and the assessment of performance of the risk models. The third part provides an overview of the several, most frequently used and standard techniques and tools, for the purpose of validation of the VaR model. We shall also present the backtesting methodologies which have been proposed and developed during the last world crisis and present the latest state of the art techniques of the risk metrics validation. The fourth part provides a brief conclusion and recommendations for future researches.

Backtesting

In the past twenty years, the banks have allocated significant funds and resources for the development of internal risk models for the purpose of better quantification of financial risks and determination of the necessary economic capital. These efforts have been acknowledged and supported by regulatory bodies. Thus, the amendment to the Basel Accord from 1996 (MRA), which referred to the market risk, has formally incorporated internal models of banks for market risk, when calculating the regulatory capital. The regulatory capital requirements for exposure to market risk are exclusively the function of the bank's own VaR assessment. The key component in implementation of MRA was the development of standards related to validation and verification of models (backtesting), which must be fulfilled, so that the bank's models could be used for the purposes of regulatory capital.

The Basle Committee believes that backtesting offers the best opportunity for incorporating suitable incentives into the internal models approach in a manner that is consistent and that will cover a variety of circumstance (Basel Committee on Banking Supervision, 1996).

In finances, the term "backtesting" is used in several different manners. Most often, backtesting implies 1) assessment of performance of trading strategies in the past or 2) assessment of the financial risk model by using historical data for risk forecasting and comparison with the realized rates of return (Christoffersen, 2009). In order to be sure of the reliability of the VaR model, their validation needs to be carried out. This further implies that the critical issue when assessing the risk model is the backtesting. Backtesting requires the application of quantitative, most often statistical methods, for the purpose of determining whether a model for the assessment of risk is adequate or not. The backtesting process can be used for three complementary purposes.

The first objective of the backtesting process is to determine whether the assessments have come close enough to the realized outputs, in order to enable the reaching of the conclusion that such assessments are statistically compatible with the relevant outputs. The backtest, which has been carried out for this purpose, involves statistical testing of hypothesis, in order to determine whether the assessment models are acceptable. The testing of hypothesis can be applied to observations which include the loss exceeding the VaR value, for the given confidence interval, or for the assessment of VaR for several confidence intervals. The second objective of backtesting is to aid risk managers when diagnosing problems they are faced with within their risk models, so as to improve them. The third objective of backtesting is to rank the performances of several alternative risk models, in order to determine which model provides the best performance assessment. A good risk model should fulfill all of the three mentioned criteria: to pass the statistical test, it should not create any concerning diagnostics and it should be well-ranked compared to the alternative methods. The significance of backtesting is obvious: if risk managers have confidence in their risk models, than the models should be properly tested and in such case, should provide proper results.

VaR models for measuring of risks are useful if providing a reasonable anticipation of risk. Therefore, the accuracy of these models should always be verified. This can be done in several ways, including backtesting which represents a procedure for verifying whether the actual losses are in accordance with the projected ones. Backtesting includes comparison of historical anticipation of VaR with portfolio incomes and is very important for managers, in regard to evaluation of errors made in assumptions, wrong parameters and inaccurate modelling. It is a method for comparison of daily profits and losses, with assessments of VaR models, for the purpose of measuring their accuracy and preciseness. Also, according the Basel Accord, backtesting plays a significant role in deciding on the use of bank's internal VaR model for determing the required capital (Terzic, 2014).

If such model is correct, the number of realized losses suits the confidence interval, i.e. if the confidence level is 99%, then the actual loss occurred in 1 % of cases. For example, if the daily VaR is 1 million euro and the confidence interval is 99%, according to the VaR model we can expect for the loss to be grater than the 1 million euro in only 1% of cases, i.e. within 2.5 days of a total of 250 working days within a year. If the number of days on which the loss exceeds 1 million, is small, equal to or somewhat greater than 2.5, the model is then correct, but if the number of days when the loss exceeds 1 million is significantly greater than predicted,based on confidence level (2.5 days), the model then, is incorrect. The number of situations in which losses occur, i.e. the number of those incomes which are beyond the confidence interval of VaR, is known as the number of outliers. In case of numerous outliers, the model has underestimated the risk (Terzic, 2014).

In order to finalize the VaR model backtesting, we need a series of data which comprises of estimated model values, on the one hand, and daily profits and losses, generated by the portfolio, on the other hand. Upon collecting a series of necessary data, the following stage is approached, the preliminary data analysis. A backtesting diagram needs to be designed, which would comprise of the realized rates of return, during a specific time horizon and the estimated VaR, and breaches should then be seeked, i.e. outliers. Dowd (2008) suggests that good practice is to supplement the backtesting diagram with a histogram of returns, which sometimes tends to provide a clearer indication of the empirical distribution of returns as well as the QQ diagram, which contains a quantile empirical distribution of returns against those predicted by return distribution. Furthermore, it also states that it is good to examine the so called descriptive statistics of returns, including the statistics of mean value, variance, asymmetry, kurtosis, scope etc as well as the number and the size of the extreme events.

Many financial institutions use and implement various verification models. Thus for example, the KPMG Advisory has implemented a backtesting process based on five steps, shown in Figure 1, in order to test the unconditioned coverage, independence and has developed appropriate solutions for possible model weaknesses (Muehlenbrock, 2011).

As it can be seen in Figure 1, the first step in implementation of the backtesting model is the graphic analysis and it enables us insight into the resuls and provides visual aid in revealing the problems. For example, in Chart 1, the assessed GARTH VaR has been shown and changes in percentages in portfolio values.

The second step is the so called "traffic light system" which is based on a binomial approach and which groups results into various categories, starting with the green (correct model) up to the red (rejected model) zone. The previously described measures of VaR validation should be supplemented in the following step, with certain statistical tests. This strategy, based on the modern statistical theory can reveal potential weaknesses of the applied VaR model. The fourth step refers to the backtesting report which sums the results and points to the possible weaknesses of the used methodology. Finally, the last step, based on the prepared report, KPMG is able to develop an adapted solution for possible problems, i.e. to assist in implementation of the improved VaR model.

Review of backtesting procedures

VaR is by far the most popular portfolio risk measure, when it comes to risk management practice. The revolution of VaR in risk management has been initiated when JP Morgan launched the RiskMetrics approach in 1994. The supervisors immediately recognized the urgent need for VaR validation methods and soon after, first researches have been carried out on the risk model backtesting.

Many authors are concerned about the adequacy of the VaR measures, especially given the fact that they compare several alternative methods. Since the end of 1990s, various types of tests have been proposed for performance assessment of the VaR model. Papers, dealing with the comparison of the VaR methodology, most often use two alternative approaches: statistical test based on the testing paradigm of hypothesis and/or the loss function. In this paper we shall elaborate on the first approach. As for this approach, several procedures based on statistical testing of hypothesis, have been proposed in literature and the authors usually choose one or several test for the performance assessment of VaR models and their comparison. The standard tests for the perforamnce assessment of VaR models are: 1) Basel approach, 2) unconditional and conditional coverage tests and 3) quantile dynamics test.

In order for all of these tests to be implemented indicator function of VaR exceptions must be defined, the so call "hit sequence" (Christoffersen, 2009).

... (1)

The negative prefix which stands before VaR in the equation (1) is due to the fact tha VaR is a positive number, according to the convention.

The Basel Committee on Banking Supervision has, in 1996, developed a backtesting framework, based on the number of outliers during the 250 daily observances, generated by VaR models of banks for the confidence interval of 99%. Depending on the results, a supervisor may pronounce a penalty which would suit the increase in capital exposed to the market risk by a scaling factor. In order to support supervisors in interpretation of the backtesting results, the Basel Committe has introduced the so called "traffic light" framework, which is related to a number of marked outliers (Basel Committee on Banking Supervision, 1996):

1. The Green zone: between zero and four outliers. This is deemed to be an acceptable result of backtesting. There is no concern in regard to this model which the bank is using and consequently, there is no penalty.

2. The Yellow zone: between five and nine outliers. The supervisor shall attempt to find out what has caused deviation from VaR and shall then decide whether a bank should be fined or not.

3. The Red zone: 10 or more outliers. This points to a major problem within a model and automatically generates a penalty, with an increase in the scaling factor by 1.

According to the Basel Accord, backtesting of internal models is obtained directly from the testing of rates of failure, i.e. the number of outliers from VaR. In order to design one such test, we should first choose the type 1 error rate, which represents the probability of rejecting a model, when it is actually correct. In such situations, the bank shall not be fined unjustifiably and we could then be able to choose a test with a small error rate, type 1. However, should the bank decide so, the supervisory body making errors of type 2 as well, shall completely trick the VaR bank reporting. The current verification of the procedure comprises of the daily recording of outliers from VaR with a confidence level of 99% in the last year. In such circumstances, a 1% of outliers is expected out of the 250 cases, i.e. 2.5 outliers during a year. In order to better understand the dilemma with which supervisory bodies are confronted, Table 1 provides error types I and II for various numbers of outliers from VaR, with a correct VaR model (i.e. with coverage of 99%) and incorrect models (i.e. with coverage rate of 97% or 95%). Thus, for example, if we were to have 5 or more outliers, the cummulative probability or the type 1 error rate, amounts to 10.8%. This represents a probability of fining a bank which has a correct VaR model, for no other reason than "bad luck". However, if we were to have 10 outliers, type 1 error rate would than fall down to 0% value.

In regard to type 2 error, table 1 shows that the type 2 error rate by 5 outliers less, amounts to 12.8%. This represents a probability of not fining a bank which wilfully underestimates its risk. This not a very low probability. However, this probability reduces as the correct model deviates more and more from the target 99% of coverage. With 95% of coverage, the type 2 error rate is only 0.5%. Therefore, it is quite probable that the supervisor would fail to notice a bank which significantly underestimates its VaR.

Christoffersen (1998) emphasizes that the problem of determining accuracy of the VaR model can most certainly be reduced to the problem of whether a "hit sequence", ... fulfills two key characteristics, and these are the unconditional coverage and independence.

The test of unconditional coverage, which has been proposed by Kupiec (1995) enables testing so as to check whether the realized deviation rate from VaR, which represents the number of days when the loss was greater than VaR, divided by the size of the sample, is in line with the confidence interval. If we were to expect for the losses which exceed the amount of the established VaR, to occur more often than α × 100% times, then this leads us to the conclusion that VaR measure systematically underestimates the portfolio risk. On the other hand, if we were to excpect deviations to occur rarely, this would perhaps be a signal that the VaR measure is perhaps too conservative. Although this test has remained until today as one of the reference tests in managing financial risks, it nevertheless shows to a low statistical power, when used on a small data series, such as a one-year period.

As for independence property, it represents powerful limitation in regard to ways in which deviations from VaR can occur. More specifically, any of the two elements from the "hit sequence" (It+j(α), It+k(α)) must be independent one from another. This sort of condition does not require for previous deviation from VaR {..., It-1(α), It(α)},to transfer any sort of information as to whether additional deviation from VaR will occur or not. If, for example, there is a greater probability of deviation from VaR happening upon previous deviation from VaR, then it implies that probability that It+1(α) = 1, having been conditioned by the event It(α) = 1, exceeds the amount of VaR, which further implies that the amount of VaR is to small and that it should be increased. It is important to understand that the features of unconditional coverage and independence of the "hit sequences" are mutually separated and different and that they both need to be fulfilled for the purpose of a precise VaR model. Mainly, a specific VaR model, in case of outliers in VaR, may fulfill either one or the other feature, but not both. Only the "hit sequences" which fulfill both of the stated properties can be deemed as relevant evidence of a precise VaR model (Christoffersen, 1998).

Christoffersen (1998) has also developed a conditional coverage test, which represents an incorporated test of hypothesis of unconditional coverage and independence.

Christoffersen and Pelletier (2004) have developed a backtesting approach of the VaR model,based on durtion. If VaR complies with the coverage rate p, it is then assumed that the hit sequence should be the Bernoulli i.i.d process with parameter p, and the duration between the outliers has no memory and that the mean value equals 1/p. The distribution of duration under null hypothesis is approximated by exponential distribution, given that it is only continuous distribution with a constant risk rate. For an alternative hypothesis they have considered the Weibull's distribution with a decreasing risk rate. Their test can also be decomposed to a test of unconditional coverage and independence test, whereby the unconditional coverage test verifies whether the mean value of duration equals 1/p, and the independence test verifies whether the risk rate is constant. They have also considered the autoagressive model for the expected conditional duration. It is also possible to define a discreet distribution for duration. Haas (2005) believes that discreet distributions have a better power toward grouping of outliers from VaR, unlike the continuous distributions.

Candelon, Colletaz, Hurlin, and Tokpavi (2011) suggest a new GMM test based on duration, for the purpose of VaR model validation. They believe that discrete distributions act the same as the continuous distributions, within the GMM approach. Berkovic, Christoffersen and Pelletier (2011) have implemented a test of discrete distribution within the likelihood ratio test (LR test), which they called the geometric test. Under null hypothesis that duration has no memory, discrete duration follows geometric distribution. This is why it is called a geometric test. Monte Carlo simulation shows how a geometric test is the most powerful, compared to the Weibull's test, which is based on continuous distribution.

Engle and Manganelli (2004) claim that a necessary request is for the hit sequence, or rather the violations from the VaR assessment, to have identical and independent distribution, and not a sufficient requirement for proper determination of VaR. If VaR prognosis is a valid measure of quantiles, the anticipation of outliers which depend on the set of information at the moment t-1 should be equal to the coverage rate. In other words, violation from VaR It shold be unbiased and should not be in correlation with any other information at the moment t-1. They have suggested a dynamic quantile test(DQ) for the backtesting of the VaR model, which has proven to be very reliable and credible. Nowadays, it is an important tool for the verification of the VaR model.

Dumitrescu, Hurlin, and Fam (2012) have expanded this approach into a model of dynamic binary choice which enables non-linear dependance between deviation likelihood from VaR and explanatory variables.

Perignon and Smith (2008) have developed an innovative backtesting framework, based on multidimensional VaR, which focuses on the left tail of distribution of the bank's incomes from trading activities. Their coverage test is a multivariate generalization of the Kupiec unconditional test (Kupiec, 1995). They have applied this new methodology of backtesting on actual bank data and have concluded that non-parametric GARCH VaR models and filtered historic simulation provide the best performance in the assessment of market risk.

Danciulescu (2010) has proposed for backtesting to be based on multivariate of Ljung-Box statistics. The test considered autocorrelations and crosscorrelation between the outliers from VaR. The procedure encompasses the creation of a joint test for the properties of unconditional coverage and independence, using deviations from several business lines. The test is easily applicable and has shown improvement compared to the univariate procedures in the performance assessment of VaR.

Colletaz et al (2013) have developed a new method for validation of the risk model, called, risk map (RM). This method calculates both the number and the size of the extreme losses and graphically sums all information on performances of the risk model. Based on the concept of super outliers, which is defined as a situation in which realized loss exceeds the amount of the standard VaR and VaR defined at an extremely low confidence interval. The main advantage of RM lies in its simplicity and can therefore be applied as a standard technique for the validation of the VaR model. In order to facilitate the implementation of this methodology, the authors have created a website which automatically generates RM.

Leccadito et al (2014) have suggested innovative, multitests for the purpose of assessing the accuracy of the VaR forecasting. Tests are based on independence tests and conditional coverage. The first test which has been proposed is the generalization of Markov test, which was proposed by Christoffersen (1998), whereby the second is type of the Person test, based on joint distribution of the total number of outliers from VaR and their lagged values. The tests have shown greater efficiency and statistical superirority, unlike the separate tests.

Pelletier and Wei (2015) have developed a new geometric VaR backtest for the evaluation of VaR prognosis. The test uses duration between deviations from VaR as well as VaR values. Findings of the research have shown that the test has great power, compared to alternative models.

Conclusion

The banks and other financial institutions have implemented numerous highly-sophisticated mathematical and statitstical techniques for managing market risks. One of the basic techniques among them is the VaR methodology, which has in the past twenty years become an industrial and regulatory standard in measuring market risk. It is the measure regulators have chosen to use for many of the calculations they carry out concerned with the setting of capital requirements for market risk. Many banks that have adopted VaR based approach to market risk measurement routinely compare daily profits and losses with model-generated risk measures to gauge the quality and accuracy of their risk measurement systems. This process, known as backtesting, has been found useful by many institutions as they have developed and introduced their risk measurement models. There are two ways in which backtesting may indicate weaknesses in a VaR model. One is in the percentage of exceptions, that is, the percentage of times the actual loss exceeds VaR. The other is in the extent to which exceptions are bunched. There are statistical tests to determine whether a VaR model should be rejected because of the percentage of exceptions or the amount of bunching. In this paper we have focused on the significance of validation of the VaR model and we have presented the latest, available, backtesting techniques. The tests differ in details, but for the majority, the common characteristic is that they focus on a specific transformation of the assessed values of VaR and ralized incomes. The backtesting procedures that we have presented in this paper can be deemed to be the final dignostics of the risk model, which the risk managers must implement or they can be used by external supervisors. The banks in Serbia should use the findings of this paper as a starting point for the validation of internal models for market risk.

References

References

Basel Committee on Banking Supervision (Januar 1996). Supervisory Framework for the Use of 'Backtesting' in Conjunction with the Internal Models Approach to Market Risk Capital Requirements

Basel Committee on Banking Supervision (July 2009). Guidelines for Computing Capital for Incremental Risk in the Trading Book

Berkowitz, J., Christoffersen, P., & Pelletier, D. (2011). Evaluating value-at-risk models with desk-level data. Management Science, 57(12), 2213-2227.

Candelon, B., Colletaz, G., Hurlin, C., & Tokpavi, S. (2011). Backtesting value-at-risk: a GMM duration-based test. Journal of Financial Econometrics, 9(2), 314-343.

Christoffersen, P. (1998). Evaluating interval forecasts. International economic review, 39(4), 841-862.

Christoffersen, P., & Pelletier, D. (2004). Backtesting value-at-risk: A duration-based approach. Journal of Financial Econometrics, 2(1), 84-108.

Christoffersen, P. (2009). Backtesting, In Encyclopedia of Quantitative Finance, R. Cont (Ed), pp 355-389, Chichester, John Wiley and Sons.

Colletaz, G., Hurlin, C., & Pérignon, C. (2013). The Risk Map: A new tool for validating risk models. Journal of Banking & Finance, 37(10), 3843-3854.

Danciulescu, C. (2010). Backtesting value-at-risk models: A multivariate approach. Center for Applied Economics & Policy Research, Working Paper, No.004-2010.

Dowd, K. (2008). Back-Testing Market Risk Models. In Handbook of Finance, pp. 93-99, New York, John Wiley and Sons.

Dumitrescu, E. I., Hurlin, C., & Pham, V. (2012). Backtesting value-at-risk: from dynamic quantile to dynamic binary tests. Finance, 33(1), 79-112.

Engle, R. F., & Manganelli, S. (2004). CAViaR: Conditional autoregressive value at risk by regression quantiles. Journal of Business & Economic Statistics, 22(4), 367-381.

Gaglianone, W. P., Lima, L. R., Linton, O., & Smith, D. R. (2012). Evaluating value-at-risk models via quantile regression. Journal of Business & Economic Statistics. 29(1), 150-160.

Haas, M. (2005). Improved Duration-Based Backtesting of Value-at-Risk. Journal of Risk, 8(2), 217-38.

Hull, J. (2012). Risk Management and Financial Institutions. New Jersey, John Wiley and Sons, Hoboken

Kupiec, P. H. (1995). Techniques for verifying the accuracy of risk measurement models. The Journal of derivatives, 3(2), 73-84.

Leccadito, A., Boffelli, S., & Urga, G. (2014). Evaluating the accuracy of value-at-risk forecasts: New multilevel tests. International Journal of Forecasting, 30(2), 206-216.

Muehlenbrock, S. (2011). Backtesting Value at Risk Models. KPMG ADVISORY FINANCIAL RISK MANAGEMENT.

Pelletier, D., & Wei, W. (2015). The Geometric-VaR Backtesting Method. Working Paper, Journal of Financial

Econometrics, forthcoming.

Pérignon C., Smith D. (2008). A New Approach to Comparing VaR Estimation Methods, The Journal of Derivatives, 16(2), 54-66.

Terzic, I. (2014). Savremene metode merenja rizika na trzistu kapitala u Srbiji. Neobjavljena doktorska disertacija, Univerzitet Singidunum, Beograd, Srbija.