Backtesting is one of the most important steps in building a successful quantitative trading strategy. It is in fact a key step that differentiates algorithmic trading from discretionary trading. Quants use their computational finance and programming skills to build complex trading strategies. However, before these strategies are executed in the live market, they are tested using historical data. Basically, the traders will feed the historical data into these algorithmic trading programs which will tell them how well their strategies performed on this historical data. This refers to backtesting and can help traders in finding flaws in their trading strategies and improvise them.

It’s also important to note that backtesting is different from simulation because backtesting relies on historical data which is real, unlike the simulated data. Also, the goal of backtesting here is to work and build a profitable trading strategy compared to stress-testing which may rely on simulated data.

Importance of Backtesting

It is important to backtest a trading strategy whether you are building a strategy from scratch or whether you are trying to implement a strategy you got from someone. In both cases, you want to be sure that the strategy works without risking real capital. You also want to be sure that you understand the strategy well and understand the risks involved. You may also want to improve the performance of the strategy by backtesting it and making small changes based on your learning from the backtests. This, however, will depend on your own intuition and knowledge about the market. Generally, when you obtain a new trading strategy such as from an academic paper, it will also have a stated performance. Using backtesting you can also ensure whether those stated goals are met or not, and if not met, what could be the reasons. The backtesting process also helps you get to know every small detail of the strategy, such as when to enter and exit, how much capital, how to prepare data, and so on. This kind of attention to every detail of the strategy separates a profitable one from a losing one.

Limitations of Backtesting

Backtesting has its own limitations. We know that history is not necessarily a good indicator of the future. Lots of things can change as time passes. The market dynamics may change, the government regulations may change which may leave the strategy useless. Similarly, the economic conditions may differ vastly compared to the past. However, with backtesting, our goal is to increase the likelihood that future performance will be as close to the backtested performance as possible. This can be done by following the best practices such as avoiding backtesting pitfalls to avoid inflation of backtesting results, keeping your strategy simple and following a solid underlying trend, not to overfit the model, conducting out-of-sample testing, and other practices. 

To Backtest Pitfalls - To Backtest or Not

Backtesting a trading strategy is a lot of hard work and a time-consuming process. As an active trader, you may come across new strategies every day. So, even before putting in any effort into a strategy it is possible to do a basic analysis of the strategy to understand whether the strategy is even worth backtesting. Let’s look at some of these methods of investigating a strategy.

• The first thing is to understand the pitfalls of backtesting. There are backtesting pitfalls which can make a strategy look great on paper but when you actually use it, it may not perform as stated. You can investigate if a strategy has incorporated these pitfalls from the beginning itself. If that’s the case you needn’t even go through the trouble of backtesting it. For example, if a strategy shows high backtest returns but has a low Sharpe ratio, it's not a good sign because a low Sharpe ratio indicates that the high returns may just be a random thing with this strategy. A high Sharpe ratio, on the other hand, indicates that that strategy has indeed produced lots of profitable trades. 
• Another important thing is to always keep in mind the benchmark. What if your trading strategy claimed a 20% return but the benchmark itself would provide, say, 30%. In that case, you would have been better off with a simple buy-and-hold strategy. So, it’s always important to remember what your benchmark is, especially for long-only or short-only strategies.
• Another important thing to keep in mind is the transaction costs involved in executing a trading strategy. If it happens to be a strategy that has a very short holding period, the transaction costs can be high which will eat into your strategy’s profits. Many times the authors who publish the trading strategies may not disclose this for various reasons. However, it’s quite important to consider transaction costs as per your setup while you analyze the strategy. 
• Another important factor that can inflate backtesting performance is ‘survivorship bias’. This is the situation where bankrupt stocks are removed from the index, and the past index values are adjusted to remove the data of the dropped stocks. While backtesting it is important to include these stocks for the specific period data. 

In backtesting, we always want to avoid a false positive. It is much better if a strategy performs poorly in a backtest than to exhibit false performance.