6 Reasons Why Backtesting Works

Last Updated on September 14, 2021 by Oddmund Groette

Backtesting is a tool that not only small retail traders use but also big institutions. The world’s most successful hedge fund, Jim Simons’ Medallion Fund, uses backtesting continuously to develop new strategies. Backtesting works!

Why does backtesting work? We have done daily backtesting for over 20 years and this article summarizes what we consider the main reasons why backtesting works. Backtesting works because you can falsify or confirm a trading idea, you can automate all your trading based on the backtests, exploit the law of large numbers, limit behavioral mistakes, and lastly you can save a lot of time in executions.

We start by explaining briefly what a backtest is:

What is a backtest?

This website is all about backtesting. But what is a backtest?

A backtest has strict rules for when to buy and when to exit. In other words, you can code the strategy and find out with 100% certainty how the strategy has performed in the past. Thus, this is a backtest on historical data.

Of course, this doesn’t give any certainties about the future, but you know if the strategy has performed well or poorly in the past. If something has performed poorly in the past it’s unlikely that it will perform well in the future.

Opposite, if a backtest proves that your idea has worked well in the past, it most likely will perform better than any idea that has performed poorly.  But of course, a positive backtest is no guarantee that it will work in the future. But we believe it’s the best indication you can get.

A backtest follows this procedure:

  1. Find an idea you want to test.
  2. Define clear and concise entry and exit parameters – they need to be quantifiable.
  3. Specify the market you want to test on.
  4. Specify the time frame you want to test.
  5. Code the strategy.
  6. Run the strategy on the in-sample period.
  7. Test the out of sample backtest.

This is all there is to it.

If the backtest returns a positive and promising result, we recommend that you paper trade the strategy for several months before you commit real money. This can save you a lot of money! We have written more about our procedures in our trading lessons based on 20 years of full-time trading and investing.

Backtesting works because you can confirm or falsify a trading idea

Backtesting works because you can easily check if something has worked in the past or not.

  • Is, for example, the Turnaround Tuesday true or just a myth? Just define the rules and start the backtest. You’ll find out in five minutes.
  • Do you believe you see an interesting pattern in the chart? Then quantify it with strict buy and sell rules and test it.

Did the strategy work in the past? If something has not worked in the past, you can easily falsify your hypothesis and go on to test another idea.

Because most ideas don’t work, you should not spend much time testing a hypothesis. Many traders waste months, even years, in both programming software and tweaking their strategies only to find out it was a waste of time. You don’t need “perfect” strategies to make money in the markets. You need many strategies that complement each other.

We have written multiple times on this website that one of the main reasons for your success (or no success) depends on your ability to test and generate trading ideas. One of our main trading lessons is that we spend about 80% of our time testing ideas back and forth between ourselves.

Backtesting works because it lets you automate

If you have successfully backtested a strategy, you can easily go “live” with the strategy. In Tradestation, you just check a box and you are good to go.

In Amibroker, you need to add code to automate and let Amibroker keep track of your positions and strategies.

Obviously, this saves you a lot of time. There is no need to check quotes, prices, or follow the markets. The computer does all the work!

Backtesting works – you can exploit the law of large numbers

Your computer can easily trade and supervise hundreds of strategies. This lets you exploit the law of large numbers and you can diversify into time frames, asset classes, directions, and types of strategies.

The main reason for the success of the Medallion Fund is twofold: they use enormous amounts of data to generate hundreds of uncorrelated strategies. Because of the low internal correlation among the strategies they can use leverage to boost returns.

However, leverage is dangerous and certainly not something we recommend. Only use leverage if you have many years of experience.

Backtesting works because it removes emotions

Ample evidence points out that individual investors underperform the averages, and women are better investors than men. The main reason for this is behavioral mistakes:

Investors tend to sell into a panic and buy after a big rise. Most of the time you need to do the complete opposite. A backtest can’t capture such mistakes and that’s why you need to stick to the trading plan.

In order to stick to the trading plan, you need to trade smaller than you’d like or prefer. This is the best way to keep detachment to money.

Likewise, women do better because they save, invest, and forget about it. They are not trying to be geniuses! They don’t have any ego.

The closer you follow the markets, the more likely you are to overrule your systems when your “intuition” tells you to sell or buy. But most of the time the intuition is plain wrong, unfortunately.

Overruling your systems and strategies is unlikely to work. You have not backtested overruling, so how do you know if it works?

Backtesting works because it saves time

Today, the 14th of September 2021, we tested about 15 hypotheses in Silvers Miners (ticker code SIL) in about 1.5 hours. One of those ideas seems promising, the rest probably just a waste of time. You can generate and test hundreds of strategies in just a single day. Even better, you can falsify or confirm the ideas quickly.

Trading is mainly about trial and error. And luckily, backtesting is a great tool for that and at the same time, it saves you a lot of time.


 

If you like this article please follow us on one or all of our platforms:

  • By registering your e-mail in the upper right corner of this webpage (for our newsletter)
  • Follow our Twitter account
  • We also recently started a Substack service. 

Disclaimer: We are not financial advisors. Please do your own due diligence and investment research or consult a financial professional. All articles are our opinions – they are not suggestions to buy or sell any securities.