Backtesting Trading Strategies: How To Backtest A Strategy

Backtesting trading strategies refers to the process of evaluating the performance of a trading strategy using historical data to simulate how it would have performed in the past.

The steps involved in how to backtest a trading strategy include defining the criteria of the strategy, selecting the market and timeframe to test it on, loading up the historical data, writing the code and implementing the backtesting, and finally, evaluating the results.

In your day-to-day life, before you buy anything — a mobile phone, a house, or a car — you would want to check its features and history to know if it is worth your money. That also applies to trading, but here, what you check are your trading strategies. With a wide range of markets to trade, you need to backtest your trading strategies to be sure they work.

Key Takeaways:

• Backtesting validates trading strategies on historical data to predict future performance.
• ✅ Backtesting course – including one trading strategy
• It’s crucial for discovering if a strategy offers a market edge.
• Strategies must have clear, quantifiable entry and exit rules for effective backtesting.
• Both free and paid tools are available for backtesting, with platforms like Amibroker and Excel being popular choices.
• Professional traders rely on backtesting to refine and test their strategies before live implementation.

A backtest is a tool that not only small retail traders use but also big institutions. The world’s most successful hedge fund, Jim Simons’ Medallion Fund, uses backtesting continuously to develop new strategies. Why? Because backtesting a trading strategy works!

Backtesting a trading strategy works because you can falsify or confirm a trading idea, you can automate all your trading based on the backtests, exploit the law of large numbers, limit behavioral mistakes, and lastly, you can save a lot of time in executions. Backtesting is not a waste of time! We have done backtesting daily for over 20 years, and this article summarizes the main reasons why you should backtest and why it works.

What is a backtest?

A backtest is a way of testing a trading strategy on historical data to find out how it has performed in the past. It is a way to simulate the historical performance of a trading strategy using historical data before committing real funds to the strategy on live trading.

Backtesting trading strategies is based on the assumption that if the strategy performed well in a particular market previously, it has a good chance of working again. And on the flip side, if the strategy did not perform well in the past for that market, it may not work well in the future.

So, let’s say you have a swing trading strategy that says that if the S&P 500 Index has a positive return in the past month, it will give a positive return over the next week. Now, you want to test that theory to know if true or not. While you can forward-test it by using a demo account and waiting for the positive months to come (you check how the market performs in the month that follows each of them), that would be very time-consuming.

A faster way to test your theory is to test it on historical price data. You go into the past to find all the times the market has a positive monthly return and then check how the market performed in the week or month that followed.

If the performance on historical data is good, you might assume that the strategy would perform well in the current market, but if the performance is poor, you may discard the strategy or tweak and re-test it. You can, of course, backtest any time period you like, the point is to measure how your predictions on past data work on future unknown data.

Thus, backtesting is a very important step in creating a trading system. It can help a trader test, optimize, and improve their strategies, thereby giving them the confidence to apply the strategy in live trading.

A backtest has strict rules for when to buy and when to exit. In other words, you can code the strategy and find out with 100% certainty how the strategy has performed in the past. Thus, this is a backtest on historical data and strict trading rules. That is why it’s called a “back test” (history). We can argue it’s a kind of quantified technical analysis – technical analysis backtesting.

Of course, this doesn’t give any certainties about the future, but you know if the strategy has performed well or poorly in the past. If something has performed poorly in the past, it’s unlikely that it will perform well in the future. However, the ever-changing market cycles make strategies perform well in certain markets, and poorly in others. This is the reality of trading.

Opposite, if a backtest proves that your idea has worked well in the past, it most likely will perform better than any idea that has performed poorly.  But of course, a positive backtest is no guarantee that it will work in the future. But we believe it’s the best indication you can get.

General StepsTo Make A Backtest

1. Find an idea you want to test.
2. Define clear and concise entry and exit parameters – they must be quantifiable.
3. Specify the market you want to test on.
4. Specify the time frame you want to test.
5. Code the strategy.
6. Run the strategy on the in-sample period.
7. Test the out of sample backtest.

This is all there is to it.

If the backtest returns a positive and promising result, we recommend that you paper trade the strategy for several months before you commit real money. This can save you a lot of money! We have written more about our procedures in our trading lessons based on 20 years of full-time trading and investing.

Why backtest a trading strategy?

You should backtest a trading strategy to find out if you have a positive expectancy, to improve a trading strategy, or to find the correlation to your other trading strategies.

Let’s explain:

• To know if you have a trading edge in the market (what is a statistical trading edge?): Many traders, even experienced traders, lose money, and they don’t lose money because they don’t know how the market works. But simply because their trading strategies don’t have any statistical edge in the market they are trading. No matter how you master trading discipline, control your emotions, and manage risk, you cannot make money from a market unless your strategy has an edge. If you want, risk only 0.1% of your account per trade, without an edge, the account would slowly bleed out. The only way to know if your strategy has an edge in a market is by backtesting it. The likelihood of trading profitably in the market is boosted if you backtest your strategy before trading. Emotional control and risk management only help a profitable strategy to make money — they can’t do anything if the strategy has no positive expectancy.
• To improve a strategy: Another reason to backtest a strategy is to see how you can tweak it to improve its performance. This involves altering some variables or adding new variables to see how they improve performance. The process is sometimes called optimization. Too much of it can lead to curve-fitting and dimmish the strategy’s robustness.
• Check for correlation in trading: for those more advanced, you might want to backtest a portfolio of trading strategies. You see, even if you have 20 very good backtested trading strategies, they might not work well together if they have many overlapping trades.

Who should consider backtesting?

Every trader should consider backtesting. The reason is simple: there is only one way to determine whether a trading strategy is profitable, and that is by gathering statistics, data, and numbers to determine whether the strategy has made money before. If not, why should the strategy start making money now?

Opposite, if the strategy has been profitable, you might have something very valuable.

How do you backtest a trading strategy?

To backtest a trading strategy you should make trading rules, select markets to backtest on, gather data, backtest it, and then evaluate your backtest.

Let’s look at how you backtest a trading strategy in more detail:

1. Define the logic of the strategy: The first step is to define the strategy you want to backtest and clearly state the criteria for each action with specific trading rules. The logic you want to backtest should be clear. For example, if your strategy is a moving average crossover system, you clearly state the type of moving average, the period of the two moving averages, and the entry and exit criteria. The entry and exit logic could be (as an example):

• When the 20-day SMA crosses above the 100-day SMA, exit any short position and enter a long position
• When the 20-day SMA crosses below the 100-day SMA, exit any long position and enter a short position

2. Select the market(s): At this stage, you specify the market you want to test the strategy on, as well as the time frame. The market you backtest must be the one you want to trade the strategy. You don’t use a strategy backtested on the silver market to trade the gold market, even though both are precious metals. Note that you can backtest the strategy on different markets and on different time frames and choose the ones with the best performance, but beware of curve fitting.
3. Get the data: You will need historical data of the market you want to backtest. While some trading platforms offer some data, they may be limited. You may have to pay to get enough historical data for your backtesting. You have to divide your data into two — in-sample data for backtesting and out-of-sample data for optimization. However, be careful about the data you backtest on. Your backtest is only as good as the quality of your data! There are plenty of bad data out there.
4. Implement your backtesting: You may have to learn a programming language like Python, but some trading platforms like TradeStation and Amibroker use languages that are easy to use.
5. Evaluate your results: After the backtesting, you need to evaluate your result to know how the strategy performed. Here are some of the performance metric statistics used in trading:

• The total number of trades: This tells you how many trades were taken during the period under study. The higher the number of trades, the more reliable the result.
• Holding time: This is the length of time that the strategy has open positions in the market. The lower the holding time, the better.
• Profit factor: This is the total profit divided by the total loss for the entire trading period. It indicates the amount of profit per unit of risk. A profit factor of more than 1 indicates a profitable strategy, but the higher the better. Note that on some platforms, this is presented as “Average size of winner/loser”.
• Maximum drawdown: This is the difference between the higher amount and lowest amount the trading capital got to during the backtesting. The lower the maximum drawdown, the better the strategy.
• The final part of a backtest is to backtest out-of sample or incubate the strategy (more about this later in the article). This involves testing the data on unknown data, ie finding out how well the backtest predicts the future unknown data.

You can backtest a trading strategy on a trading platform or in a spreadsheet. Most trading platforms have a strategy tester section where you can backtest your strategy, but not all of them are free to use.

Alternatively, you can use Excel or a spreadsheet, which is a free and very good backtesting tool.

How to backtest a trading strategy – examples

Here is an example of how you backtest a trading strategy:

Let’s say you have a swing trading strategy that says that if the S&P 500 Index has a positive return in the past month, it will give a positive return over the next week. You can backtest it easily to find out if it makes money. In this case, your market is the S&P 500 Index — you can test it on e-mini S&P 500 futures or the SPY ETF (SPY ETF trading).

Your strategy rules are:

• Go long on the next day’s open if the monthly bar closes positively
• Exit at the close of the trading week

You can get the market data of more than 60 years since it is a monthly signal. After testing, you evaluate the performance using stats like profit factor, Sharpe ratio, maximum drawdown, or any other statistic that measure the performance of a trading strategy. If the performance on historical data is good, you can assume that the strategy would perform well in the current market, but if the performance is poor, you may discard the strategy or tweak and retest it.

How to backtest a trading strategy – Turnaround Tuesday strategy

Let’s end this article section and make another specific backtest with trading rules and settings. The strategy is called the Turnaround Tuesday strategy and is one of the most well-known strategies there is, yet it’s still working pretty well.

We make the following trading rules:

1. Today is Monday.
2. The close must be at least 1% lower than Friday’s close.
3. If one and two are true, then enter at the close.
4. Exit at the close on Tuesday.

This is an overnight trading strategy that owns S&P 500 (SPY) for 24 hours.

This is what the equity curve looks like:

The 171 trades have an average gain of 0.67%, which we consider excellent considering you only hold the position for 24 hours! It has worked pretty well even during bear markets.

We did this backtest in less than two minutes by using Amibroker. This shows the power of validating or scrapping your trading idea by backtesting. This is why backtesting is so powerful!

If you become a member, you get access to our premium backtested trading strategies.

How to backtest a trading strategy for free?

You can backtest a trading strategy for free using software like Microsoft Excel (Libre Office), TradingView, NinjaTrader, Trade Brains, etc.

As a retail trader, you may want a free platform for backtesting, and there is some free software available in the market for backtesting a trading strategy.

How to backtest a trading strategy on paid platforms?

You can backtest trading a trading strategy on many paid platforms. The paid platforms may offer more features than the traditional free spreadsheet. One of the most commonly used paid platforms is Amibroker – a platform we have used for many years. The analysis window of the Amibroker platform allows you to back-test your trading strategy on historical data. At about 450 USD for a lifetime license, the platform is pretty cheap, and it offers full customization features for backtesting and some lightning-fast optimization features. It is easy to test strategies on a portfolio level with the platform.

• Amibroker review

Practically all of the backtest done on this website are done by Amibroker or with TradeStation. We are extremely pleased, and we also use it for live trading by adding just a little extra code. Considering the cheap and its features, we believe Amibroker is a good option for most retail traders. Based on our experiences, we have created an online Amibroker course that consists of several modules (basic, live trading, and trading strategies):

Other interesting and relevant articles about backtesting platforms are found in our trading platform guide.

Backtesting course – Quantified Strategies

But first, a humble reminder that we have made a backtesting course for beginners. We have been trading (successfully) and backtesting since the end of the 1990s, and thus have developed a pretty extensive knowledge about backtesting and historical analysis of trading strategies. We have compiled that knowledge into an inexpensive backtesting course.

Related Reading: Time Series Analysis Strategy

How long back in history should a strategy be backtested?

A trading strategy should be backtested for as long back in history as possible. However, it depends on a lot of factors. It is easy to say backtest for a period of one year or two, but statistically, it is not only a question of duration but also of sample size. That said, we recommend including several types of markets, like bear and bull markets. Bull and bear markets behave differently.

When backtesting, the key thing is that you want to have enough sample size (the number of trades or observations) that can be representative of the population (in this case, the market).

So, in terms of duration, you would need as much time as can give you enough trades to have a statistically reliable result. While a sample of 250 trades may be sufficient, the bigger the sample size is, the smaller the margin of error (in most cases), and the more reliable the result. If your trading system generates enough trades, a sample of 500 – 750 trades is good. The best is to have both a large sample size and a long test period.

The number of trades you get would depend on the strategy and how many trades it generates per day, week, or month. For a day trading strategy that generates a trade per day, it means you get about 250 trades per year, so backtesting over a 2-3 year duration should be able to give you enough data to make reliable assumptions, yes?

Not so fast. The problem is that 2-3 years is a short period and normally only included just one business cycle. We like to backtest using about 20 years, so we are sure we have included bear markets in our dataset.

On the other hand, a swing trading strategy that generates only four trades per month would get you only about 48 trades per year. So, to get up to 500 trades sample that would ensure reliable assumption, you will have to backtest over a period of more than 10 years.

Thus, having fewer trades over a longer time span might be better than having many trades over a shorter time span.

How do you backtest a strategy in Excel or a spreadsheet?

You backtest a trading strategy in Excel or a spreadsheet by getting data, loading the data into Excel, making the trading rules, coding the trading rules, and then backtesting it.

Let’s explain:

• Get your data into Excel: There are two basic ways to do this: The first option is to manually download historical data from Yahoo Finance directly as CSV and then load it into Excel. This works, but you’ll need to re-download that historical data and then copy and paste either the entire dataset or a subset to update your strategy. The other option is using code to automatically grab data from Yahoo Finance. You can search on Google for some VBA codes you can use. Some third-party tools, like AnalyzerXL, can make the job simple.
• Create your indicator: Now that you have got your data into Excel, you use them to construct an indicator or indicators for your strategy. You can use different columns to create different indicators or different aspects of an indicator, with each one taking part in the calculation. You can use other add-ons or third-party tools for more functionality.
• Formulate your trading rules: After creating your indicator, you need to formulate your trading rules. You create the calculation in a new column you name the “Signal” column. Your trading rule can be as simple as going long if the calculation is above a certain level and short if below it. But you could also have more complex rules with criteria for when to vary the position size or when to stay out of the market.
• Get the equity curve: You can get the equity curve with the Excel function, but if you need other trading stats like CAGR and the Sharpe ratio to compare the strategy with other strategies, you will have to do a bit of work or use a third-party tool like AnalyzerXL.

We used Excel as our main backtesting tool all the way up to 2017. Excel is a lot better than most traders imagine! You don’t need any fancy tools to backtest, the main asset is, after all, you, who put in the trading rules. As a matter of fact, Excel can be a very useful tool because you, in most cases, need to test a strategy on one instrument at a time. This way, you see small details you otherwise wouldn’t.

We even used Excel for live trading. We had a VBA script that we used for both sending orders, but also when we closed our positions. It worked really well, but it’s not as dynamic as dedicated software platforms (like Amibroker, Tradestation, etc.).

Do professional traders backtest?

Yes, professional traders always backtest their strategies before deploying them. They know how important backtesting is to the profitability of a trading strategy and cannot afford to make the mistake of trading a strategy that is not backtested. Some even go ahead to forward-test their strategies — if they are intraday trading strategies that can generate enough trades within a short period — before committing real money to them.

Even professional traders who use discretionary trading methods still backtest their strategies. They do so manually by either going back in time to check the occasions where their trade setups occurred and how the market reacted. All serious traders keep a trading journal. Alternatively, they can use strategy tester software that prints historical data as though they are in real-time and then trade their setups as they occur.

Can you backtest for free?

Yes, you can backtest for free, for example by using Excel or TradingView. Some trading platforms allow you to test your strategy for free, but you may have to look for historical data from market data vendors to get enough data. Some of the free backtesting platforms include TradingView, NinjaTrader, and so on.

Alternatively, you can use free software like Microsoft Excel, which enables you to backtest with the Excel function. However, you have to source your data elsewhere. A common source of data is Yahoo Finance, where you can either download the data manually or write a code that can do that. However, if using free data please be careful about bad data.

What is the best backtesting software?

The best backtesting software is Amibroker or Tradestation. We recommend using a stand-alone platform, and not Python or Excel/spreadsheet. Most trading platforms have a strategy tester or analysis window where you can backtest your strategy, but not all of them are free to use.

Some of the free backtesting software are Microsoft Excel, TradingView, NinjaTrader, Trade Station, Trade Brains, etc.

How to backtest a strategy in Python?

To backtest a strategy in Python you need to follow the steps below:

1. Import the necessary libraries for backtesting
2. Download the needed market data
3. Calculate daily returns
4. Create strategy-based data columns
5. Create strategy indicators
6. Create signals and positions
7. Implement the backtesting
8. Analyze results

Python is a free open-source and cross-platform language with a rich library for almost every imaginable task, including automated trading. Thus, Python is an excellent choice for backtesting a trading strategy.

We have written about Python trading strategy in a previous article.

How do you backtest strategies without coding?

To backtest a trading strategy without coding, you have to use a code-free trading software, like Excel or spreadsheet, for example. Such a platform allows you to create codes with a simple drag-and-drop interface.

Examples of code-free platforms include TradeStation, Amibroker, MetaTrader 5, TradingView, QuantShare, and Forex Tester.

What is curve fitting?

Curve fitting is a statistical technique used to find the best-fitting mathematical function that describes the relationship between variables in a dataset. The enemy of a backtest is curve fitting.

Let’s start by defining what curve fitting is:

Curve fitting is when a strategy or edge is not fit to market behavior, but market noise, leading to failure in live trading. Curve fitting is overoptimization that is unlikely to fit into future unknown data.

We might argue curve fitting is inherent in all kinds of backtesting and impossible to avoid. The aim should be to minimize it.

How can you minimize curve fitting?

You minimize curve fitting by using as few parameters as possible, you check your strategy for robustness, and you use out sof sample and walk forward (see next section). But above all, you reduce curve fitting by being experienced. The more you backtest, the easier it is to spot curve fitting.

We have explained the curve-fitting trap in a separate article.

What is out-of-sample backtesting?

Out-of-sample backtesting is a method used to evaluate the performance of a financial or trading strategy using data that was not part of the original dataset used to develop the strategy. It helps assess how well the strategy might perform in the real world by testing it on unseen data to check for robustness and prevent overfitting.

One of the most important things in backtesting is to use your trading rules on unknown data. Why? Because one of the most common traps when backtesting is to curve fit. The more variables you use to get a trading strategy, the more likely you are fitting the rules to the data.

To avoid this, you need to backtest on unknown or future data. For example, if you have data from the year 2000 until today, you can make rules based on the data until 2017 and then test the trading rules on the data from 2017. If the strategy performs well on the unknown data, you might have something going.

We have explained out-of-sample backtesting in a separate article.

What is walk-forward testing?

Walk-forward testing is a method used in financial modeling and time series analysis to evaluate the performance of a trading or forecasting strategy. It involves updating the strategy regularly, typically on a rolling basis, to simulate real-world conditions and assess its effectiveness over time. This helps in identifying any changes in strategy performance and ensures that it remains robust and adaptable in evolving market conditions.

Another way of testing your backtested strategy on unknown data is a method called walk-forward. It’s a kind of optimization where you “walk” forward in your dataset, hence the name. We have explained Walk-forward here.

What are the most common trading performance and metrics?

The most common trading performance and metrics are the Sharpe Ratio, Ulcer Index, Treynor Ratio, max drawdown, Sortino Ratio, Risk-adjusted return, and Profit Factor. There are plenty to choose from!

There are many ways to evaluate a backtest. Just looking at the end result, the CAGR or the annual returns, might be very misleading. Let’s look at the most common metrics:

1. Drawdown: Drawdown measures the decline in the value of a trading account from its peak to its lowest point. It helps traders understand the potential losses they might incur during periods of market downturns.
2. Sharpe Ratio: The Sharpe ratio is the most widely used performance metric in backtesting and trading. It measures the risk-adjusted return of a trading strategy by taking into account the volatility of returns. A higher Sharpe ratio indicates a better risk-adjusted return.
3. Profit Factor: The profit factor measures the ratio of the total profit to the total loss generated by a trading strategy. A profit factor greater than 1 indicates a profitable strategy.
4. Sortino Ratio: Similar to the Sharpe ratio, the Sortino ratio focuses on downside risk by considering only the standard deviation of negative returns, providing a more specific view of risk-adjusted returns.
5. Ulcer Index: The Ulcer Index quantifies the depth and duration of drawdowns.
6. Treynor Ratio: Similar to the Sharpe ratio, the Treynor ratio considers the risk-free rate in the denominator, making it useful for assessing the risk-adjusted performance of portfolios with different risk profiles.

When you have completed a strategy backtest, you’d like to know if it’s good or bad. How robust is the strategy? Is it likely to be a result of chance? This is what strategy performance and metrics are all about.

If you’re a short-term trader, we are pretty confident in saying that most traders would abandon a strategy if the drawdowns are too big no matter how high the returns. This is why you need to look at strategy and system performance metrics.

What is survivorship bias in backtesting?

Survivorship bias in backtesting happens when you are considering only the data of successful stocks or strategies, leading to an overestimation of performance. We can call it hindsight bias.

Survivorship bias in trading and backtesting is about the things we don’t see or to a certain degree ignore. We tend to see the winners and not the losers. Unfortunately, this is very typical in trading and backtesting. To avoid this, you need to understand what survivorship bias in trading is.

We like success stories about super-traders/investors or entrepreneurs, and we “forget” to look at the probabilities for success and look at all those who fail. Decision-making is not only about tangible results but just as much about what Nassim Taleb calls alternative histories. The same goes for economics: it’s just as much about a study of what we don’t see, opportunity costs, and unintended consequences.

Let’s give you an example of survivorship bias in backtesting:

Let’s assume you want to backtest a momentum trading strategy among the S&P 500 stocks. You download all of the tickers from today’s components, and find out that the strategy has worked very well.

But you are wrong!

You have not included all the stocks that have been taken out from the index. Unfortunately, the ones taken out are more likely to be poor performers – not star performers. Thus, you are overestimating the results. Our backtests show that this is a significant problem.

The importance of good datasets

Good datasets are the cornerstone of backtesting. What’s the point of backtesting bad or erroneous data? Obviously, you’ll make conclusions on the wrong assumptions.

Good data is important in trading. After over twenty years of day trading, we have experienced the expensive way the importance of good data. The famous saying “garbage in, garbage out” is indeed true.

Your backtest is only as good as the data you are testing on. Make sure you are backtesting on reliable and “clean” data. In the long run, it pays off to spend money on a good data source for backtesting.

We have probably lost tens of thousands of dollars on trading strategies that are based on “garbage”. Sad but true.

Yahoo Finance is often wrong

The problem is, there is not much you can do about it. Or is it? By writing this blog, we’ve been contacted by several people. Yesterday, one trader sent me his own data on SPY, which he had downloaded from Interactive Brokers (IB). We’ll test this dataset to see the differences between that and EOD data from Yahoo!finance. No doubt the dataset downloaded from IB is better than what you get from many providers.

First, we’ll show you two errors in SPY that we still can remember to this very day (in Yahoo!Finance):

This example is from the 30th of November 2011.

It’s correct that it was a big gap-up opening, but the low is completely wrong. Even in a paid data feed such as IQFeed.net this low price is included (on EOD data, not intraday data). In many strategies, if you rely on the low of the day to set profit targets, this will turn out to be a huge winner. But in reality, this low trade never happened. The fact is that this day had a low that was only some 20 cents lower than the open! Not 2 dollars as shown here.

Here is the second example:

This one is from the 9th of April 2012. It shows the gap is filled, but it’s fake. The high of the day was 75 cents higher than the open, not close to 2 dollars as shown in the chart! Fading the gap this turns out to be a fake huge winner.

Worth noting is that the CLOSE is basically 100% right. OPEN is also reasonably correct. It is the HIGH and LOW prices of the day which are sometimes (very) wrong.

A comparison between Yahoo!finance vs Interactive Brokers

Below is a comparison of the quotes comparing the manually downloaded quotes from IB and the EOD quotes from Yahoo!finance. It shows the percentage difference between the OPEN to HIGH and OPEN to LOW (the OPEN to HIGH from IB is deducted from the OPEN to HIGH from Yahoo!finance).

The first bar shows that Yahoo! Finance has many high quotes that are much higher than IB’s. The second chart shows the same attributes: Yahoo! Finance’s low is much lower than IB’s.

The question is: are these differences so brutal that it makes a theoretically good strategy useless?

We have previously written about opening gaps in SPY. And yes, the results are a lot worse. We tested all three options: EOD from Yahoo/Finance, intraday data collected from IB, and intraday data from IQFeed.net. All three yield significantly different numbers! When using EOD data from IQFeed, we get the same result as in Yahoo! Finance.

So the conclusion must be: if you’re testing on only the CLOSE and OPEN data, you’re (mostly) on solid ground no matter your data provider.

You must be careful if you’re using the HIGH and LOW on EOD. Always test the strategies by paper trading: Just on the quotes you actually see, or trade as small as you can for a period.

Should you adjust for dividends in backtesting?

You should always adjust for dividends when you are backtesting. For example, if you ignore dividends, long positions are underestimating returns and performance, and the opposite is for shorts. As a short seller, you are borrowing shares, and thus you need to pay the owner the the dividend.

This might have a huge impact on the results, and especially so for short positions.

Why include dividends in backtests?

Dividends must be included because omission leads to understatement of long strategies and an underestimation of any short strategy. When you backtest, a lot can go wrong, so you better make sure you are accurate with the things that you can control.

One of the things you control is to make sure dividend payments are included in the backtest. 

Dividends matter for both long and short strategies:

• Long positions: if you exclude dividends you are most likely underestimating the returns and performance; while
• Short positions: you are overestimating the returns and performance because a short seller needs to pay the dividend to the real owner of the shares. 

Thus, you must include dividends.

To show you why it matters, we make a backtest of a trading strategy as a practical example.

We backtest the same strategy but with two different datasets:

• S&P 500 cash index; and
• SPY, the ETF that tracks S&P 500 but has dividends (and reinvestment) included in the dataset.

You must keep in mind that it’s the dividends that are reinvested that matter – not the dividend payment.

We might argue dividends are included in the cash index, but they are not reinvested.

In the dataset we have for SPY the dividends paid out from SPY is reinvested. Because the dividend payments are reinvested, the returns get better because the number of shares snowballs over time.

For the backtest, we used the following trading rules shown below. This is just an example to show the difference between reinvested dividends vs. not, so don’t focus much in the strategy.

• When the 2-day RSI drops below 10 we buy; and
• We sell when the 2-day RSI rises above 60. 

For the cash index of the S&P 500, we get an average of 0.49% [of trade from 1993 until today. The corresponding average for SPY is 0.56%.

For many, this might not sound like a lot of difference. But over time, it really does matter.

To show you why dividends matter, please have a look at the two graphs below. The first one is for the cash index of S&P 500:

From 1993 until today your equity grows from 100 000 to 554 000. 

But if you had reinvested the dividends, you would have compounded better and increased your equity to 668 000:

That’s a hell of a difference, if you ask us. Thus, you are underestimating the potential returns if you backtest the cash index without reinvested dividends.

Some argue it’s not relevant because many traders and investors don’t reinvest the dividends, but we disagree. 

Keep in mind that this is reversed when you are backtesting a short strategy. Datasets without dividend adjustment show much better returns than you’ll get in real life. The reason is simple: short sellers are required to pay the dividends to the owner they borrow shares from.

How do you adjust for dividends when backtesting?

You adjust for dividends when you’re backtesting by dividing the adjusted price by the unadjusted price. It’s better explained with an example:

SPY went ex-dividend on the 14th of December:

Date	Open	High	Low	Close	Adj. Close	Volume		Factor
2023-12-14	472.5	473.730011	469.25	472.01001	470.104004	119026000		0.9960
2023-12-15	469.48999	470.700012	467.429993	469.329987	469.329987	141319300		1.0000

The adjusted close is the close minus the dividend.

You need to find the division factor, which you do by dividing 4701.104 by 471.01. The result is 0.996, which is the “factor.” Then you multiply the factor number with open, high, low, and close to find the adjusted data:

Date	Open	High	Low	Close	Volume
2023-12-14	470.592015389674	471.817059528174	467.355139093343	470.104004	119026000
2023-12-15	469.48999	470.700012	467.429993	469.329987	141319300

However, most data providers provide you with both adjusted and unadjusted data.

What are data gaps in backtesting?

Data gaps in backtesting occur when historical data is missing that is used to simulate the performance of a trading strategy. This can happen for a variety of reasons, such as exchange closures, data provider errors, or simply because the data is not available for a particular period of time. It happens frequently for illiquid stocks (but that is not a problem) because they might not trade a specific day.

But are data gaps a problem in backtesting?

Yes, data gaps and “holes” in the datasets can be a major problem in your backtest because they can skew the results – for better or worse. Let’s give you one obvious example: if a trading strategy has a drawdown during a period of missing data, the backtest does not account for this, and thus, you overestimate the backtest and strategy.

How to handle data gaps efficiently in backtesting?

You handle gaps in the dataset by either ignoring the gap, replacing it with estimated values, or you choose another provider. There is no clear answer to what is the best way to handle missing data. It depends on the specific situation. If it’s just a few missing observations over 20 years, a few of them missing might not be a big deal (but obviously you don’t know so it adds uncertainty). This is why you should always use a reliable data and quote provider!

What is the best way to avoid data gaps in backtesting?

The best way to avoid data gaps is to use a reliable data provider, like Norgate, for example. This is the best method by far. Two other methods are to identify any hols BEFORE you backtest, and you can compare two datasets against each other. With the rise in AI, the possibility of using machine learning is also an option for the more advanced (and professional).

What are the best historical data providers for backtesting?

The best data provider for backtesting is Norgate, but IQFeed and free data from Yahoo! can also be used. Below is a list of what we consider good data providers. We can add that we have used all of them ourselves, thus we have hands-on experience:

Data Provider	What They Offer
Norgate	Historical US (and some foreign) stock, futures, and options end-of-day data (no intraday or real-time data)
IQFeed	Real-time stock, futures, forex, and options data
eSignal	Real-time stock, futures, forex, and options data
Yahoo!finance	Tons of free data, and the pretty good data as well

If you are trading liquid stocks and ETFs, you get a long way by using free data from Yahoo!.

We have made a complete list of historical data sources for backtesting.

What is backtesting bias?

Backtesting bias refers to potential flaws and errors in your backtest that might not represent true results when you start trading your strategy live. It occurs due to multiple reasons, the most obvious being curve fitting, slippage, commissions, curve fitting, survivorship bias, erroneous data, look-ahead bias, etc. Most of these biases are covered in this article under separate headings.

Unfortunately, backtesting is not without its flaws, but it’s mainly based on the trading rules and the data put in. Backtesting frequently differs between simulated results and live trading. Backtesting is influenced by many factors such as mathematics, statistics, psychology, and more, making it susceptible to biases that can distort the outcomes heavily and render a backtest more or less useless.

To avoid backtesting bias, traders must develop their strategies and test them in good faith, avoiding bias as much as possible. They must be strict about testing with different data sets from those they train their models on.

Let’s look at factors that lead to backtesting bias:

Optimization Bias

Optimization bias, akin to Murphy’s Law, suggests that if something can go wrong, it will. This bias, also known as data snooping bias and curve fitting, arises when an algorithm is overloaded with numerous parameters, fine-tuned according to available data. Consequently, such an approach tests the algorithm solely on past events, and is unlikely to predict the future well.

To avoid optimization bias, keep the simulation system as straightforward as possible – making it simple. Reduce the number of parameters. The fewer rules you have in the strategy, the more likely it is to hold up well in the future. Please read simple vs complex trading strategies.

Furthermore, after completing backtesting, it’s recommended to subject the algorithm to new, unfamiliar data to validate its authenticity and effectiveness. This can be out of sample backtesting or walk forward.

Look-ahead Bias: The Perils of Foresight

Look-ahead bias is the temptation to use future or hypothetical information in a backtest when you have access to the entire dataset. This is much easier than you imagine, and we have done this ourselves many times. It’s not only the trading rules, but it could also be a result of inadvertently setting the wrong settings in the software.

Conducting backtests on the same dataset increases the likelihood of unintentionally introducing a look-ahead bias into the system. This bias can manifest in subtle technical glitches or significant deviations in maximal and minimal values, ultimately impacting live trading results.

To counter look-ahead bias, it’s imperative to ensure that both live trading and backtesting employ the same algorithm or code. Doing so eliminates the risk of the program inadvertently peering into the future, thereby preventing look-ahead bias. We always recommend using a demo account for months before you do live trading. This is an extremely good filter for weeding out bad strategies.

Survivorship Bias: The Neglected Perspective

Survivorship bias often goes unnoticed by coders and data scientists. Backtesting with a current stock database exclusively considers stocks that are currently active, omitting those that have been delisted. This phenomenon is aptly labeled as survivorship bias and can distort a trading strategy massively.

Consider a strategy aimed at outperforming the S&P 500. If you backtest exclusively with stocks currently constituting the index, your results are highly likely tainted by survivorship bias. To mitigate this bias, consider databases that include delisted stocks or opt for more recent data when conducting your backtests. Norgate has databases that include delisted stocks.

Neglecting Market Impacts: The Pricing Oversight

Backtesting data history does not account for the actual execution of trades, potentially leading to an oversight of market impact. Since trading and pricing are interlinked, disregarding market impact can introduce bias into backtesting results.

The most likely cause is slippage. If you are trading illiquid stocks, slippage is likely to distort a lot. Opposite, if you are trading super liquid stocks like SPY or Apple, slippage is unlikely to be much of a problem.

How much is commission and slippage in live trading? The link provides data and statistics for our trading, and for SPY and QQQ the trading frictions are close to zero.

To rectify this oversight, always assume that when you trade, prices will move against you. This conservative approach eliminates bias, providing you with more accurate results and a clearer understanding of market impacts. As a rule of thumb, ALWAYS assume a backtest to perform worse in live trading.

Any backtest should be rigorously tested and be met with skepticism. Instead of viewing backtesting as a validation of the strategy, consider backtesting as a filtration process for eliminating flawed strategies. Every backtest is in one way or another somewhat curve fitted based on the past. By adhering to this strict methodology, you can achieve more precise and unbiased trading strategies.

Other factors leading to backtesting bias

Data-Snooping Bias: This bias occurs when data is analyzed multiple times, leading to the possibility of finding patterns that are not statistically significant. It can result in overfitting models to noise in the data.

Time-Period Bias: This bias arises when the selected time period for analysis influences the results, leading to conclusions that may not hold true in other time periods.

Selection Bias: This occurs when certain data points are systematically excluded or included in a dataset, skewing the analysis and potentially leading to incorrect conclusions.

Model Specification Bias: This bias arises when the chosen model does not accurately reflect the underlying data-generating process, leading to inaccurate predictions or estimations.

Out-of-Sample Testing: This involves testing a model’s performance on data that it has not been trained on, providing a more accurate assessment of its predictive power and generalizability.

In-Sample Testing: This involves testing a model’s performance on the same data it was trained on, which may lead to overly optimistic assessments of its predictive ability and potential overfitting.

Monte Carlo Simulation: This method uses random sampling to analyze the behavior of complex systems or models, generating a large number of possible outcomes to assess the likelihood of different scenarios.

Parameter Sensitivity Analysis: This involves assessing how changes in model parameters affect its output, helping to identify which parameters have the most significant impact on results.

Cross-Validation: This technique involves splitting data into multiple subsets, training the model on some subsets, and then testing it on the remaining subset. It helps assess a model’s performance and generalization ability.

How can you avoid backtesting bias?

You avoid most of the backtesting bias mostly by experience. When you are inexperienced, it’s extremely hard to avoid some or all of these biases. Thus, you should keep plugging along, and you’ll learn by trial, error, and experience.

In addition to gaining experience, employing rigorous methodology is essential for avoiding backtesting bias. This involves meticulously designing backtests with clear hypotheses, selecting appropriate historical data, and implementing robust validation techniques. Incorporating out-of-sample testing and sensitivity analysis can help to mitigate the risk of overfitting and ensure the robustness of trading strategies.

Furthermore, maintaining discipline and objectivity throughout the backtesting process, while remaining aware of cognitive biases, can help to prevent subjective interpretation of results and foster a more reliable evaluation of trading strategies.

Should a trading strategy work on all assets?

No, a trading strategy should not work on all assets, and this makes perfect sense. Why should, for example, a 3-day RSI work on both SPY and XLE?

We frequently see discussions on, for example, Twitter, where one trader trades different strategies for different instruments/stocks, while another trader believes this is data mining. For example, Buying when RSI(3) is oversold might work on different stocks and not on others.

If we trade this strategy on the stocks it works on and not on the stocks it does not work on, is this data mining and curve fitting? Or should we only trade strategies that work more or less on all S&P stocks?

We believe the first is correct. Why should one strategy work on both SPY and XLE? These are completely different stocks from different sectors. There is no logic to implement the same strategies in different sectors. We have tested thousands of strategies and all sectors behave differently. Even within sectors, you have to regroup the stocks.

How to pick stocks for each strategy?

We pick stocks/assets for our strategies based on how they perform using this strategy. If a stock does not show an upward-sloping equity curve, we simply reject it from that asset.

Is this data mining or curve fitting? We don’t think so. This is the way we have done it over the last 20 years, and it works well for us.

Have we just been lucky? We don’t think so. Our trading is “out of sample” testing, and our group of stocks performs better than any random group. We have tested this a lot of times. The key to successful trading is to look for patterns/attributes in stocks and trade what you find.

Of course, it’s only logical that stocks have different patterns (at least to us). Sometimes you just happen to find a random pattern, so there must be some kind of logic behind why this pattern should exist.

Trade many uncorrelated strategies

We believe many will reject our methodology. That’s fine. We read a lot of blogs and see a whole lot of different theories. Usually, the blog post ends like “this is not recommended as a stand-alone strategy”.

We have picked up so many ideas from blogs and traded them, and still, the author thinks his strategy is not worthwhile as a tradeable strategy. The best thing is to trade a lot and trade with different strategies. This is the holy grail of trading! Even some erratic equity curves contribute when having about 20 uncorrelated strategies. A backtest might not be tradeable on its own, but together with other strategies it might be complementary.

Trading and backtesting should be a little boring

At the end some words on backtesting, boredom and introverted traders:

We believe backtesting and trading should be a little boring. Perhaps contradictory, but the more boring, the better you can do.

Why? Trading only suits people with the following attitudes: patience, analytical, and somewhat risk-averse.

Yes, you have to take risks in trading, but it needs to be balanced. All the traders we know that have lasted more than ten years have been somewhat introverted and very rational/analytical and not risk-takers. They backtest and base their decisions on that and include a margin of safety.

They don’t trade to get a rush, but to make money. They are happy to backtest strategies, but not keen on execution. When there is little prey in the markets, they simply wait out for better times. Of course, they have made money before and have the financial means to wait for many months.

Does backtesting work?

Backtesting works because it is the best way to approach trading and investing for most people. Let’s look at some arguments for why backtesting works:

Backtesting lets you confirm or falsify a trading idea

Backtesting works because you can easily check if something has worked in the past or not.

• Is, for example, the Turnaround Tuesday strategy in stocks true or just a myth? Just define the rules and start the backtest. You’ll find out in five minutes.
• Do you believe you see an interesting pattern in the chart? Then quantify it with strict buy and sell rules and test it.

Did the strategy work in the past? If something has not worked in the past, you can easily falsify your hypothesis and go on to test another idea.

Because most ideas don’t work, you should not spend much time testing a hypothesis. Many traders waste months, even years, in both programming software and tweaking their strategies only to find out it was a waste of time. You don’t need “perfect” strategies to make money in the markets. You need many strategies that complement each other.

• What is the Holy Grail in trading?
• Why build a portfolio of quantified strategies

We have written multiple times on this website that one of the main reasons for your success (or no success) depends on your ability to test and generate trading ideas. One of our main trading lessons is that we spend about 80% of our time testing ideas back and forth between ourselves.

Backtesting works because it lets you automate

If you have successfully backtested a strategy, you can easily go “live” with the strategy. In Tradestation, you just check a box and you are good to go.

In Amibroker, you need to add code to automate and let Amibroker keep track of your positions and strategies. We have even made a course that lets you automate your Amibroker strategies to Interactive Brokers:

• How to trade live with Amibroker (Automated trading Interactive Brokers)

Obviously, this saves you a lot of time. There is no need to check quotes, prices, or follow the markets. The computer does all the work!

Backtesting exploits the law of large numbers

Your computer can easily trade and supervise hundreds of strategies. This lets you exploit the law of large numbers and you can diversify into time frames, asset classes, directions, and types of strategies.

The main reason for the Medallion Fund’s success is twofold: They use enormous amounts of data to generate hundreds of uncorrelated strategies. Because of the low internal correlation among the strategies, they can use leverage to boost returns.

Low correlation among your trading strategies is one of the most important factors in trading. Why? Because if you lose money in one strategy, you might gain in another uncorrelated strategy. This is a topic we have covered in many articles:

• What does correlation mean in trading? (Trading strategies and correlations)
• Uncorrelated assets and strategies – benefits and advantages (examples and backtests)
• Does your trading strategy complement your portfolio of strategies?

However, leverage is dangerous and certainly not something we recommend. Only use leverage if you have many years of experience.

Backtesting reduces or removes emotions

Ample evidence points out that individual investors underperform the averages, and women are better investors than men. The main reason for this is behavioral mistakes:

• How to overcome trading biases
• The most common trading biases

Investors tend to sell into a panic and buy after a big rise. Most of the time you need to do the complete opposite. A backtest can’t capture such mistakes and that’s why you need to stick to the trading plan.

In order to stick to the trading plan, you need to trade smaller than you’d like or prefer. This is the best way to keep detachment to money.

Likewise, women do better because they save, invest, and forget about it. They are not trying to be geniuses! They don’t have any ego.

The closer you follow the markets, the more likely you are to overrule your systems when your “intuition” tells you to sell or buy. But most of the time the intuition is plain wrong, unfortunately.

Overruling your systems and strategies is unlikely to work. You have not backtested overruling, so how do you know if it works? That’s why you backtest trading strategy.

Backtesting saves you time

Today we tested about 15 hypotheses in Silvers Miners (ticker code SIL) in about 1.5 hours. One of those ideas seems promising, the rest are probably just a waste of time. You can generate and test hundreds of strategies in just a single day. Even better, you can falsify or confirm ideas quickly.

Trading is mainly about trial and error. And luckily, backtesting a trading strategy is a great tool for that and at the same time, it saves you a lot of time.

Disadvantages of backtesting

Of course, there are also drawbacks, disadvantages, and negatives with backtesting. You rarely manage to find trading strategies that perform better in live trading than in tests. Why is it so? This is because of the disadvantages of backtesting. You need experience in testing to avoid the many pitfalls along the way.

Let’s look at the disadvantages of backtesting. There are many reasons why backtesting doesn’t work, like curve fitting, market cycles, chance, luck, and randomness. But the good thing is that you can avoid or at least minimize many of the disadvantages of testing. We are proponents of quantifying trading strategies, but you need to understand how to avoid the many pitfalls in testing. We explain why:

Live trading will never be as good as the backtest

As a rule of thumb, you can expect your trading strategy to perform worse in live trading than in the backtest. Real results will never be as good as theoretical backtesting. It’s almost 100% certain that the real-time results will be worse than the actual backtesting. What Can You Expect from a Trading Strategy Backtest When You Are Trading It Live?

=Why? Below, you’ll find some personal thoughts on why the results differ so much.

The backtest might be liable to favorable conditions

The backtest is performed in a certain period, and the markets may have been favorable to that trading strategy in that period. Most traders neglect this aspect. Testing over a longer time frame might minimize this.

Take, for example, so-called trend-following strategies. They perform quite badly in certain periods covering many years and much better in others.

If you are testing a moving average breakout, it might yield mediocre results over a five-year period. However, this is a typical strategy that needs to be tested over at least 10 years.

Obviously, you don’t want to change a strategy in response to one year just because something didn’t work. That’s when you’re almost guaranteed it would have worked the next year had you kept it as it was.

The ever-changing market cycles make trading a difficult task to follow. You have to accept drawdowns to make money, and every strategy has drawdowns.

• Trend following strategies and systems explained
• How to deal with drawdowns in trading

Backtests are after the fact – hindsight bias

You can only trade the strategy after the fact.

For example, you are using entry at the close. The problem is, you only know if it’s a trade after the close. To trade on the close, you might have to guess/estimate there wil be a trade at the close.

So, when backtesting, it’s crucial to consider this factor. One way to do this is to trade at the open the day after the signal.

Backtesting involves elements of curve fitting

Another reason is the curve-fitting aspect. This certainly applies if you have a lot of parameters or variables.

It’s easy to create a system that has performed remarkably well. You just need to input a lot of parameters. That will explain the past, but most likely not the future.

The simpler the system, the more likely it’s to stand the test of time.

Curtis Faith explains in The Way of The Turtle some trend-following strategies that are incredibly simple. Over several decades, they have worked well in currencies and commodities (not on stocks).

However, over a period of 1-3 years, they sometimes experience quite huge drawdowns. Still, these systems are so simple that they are less prone to be curve-fitted.

Backtesting involves survivorship bias

Another problem in backtesting is survivorship bias. This relates to the use of stocks/tickers that have “survived” the testing period.

For example, in 2008/2009, many stocks went bust (Lehman being an example). This means that companies that have gone bankrupt are excluded from the analysis at later dates.

In day trading, this might be less of an issue, but not when testing over a much longer time frame. If you download quotes for REITs back to 2005, this will exclude several stocks that went bust during the financial crisis.

Also, if you backtest indices, you are unlikely to be a victim of survivorship bias.

Chance, luck, and randomness overestimate backtesting results

If you’re testing many strategies, some will show good returns simply by chance. Unless there is a logical reason for a strategy, you are guaranteed to find many good strategies the more you test. Hence, we recommend there should be some reasoning behind the parameters.

Backtesting involves garbage in, garbage out

The quotes and data you buy or download are usually not 100% correct. If you use high and low in your research, you can be sure there are errors compared to live trading. There are a lot of wrong quotes on high and low!

In real trading, this will have a huge impact, probably most of all the factors mentioned in this article.

Transaction costs in backtesting are unknown

Slippage and commissions are easy to underestimate. Commissions are easy to calculate, but slippage is not. If you are a serious trader, you should keep records so you can calculate total costs for commissions and slippage.

This is a huge unknown, especially if you base your strategies on chasing the stock. If you wait to get hit, this is, of course, less of an issue.

Markets change – no backtest can accommodate change

The market changes all the time. The future is unpredictable, and you can bet there will be totally random and dramatic changes in the marketplace. No one expected terrorists to hijack planes and send them into a skyscraper. No backtest can capture black swan events.

Such totally unpredictable disasters will happen sooner or later. Correlations among different asset classes also increase during such happenings. You can never backtest such things.

Backtesting ignores trading biases and behavioral mistakes

You need to understand the most obvious trading biases. The psychological aspect is just as important as the strategy. Can you handle drawdowns and continue trading? Can you actually follow the strategy?

Based on my personal experience, this is something you have to consider thoroughly before you implement a strategy. It’s a lot easier said than done to follow a strategy 100%.

Backtesting should always prepare for the worst

As a rule of thumb, it might be wise to expect a maximum of 50% of the profits from testing. You can exaggerate slippage and commissions and expect a much higher drawdown than in the backtest.

Hope for the best, but prepare for the worst. Never be too optimistic when seeing a very nice equity curve; the downfall will be bigger. Only real trading matters.

Disadvantages of backtesting – conclusion

To get good results from backtesting, you need to understand the potential pitfalls of testing. A backtest can’t capture that markets change, are random, that you tend to curve fit, and you make behavioral mistakes. However, if you are street smart, you can even offset some of the disadvantages of backtesting.

A piece of advice on backtesting

We’ll finish the article with a piece of very good advice (at least it works for us):

It might be very boring and tedious to test strategies manually, but believe me, you can learn a lot more. A program makes it easy to use and scan thousands of stocks, but you will miss details.

However, by doing testing by hand, you can extract info that would otherwise get lost. It might be a little boring doing it the old-fashioned way, but consider it as an investment.

For example, by testing one stock at a time by pasting quotes into Excel, instead of scanning 1000 stocks in one minute using a fancy program, you can look at each different stock one by one, examining its equity curve, max drawdown, etc.

If you use scanning, spend some more time doing it manually stock by stock.

Does backtesting work? Conclusion

Backtesting works because it’s the closest simulation you get to real trading. But backtesting only works if you can manage and understand how to backtest valid and logical ideas. Second, you need to understand the disadvantages of backtesting mentioned in this article.

We have probably tested close to 5,000 ideas. So far, no strategy has performed better in real life than in backtesting, so consider this a fact of life.

When executing a tested strategy live, you discover a lot of factors you didn’t think of when testing. Hence, always start with the minimum amount of money and always papertrade a strategy before you go live. Always trade a smaller size than you’d like to.

What is the history of backtesting trading strategies?

The history of backtesting trading strategies is relatively new: It was only in the 1960s and 1970s that systematic traders obtained the computer power needed to backtest.

Let’s look at the history of backtesting trading strategies in chronological order:

1. Pre-Computers Era: Before the advent of computers, backtesting was rudimentary and manual. Traders relied on historical price charts and paper trading. Let’s call it the pen-and-paper period of backtesting.
2. Spreadsheet Era: The introduction of spreadsheets in the late 1970s and 1980s revolutionized data analysis. Traders began using these tools for simple backtesting. We used spreadsheets ourselves up until 2018 when we switched to Amibroker and Tradestation.
3. Technology improved in the 1980s and 90s:
4. Rise of Personal Computers: In the 1980s and 1990s, the widespread use of personal computers made backtesting more accessible. Software for technical analysis started including basic backtesting features. Metastock was one of the first trading platforms. 
5. Internet and Data Access: The internet era in the late 1990s provided traders with easier access to historical data and enhanced computational capabilities. This was a revolution for trading!
6. Sophistication and Institutional Use
7. However, it was not until the 2000s that we witnessed a rise in quant trading.
8. Quantitative Analysis: The 2000s saw a surge in quantitative trading. Hedge funds and institutional traders developed complex algorithms and models for backtesting. This was also when Jim Simons and his Medallion fund managed spectacular returns. 
9. Risk Models and Optimization: The focus shifted not just to returns but also to risk management and optimization.
10. Recent Developments
11. Machine Learning and AI: The integration of machine learning and AI in the 2010s has brought about sophisticated predictive models and more dynamic backtesting frameworks.
12. High-Frequency Trading (HFT): HFT strategies require backtesting at incredibly high speeds and over vast datasets.
13. Cloud Computing and Big Data: These technologies have enabled more extensive and efficient backtesting processes, handling larger datasets and more complex strategies.
14. Current Trends
15. Accessibility to Retail Traders: User-friendly platforms now offer backtesting tools to retail traders, democratizing the process.
16. Regulatory and Ethical Considerations: With the growing complexity, there’s an increased focus on the ethical and regulatory aspects of backtesting.
17. Challenges and Criticisms
18. Overfitting: There’s a risk of developing strategies that are overly optimized for historical data but perform poorly in real-world conditions.
19. Data Quality and Integrity: The accuracy of backtesting is highly dependent on the quality of the historical data used.
20. This outline provides a broad view of the evolution of backtesting in trading strategies. The field continues to evolve with technological advancements and changing market dynamics.

Facts and Frequently Asked Questions about backtesting

Let’s end this long article with some FAQ and facts about backtesting:

Why is backtesting important?

Backtesting is important because it helps you determine facts – the opposite of anecdotal evidence (which financial markets are full of). By analyzing trading statistics and historical performance, you can falsify or confirm your trading idea. You can also determine the historical probability of a trade’s success or failure.

Why would you trade your money if you really have no idea if the strategy has worked in the past? Our experience is that most traders don’t have any positive statistical edge in the first place, thus making most of the focus on psychology and money management a wasteful exercise.

Trading is very much a game of statistics and probabilities, and thus you need to know the odds of your strategy. When you have a tradable strategy, you just need to push buttons and execute.

Furthermore, backtesting helps you to get other and perhaps better trading ideas. Trading is about trial and error! Here’s how to find and generate trading ideas.

Any strategy that has not worked in the past, is unlikely to yield many positive results in the future. Opposite, a strategy that has shown promise in the past, might be an inefficiency that will work in the future, and you might even be able to optimize and improve the strategy without being liable to curve fitting.

How long should you backtest a trading system?

A trading system should be backtested over many years, but there are no hard rules. The most important criteria is that you should have many observations.

We use the following rules of thumb:

• At least 100 trades (with a few exceptions)
• Must include different business cycles (at least a bear market)

These two are the only two rules we use. However, we like to test as far back in time as possible.

Any end-of-day trading strategy is normally backtested decades back, while we use about ten years for intraday data (for day trading).

How to backtest day trading strategies?

There is no difference between backtesting on daily bars or intraday bars (5 min bars, for example).

However, backtesting on intraday historical data is more demanding than using end-of-day data. There are two reasons for that:

• It’s technically more different (both code and rules). Some rules are actually very difficult to quantify.
• Intraday data are less likely to work well in the future. There is more randomness involved in the time series data.

Is backtesting a waste of time?

Backtesting is not a waste of time!

Most tests end up worthless, but at least you are improving your backtesting, and you get to see what is not working, which is also a great asset to have. Trading is about compounding knowledge!

What is the best backtesting software?

The best backtesting software is Amibroker and Tradestation. However, there is no answer that fits all traders. It depends on many factors. What works for you, is not necessarily the best choice for other traders.

For example, we use Amibroker. It’s a fantastic program for backtesting (fast and cheap), but it’s a bit cumbersome if you want to trade your tested strategies live. You need to write a few extra lines of code to both connect to Interactive Brokers (or the broker of your choice) and to keep track of your strategies.

Other quantitative traders might stick to Tradestation, Multicharts, Ninjatrader, etc.

The fact is that trading platforms are kind of a commodity product that is hard to differentiate from.

How can you backtest option trading strategies?

You backtest option strategies like you do with stocks or futures. That said, it’s a bit more complicated due to the many different strikes and expirations. You certainly would need experience from end-of-day backtesting before you venture into testing options.

Backtesting options also involve a higher risk of strategy deterioration due to high costs to slippage and commissions.

How do you backtest forex strategies?

The process and principles of backtesting forex are the same for any backtest, no matter the market. Forex’s main advantage is that data is abundant.

But sadly, we believe forex is mostly a waste of time. It’s a low bar to trade it, but most quantitative forex traders lose and stop trading. We believe it’s a bad idea to start forex trading.

Is backtesting accurate?

Backtesting is not accurate, but it’s the most accurate estimation of the future you can get if you backtest correctly. However, the future is unknown – also with a backtest.

But in our opinion, it’s the best estimation of the future you’ll ever have.

Can you backtest using Excel or another spreadsheet?

Yes, you can backtest using Excel and spreadsheets – they are a very underrated backtesting software. We both did backtesting and live trading (VBA script) for our successful day trading until 2018.

That said, spreadsheets are limited, especially if you want to test portfolios.

Additionally, Excel has limitations regarding trading history analysis and trading performance metrics. With software, you get all this with the click of a button, while in Excel, you need to “code” it.

Strategy backtesting software

If you are serious about trading, we don’t recommend Excel but that you purchase some specific trading software, like Amibroker, for example. It’s cheap, reliable, and lightning-fast.

Is Python good for backtesting?

Python is very good for backtesting, but it’s not a “one-click” software, like, for example, Amibroker and Tradestation. Every backtest has to be done from scratch, but you can use templates to save time. Furthermore, there are plenty of snippets and code available on the internet.

Python is extremely popular among quant traders, which we find a bit odd. However, we are not coders, either. We suspect it’s mainly programmers and coders who like using Python.

But why start from “scratch” when you can use Amibroker, Tradestation, etc.? To us, it doesn’t make sense, even though Python has a ton of free resources. You should make an analysis of trades, not spend time coding. Any strategy test should be made as fast as possible.

How many trades should you backtest?

We believe it’s more important to include different business cycles than to specifically focus on the number of trades, even though the latter should be at least 100 – preferably more.

Can technical analysis be backtested?

Yes, technical analysis can be backtested, but it’s difficult to backtest chart patterns due to the inherent subjectivity.

We have a rule: if it can’t be backtested, don’t trade it. This applies to technical analysis as well. We mention it because it’s very tricky to test technical chart patterns. Most patterns are not backtested, and thus, you should avoid them.

What is the the best platform to backtest trading strategies?

Amibroker and Tradestation are the two best platforms for backtesting.

We use Amibroker because it’s the best platform to backtest trading strategies. But as you might guess, other traders might disagree.

The most used platform among professional and independent traders is probably Tradestation because it is simple to use for both backtesting and live trading.

Thus, the best platform is what you’ll use it for.

Can you backtest crypto strategies?

Crypto and Bitcoin can easily be backtested. But all cryptos have limitations because of their short and erratic history. Because of this, it’s unlikely that the past predicts the future very well.

What is the best website about backtesting?

We are biased, but we believe the blog and website you are reading now are among the best. In addition to this one, we can recommend the Quantocracy website, which contains a nice summary of quant articles.

Is backtesting and forward testing different?

Backtesting and forward testing are the same thing. Backtesting is when you test known data while forward testing is when you test the trading rules made on the known data on future unknown data.

Can you automate the backtesting process?

Yes, with the rise of AI, you can automate backtesting using programming languages like Python. Automated trading platforms and algorithms can be developed to execute the backtest automatically and analyze the results.

Moreover, some programs allow you to look for a trading strategy that is based on a zillion trading rules. For example, you can program the software to look for the best settings for any technical trading indicator. But we believe such software requires a TON of experience due to all the curve fitting that takes place.

Can you backtest trading strategies for free?

You can find web-based software that lets you backtest for free. That is an option if you’d like to find out if it is for you, but it’s not a viable long-term solution. The best option is to spend some time finding the right software and then purchasing it.

What is Walk Forward Backtesting?

A walk-forward optimization is when you use both in-sample and out-of-sample tests frequently.

It’s done like this:

If you have 20 years of data, you might divide the data into ten equal parts (in length). You then have two years of data for each set. The first year is the in-sample backtest, and the second is the out-of-sample test (where you test the trading rules made in year one).

This is repeated ten times, and the final results are evaluated to make the final parameters for the strategy.

It’s a bit cumbersome, but most software platforms offer this ability. We don’t use it ourselves because we have not found it very useful.

What are anchored and non-anchored backtesting?

Anchored and unanchored (non-anchored) are two forms of walk-forward backtesting.

An anchored walk-forward test uses a fixed start date in each simulation, while a non-anchored uses a start date that is moved gradually forward by an amount equal to the length of each out-of-sample period.

Can you backtest trading strategies in Tradingview?

You can backtest trading strategies in TradingView, but it’s far from the best backtesting platform. Tradingview is a very popular platform that has gained many users with the rise of crypto. Unfortunately, it has many drawbacks and limitations, and you can’t connect to other brokers to place trades and do live trading.

What is the best data provider for backtesting?

The best data provider for backtesting is Norgate. Norgate has “clean” and error-free end-of-day data. Any quantified and data-driven strategy needs good data, otherwise, the backtest is useless.

You can also download free data from Yahoo and Google, but beware of incorrect High and Low readings. The open and close are mostly correct, though, at least on the most liquid tickers.

Backtesting Glossary

Let’s end the article with a short glossary about backtesting:

1. Backtesting: The process of testing a trading strategy or model using historical data to assess its performance. It’s data-driven testing.
2. Historical Data: Past price and volume data of financial instruments used for backtesting.
3. Trading Strategy: A set of rules and criteria used to make trading decisions.
4. Algorithmic Trading: Trading strategies that use computer algorithms to execute trades automatically.
5. Simulation: The process of mimicking trading using historical data to evaluate strategy performance.
6. Performance Metrics: Measures like profit, loss, Sharpe ratio, and drawdown used to evaluate strategy performance.
7. Sharpe Ratio: A risk-adjusted performance measure that quantifies the return per unit of risk taken.
8. Drawdown: The peak-to-trough decline in a trading strategy’s equity curve.
9. Equity Curve: A graphical representation of a trading strategy’s cumulative profit or loss over time.
10. Risk Management: Strategies and techniques to control and mitigate trading risks.
11. Position Sizing: Determining the amount of capital to allocate to each trade or position.
12. Benchmark: A reference point used to compare a strategy’s performance, like a market index.
13. Trade Entry: The specific conditions and criteria that trigger buying or selling.
14. Trade Exit: The conditions and criteria for closing a trade or position.
15. Stop-Loss: An order placed to limit potential losses by selling a position at a predetermined price.
16. Take Profit: An order to lock in profits by selling a position at a specified price level.
17. Risk-Reward Ratio: The ratio between the potential loss and potential profit in a trade.
18. Volatility: A measure of the price fluctuation of a financial instrument.
19. Market Order: An order to buy or sell immediately at the current market price.
20. Limit Order: An order to buy or sell at a specific price or better.
21. Slippage: The difference between the expected price and the actual execution price of a trade.
22. Backtest Overfitting: Over-optimizing a strategy to perform well on historical data but not on new data.
23. Out-of-Sample Testing: Testing a strategy on data not used during the initial backtest.
24. Walk-Forward Analysis: A rolling approach to backtesting where the model is periodically re-optimized.
25. Monte Carlo Simulation: A statistical technique that uses random sampling to simulate various market scenarios.
26. CAGR (Compound Annual Growth Rate): The annualized rate at which an investment grows over time.
27. Mean Reversion: A trading strategy based on the belief that asset prices will revert to their historical average.
28. Trend Following: A strategy that seeks to profit from ongoing price trends.
29. Machine Learning Model: A statistical model that can learn and make predictions from data.
30. Data Snooping Bias: The risk of finding false patterns in data due to multiple tests or parameter adjustments.
31. Calmar Ratio: A performance measure that relates the annualized return to the maximum drawdown.
32. Correlation: A statistical measure of the relationship between two or more assets.
33. Benchmark Alpha: The excess return of a strategy over its benchmark.
34. Backtest Period: The historical timeframe used for testing a trading strategy.
35. Quantitative Analyst (Quant): A professional who develops and tests quantitative trading strategies.
36. Trading System: A complete set of rules and processes for executing trades.
37. Portfolio Optimization: The process of selecting the best combination of assets for a portfolio.
38. Slippage Model: A method for estimating the expected slippage in a backtest.
39. Risk-Free Rate: The theoretical return on a risk-free investment like a Treasury bond.
40. Walk-Forward Optimization: A technique that combines optimization and out-of-sample testing for strategy development.
41. Market Impact: The effect of a trade’s execution on the market price of the asset being traded.
42. Order Execution: The process of submitting and completing buy or sell orders in the market.
43. Liquidity: The ease with which an asset can be bought or sold without causing significant price fluctuations.
44. Trading Costs: Expenses incurred during trading, including commissions, slippage, and spreads.
45. Backtest Sensitivity Analysis: Testing a trading strategy’s robustness to changes in input parameters or assumptions.
46. Walk-Forward Efficiency: Assessing how well a strategy adapts to changing market conditions during walk-forward analysis.
47. Risk-adjusted Return: A measure of a strategy’s performance that takes into account both returns and risk.
48. Portfolio Backtesting: Testing the performance of a combination of multiple assets within a portfolio.
49. Leverage: The use of borrowed capital to increase the size of a trading position.
50. Diversification: Spreading investments across different assets to reduce risk.
51. Sliding Stop: An order that adjusts the stop-loss level as a trade moves in a favorable direction.
52. Risk Parity: A portfolio allocation strategy that equalizes risk contribution from each asset.
53. Market Regime: The current state or condition of the financial markets, such as bull or bear market.
54. Monte Carlo Portfolio Simulation: Using Monte Carlo methods to simulate the performance of a portfolio.
55. Systematic Risk: The risk associated with overall market movements that cannot be eliminated through diversification.
56. Non-Systematic Risk: Risk specific to individual assets and can be reduced through diversification.
57. Maximum Adverse Excursion (MAE): The largest loss experienced during a trade before it becomes profitable.
58. Maximum Favorable Excursion (MFE): The largest profit experienced during a trade before it reverses.
59. Algorithmic Trading Platform: A software or service used to execute algorithmic trading strategies.
60. Performance Attribution: The process of analyzing a portfolio’s returns to identify the sources of performance.