Curve fitting is always mentioned negatively in trading. This article takes the contrarian view and explains why curve fitting is good. Curve fitting is good in the sense that not all stocks can fit into a certain trading strategy or style.
If you are new to systematic trading, we recommend you have a look at our complete backtesting guide. This article assumes you have at least a basic knowledge of backtesting.
First, let’s start with explaining…..
What is curve fitting?
Curve fitting is the process of optimizing the parameters of a trading strategy to fit the historical data. This can be done by manually adjusting the parameters or by using an optimization algorithm.
The problem with curve fitting is that it can lead to overfitting. Overfitting is when a strategy performs well on historical data but poorly on new data. This is because the strategy has been optimized to fit the specific noise in the historical data, rather than the true underlying market dynamics.
Curve fitting is likely to happen the more parameters you put into the strategy. The more parameters, the more likely the strategy won’t adapt to future data. As a general rule, a trading strategy is more likely to survive the simper it is. Complex strategies are very unlikely to last for long.
Unfortunately, all backtesting is partially curve-fitting, because history rarely repeats itself.
If you are worried about curve fitting, you might want to either run out of sample tests, run walk-forward analyses, or better, run your strategies live in a demo account for many months as an incubation period. We recommend reading our article called out of sample backtesting.
When curve fitting is good – an example
However, we have been using curve-fitted trading strategies with success for a long time. In this section, we’ll give you an example of how we did curve fitting when we were systematic and automated day traders from 2001 until 2018.
This is what we did for a stock trading strategy (trading multiple stock tickers, for example, the whole universe on NYSE but using filters for size, volatility, volume, etc.):
- Define a trading strategy with specific trading rules.
- Backtest the strategy on a wide range of listed stocks.
- Pick the stocks that have worked well over the last N years, for example, 5 years. Test those stocks going forward.
- Is there a pattern? Are there groups of stocks that have performed badly?
- Exclude sectors/groups performing poorly.
- We excluded all stocks that performed poorly.
- Rinse and repeat quarterly, semi-annually, or annually (we did quarterly).
This might not be a recipe for success for all traders, but it worked for us. And in our opinion, it makes sense. Please continue reading to better understand why.
Is Jim Simons curve fitting?
We are looking for patterns, and we might argue that if you reject 10 patterns but trade one, you are curve-fitting.
Can we argue that curve fitting is the edge? Look at what Jim Simons and his team have done in the Medallion fund. Jim Simons has multiple times said that they don’t ask why a strategy might work, they just trade it as long as it’s statistically significant.
We might argue this is curve fitting. After all, they test a vast number of hypotheses and scrap those that don’t work, and keep those that do work. If you backtest thousands of strategies you’ll be sure to find something by chance, simply because you backtest a lot.
Is curve fitting a trading edge?
What could be the deciding factors for including or excluding a stock? We list the two most obvious factors:
- Sectors or industries. Some sectors don’t work. Oil stocks and commodities are notoriously bad, in our opinion. But it might change over time. For example, REITS were good in the early 2000, but it all changed in 2007 when the financial crisis started coming in.
- Volatility: volatile stocks are not good. Of course, it depends on the type of strategy. We have always regarded the most boring stocks as the best one to trade. Volatile smallcap stocks have not made money since at least 1963!
Perhaps curve fitting is a trading edge in itself?
There are other factors we looked at, especially when we were day trading stocks, but we believe you’ll make some interesting discoveries yourself if you try this approach.
This is not a recommendation to make complex strategies. But you might consider giving the above approach a try. There are many reasons why a particular strategy might not work for certain stocks and vice versa, other stocks might behave differently.
To summarize, don’t expect a strategy to work on all stocks or all markets. Markets differ. But make sure you know what you are doing! This is a part of the backtesting process that is highly dependent on experience.
Disclaimer: Do your own backtesting.