ALPHA GROWTH CAPITAL
  • Home
  • Portfolio
  • Services
  • Free Resources
    • Macro Economic Models ​
    • S&P500 Performance Indicator
    • Historical Data
    • AGC Monthly Newsletter
  • Join Us
  • Articles
  • Contact Us
  • Terms & Conditions

Sharing Our Thoughts

Backtesting Strategies, Does It Work?

21/9/2021

Comments

 
If you are a quant investor or a wanna be one, the first thing you do is to define an investment strategy, backtest it and eventually launch it live.

How many times have happened that the strategy has not performed on a real or demo account? If the answer is nearly always, let me explain you why it has happened and let me suggest you what I think is the best way to perform a backtest prior launching your new strategy live.

There might be four main reasons why your backtest did not work in real life:
  1. You made a mistake in setting up the equations of your model
  2. Your model is over-fitted
  3. The market conditions are different from the ones you have used to train your model (e.g. new correlation among asset classes)
  4. You have not given enough time to the model to perform

Assuming you have not done any mistake in setting up the equations and you are patient enough, most likely, your strategy did not work in real life because your model is over-fitted.

With the term “over-fit” I mean that the model knows quite well how to behave if the past would exactly represent itself while it would partially/total fail if future events are not exactly the same as past ones.

Let me illustrate this with an example. 

Imagine you have figured out the importance of rotating your investments among uncorrelated asset classes depending on the market conditions, now what you will try to do is to define some rules that will tell you in which asset class to be invested in. It is 2021, so you decide to use Machine Learning and more specifically you define a classification problem. Typically a classification problem tells you what to select depending on a given set of input variables. Input variables that in the case of investing could be: market volatility, return of a given asset, volatility of a given asset,… You then take your Python or Matlab software, build your model, train your data and… you are astonished by the outstanding performances of your strategy. I did the exercise and I have assumed that the strategy rotates among three asset classes: US T-bond, gold and US equities. I have then assumed that my holding period is between one week and one month. The result: 
Picture
You’ll then get super exited because when you look at the backtested data, you know that in no time you’ll be rich and you are confident enough that the strategy is robust enough to survive a market crash because your model was also trained with data including bad years like 2007, 2008 and 2020. 

What next? You go to your brokerage account, start implement the strategy, the first month does not go as expected, so you say that it was a one-off. Then eventually there is an equity market downturn and your drawdown is higher than 15%, you lose money and you: in the best case ask yourself what has happened and in the worst case you blame the rigged market.

What went wrong?

Long story short: your model is over fitted and this is not the way you run a back test.

The way I personally run a backtest is to mirror exactly what I would do in real life. If I have to take an investment decision in e.g. March 2016, the data I would have in real life to train my model are the ones available up to March 2016 and not September 2021. The way that a backtest should be structured is to iteratively train your model within the backtest. By doing so, your model is still over fitted (assuming we are still dealing with the previous example) but the outcome of using it is closer to what would happen in real life. The differences would be due to e.g. fees, orders slippage,…

​By running a backtest in this way, the performances of the strategy are now:
Picture
A huge difference is not it?

In my opinion, this is the most appropriate way to backtest a strategy and provide a very good idea on how your investment would perform in real life without losing money while implementing a possibly over fitted investment strategy.

I hope you find it useful and you have learned something new, to the next time!
Comments

    ​Join our monthly newsletter, it is free!

    At the beginning of the month, we send out an email reviewing the performances of NEXT-alpha, provide the S&P500 and Gold outlooks based on our proprietary models and reviewing the current economic environment. ​
Subscribe to Newsletter

Subscription*​

Suitable for individuals that would like to implement the strategy themselves in their brokerage account.
​
It is only 37.5 USD/month! To subscribe Contact us!
​

Separately Managed Account

Suitable for investors with at least $20,000 to invest that would like Alpha Growth Capital to manage the asset on the client behalf.

Trades and performances are reported in real time, the client has the intraday liquidity, the funds are held with Exante and there is no lock-up period.

​Want to know more? Contact us!
​
SMAs are only provided to certain countries. 

The information, analysis, data and articles provided in this and through this website are for informational purpose only. Nothing should be considered as an investment advice. Alpha Growth Capital LLC does not make any recommendation to buy, sell or hold any security or position. The website and information provided through it are marketed “as is”. There is no guarantee that anything presented and provided on this and through this website is complete, accurate and correct. Relying on the information provided on this website and through its communication channels is done entirely at the individual own risk. The owner of Alpha Growth Capital is not a registered investment advisor under any security law and nothing provided in this and through this website should be interpreted as a solicitation to buy, hold or sell any mentioned financial product or service. Past performance is not indicative of future results. Any financial decision is at the sole responsibility of the individual.
​

By navigating in this website, you agree to its Terms & Conditions
​
  • Home
  • Portfolio
  • Services
  • Free Resources
    • Macro Economic Models ​
    • S&P500 Performance Indicator
    • Historical Data
    • AGC Monthly Newsletter
  • Join Us
  • Articles
  • Contact Us
  • Terms & Conditions