Let's discuss the most dreaded aspect of automated trading - overfitted algorithms.
First, what is overfitting?
I'm glad you asked. Because if you don't already know, this is an absolutely vital part of learning to be an automated trader.
Overfitting an algorithm is simply the act of making it fit past data so well (rendering nearly perfect backtests) that it cannot possibly perform well in a future fashion. I've uploaded the Wikipedia article picture so you can have a visual reference.
In the visual we see red and blue dots (wins and losses in our example), and a black and green line (the algorithm settings).
An algorithm which is overfitted will behave like the green line on past data. It will perfectly encapsulate the wins, delivering a perfect (or near-perfect) backtest.
But this is a problem - in forward usage we cannot expect the blue and red dots to end up in the same places (assume their distribution is more random and will slightly change places in future data, which they will). Therefore, the green line will not encapsulate the data moving forward.
Instead, the black line represents proper fitment. It is much more generic, separating the blue and red dots in a manner that is not exacting, but still gets most of them. This is indicative of a fitment that can continue to operate in a forward fashion as the red and blue dots change position around their border.
How do we avoid overfitting?
Overfitting occurs when we become very specific with our settings. Examples of potentially overfitted settings:
Odd timeframes: 4m for example
Timestring: 0935-1015,1045-1115,1130-1155,1300-1325,1400-1430:345
Static take profit: 1.074
Hard stop: 3.29
What do we do instead?
Everything you choose must have a good reason within normally accepted bounds as well as not being a number dialed in down to tenths or hundreths.
E.g. we know that market volume is better in the morning, but we want to skip the opening bar - so 0935-1200 is a reasonable timeframe.
E.g. we know that the strategies were built to take 1 as the standard take profit, so selecting large rounded floats makes more sense - 1.0, 1.5, 2.0. Otherwise a dialed in number can be overfitted to the exact amount trades may have moved in the past.
These concepts will be especially important with the trade filters coming in EVO v1.6. I will post information on them as we get closer to releasing it.
How do I know if it's overfitted?
If your new settings fail terribly immediately, or run for a week and then fail, it's probably overfitted.
