Selecting good trading systems

    With any trading type, we know only its past performance. And, as the saying goes: “past performance does not guarantee…. “. We can look at some statistics to gauge the probability of future profit, but it’s a lot of guesswork. For example, we can look at the profit factor or a total profit made by a method. If the method has lost money so far, it’s unlikely to become profitable. 
    I wanted to put this into perspective and get actual numbers. So, I’ve tested some trend-following methods: 
  •     Single SMA ( entry/exit when price above/below SMA) 
  •     SMA Cross ( entry/exit when short SMA above/below long SMA) 
  •     Donchian Channel (entry/exit when price above/below the channel of given length) 
  •     Donchian Double Channel (entry/exit when price above first channel / below the second channel) 

    I’ve tested various lengths for each system, usually within 5 – 350 trading days with a step of 5. This way, I got 7656 different trading systems. The in-sample period was 2007 – 2014, out-of-sample was 2014 – 2021. 
    I trained PrimeClue to recognize which trading systems will be good in the out-of-sample period. Training features were: profit factor, accuracy, number of trades, expectancy, max drawdown, total profit, last day drawdown, average trade profit, and number of rules (1 for single Sma/Donchian, 2 for the others). All values were extracted from trading during the in-sample period. Then, I checked each system’s profit during the out-of-sample period. I labeled each system with three labels: 
  1.     Profit better than the median of all systems (higher than 197% profit) 
  2.     Profit in the top 10% of all systems (higher than 386% profit) 
  3.     Profit in the top 1% of all systems (higher than 600% profit)
Take note that training features hold no information about the type of system, so a trained model can be applied to fundamental analysis results as well. Correlation between features and future performance varied: 
Metric Correlation
Profit factor 0.55
Accuracy 0.33
Number of trades -0.20
Expectancy 0.56
Max drawdown -0.46
Total profit 0.51
Last day drawdown -0.49
Average trade 0.54
Number of rules 0.04

    Expectancy has the highest correlation to out-of-sample performance. We can try to use this to select a system for the future. Let’s say that for any system where in-sample expectancy was higher than the median (0.33), we guess that system’s profit during out-of-sample will be higher than the median (of all systems). Processing like this would give us an accuracy of 70.1%
    Artificial Intelligence can find some more rules to work with and eventually achieve higher scores. The table below shows the accuracy for positive predictions and AUC for each label. It also shows the number of nodes in each trained model. Each training lasted 5 minutes.

Label % Accuracy AUC Model nodes
Top 50% 75 0.81 24
Top 10% 46 0.86 14
Top 1% 7 0.82 17

It seems like PrimeClue can train models to give results much higher than random (AUC above 0.5 and predictions accuracy higher than respective %). 

    I’m pleased with the results. I will use this method to accept/reject systems for my trading.


Popular posts from this blog

Improving trading system profitability with Artificial Intelligence

Fighting credit card fraud with AI