Welcome to NexusFi: the best trading community on the planet, with over 150,000 members Sign Up Now for Free
Genuine reviews from real traders, not fake reviews from stealth vendors
Quality education from leading professional traders
We are a friendly, helpful, and positive community
We do not tolerate rude behavior, trolling, or vendors advertising in posts
We are here to help, just let us know what you need
You'll need to register in order to view the content of the threads and start contributing to our community. It's free for basic access, or support us by becoming an Elite Member -- see if you qualify for a discount below.
-- Big Mike, Site Administrator
(If you already have an account, login at the top of the page)
What are the characteristics that make a BOT a good BOT ?
I am curious to learn what people would give as a good characteristic for an automated trading BOT with regards to the trading results of the BOT ?
Let's back-test the BOT over 12-14 months...
- average MFE / MAE
- e-ratio
- number of trades per year / per month
- number of different instruments able to trade with the same BOT
- annual return / equity curve
- anti-curve fitting measures
- any other criteria ?
I am by no means a programer (and incidentally I've followed your treads and webinar with great interest). However, as I've explored automation projects and talked to different quant connections, one thing that I learned from the ones I've talked to is that they don't take the training wheels off until 6 months of live monitoring and they don't' put real money behind it until 12 months.
Then more interesting, is the trail period where they cram as much money behind it to see what the breaking point of the algo is volume wise before scaling back down.
Anyways, that's what I've learned in the last few months for whatever it's worth.
Since you don't have this on your list, I though I would mention that Ed Seykota uses MAR (annual return / max drawdown) as one of the measures. Not sure if he uses other criteria as well.
Since you are only using 12-14 months of data, I am assuming that your systems are fairly short-term in nature. If that is the case, may I suggest to try and get additional out-of-sample data. When I was still doing these type of tests, out-of-sample data really tended to throw a spanner in the works.
I think one of the most important things you have to think about is how much pain a particular strategy will generate in live-trading and whether you, as the trader, can endure that. As the late Bruce Babcock wrote many years ago:
In keeping with this, I look at the % of profitable trades, the % of profitable days, months and years, the amount of time between new equity highs, the ratio of average annual gain to average annual max drawdown, the relative smoothness of the equity curve, and the distribution of gains across markets (assuming you're trading a portfolio of markets, not just a single market).
One other thing I look to do is to test whether the distribution of returns that my system generates is distinguishable from a random system. There are statistical tests available to do that.
- Win rate
- Profit factor
- Avg Winner / Avg Loser
- Net Profit / Max drawdown
- Avg Trade Profit Amount
- Commissions / Net Profit - I don't like systems where commissions take away more than 5-10% of my profit
- Smoothness of equity curve
- Best hours of day or days of week it works best - some strategies don't trade on Mondays, others avoid Fridays, only trade RTH AM, etc.
- Does it work well trading longs and shorts? Some strategies I only trade one side, depending on instrument and market regime
Regarding the smoothness of the equity curve, Curtis Faith introduced me to the r-squared idea. I believe it is in the book Way of the Turtle, but I may be mistaken. There may also be some reference to this on the TradingBlox site. I can't recall the exact details of how he used it, but several others on the forums found it extremely useful.
In my own LTTF trading and backtesting on forex, I generally found that the smoother the backtested equity curve, the less likely the system was to perform well going forward. Bear in mind though that these systems did not trade every day, heck sometimes not even every month, and needed strong trends to be profitable.
Part of "smootheness" depends on what type of trading software you are using. For example, Ninjatrader shows equity curves based on time, and not NAV, but cash balance. So while you may be holding a losing position if it eventually turns into a winner you see nothing but straight-up equity curves even though some of those trades may have been difficult to hold.
Other systems like Tradestation can show equity curves based on # of trades, so while there may be lots of time between trades, you are just looking at each individual trade result irregardless of time.
I'd like to say I have a method but I can't really articulate it now so maybe it's more eyeballing than I'd like to admit.
Thinking about it now, I'd say first focus on Win Rate and Profit Factor first and foremost, especially PF. If I can get something 1.2 or higher then I dig deeper. If it's less than 1.2 then I bail. Once I have a candidate I look at win rate, how often it trades, how much I have to spend on commission, avg win/avg loss size, etc.
If my avg win/loss is less than 1 I better be getting win rates over 50%. If my avg winner/avg loser is greater than 1 than I don't need such a high win rate. I look at MFE and MAE to see if I should adjust my exit/stop placement/profit target strategies to see if I can increase my avg win/avg loss ratio.
I also look at individual trades on the equity curve to make sure I'm not getting most of my gains from a single winning trade. Conversely, if most of my losses are on one or two big losing trades then I may see if there's a way to filter out the big losers while keeping most of the winners.
From there I may dig more into seasonality such as days of the week, hours of the day, months of the year, and try to determine when is the best time(s) to run this strategy on the selected time frame. I think many traders underestimate the importance of seasonal patterns/trends.
Lastly I start to analyze the distribution of wins/losses. Using the "Periods" tab in Ninja strategy analyzer I look at each month and year of the trading period to see how it consistently makes money. Do I sometime have to take 5 losing months in a row and if so how long will I have to recover from drawdown? What I'm looking for here is how much pain I may have to endure and, while being honest with myself, determine if I can or want to go through that pain. From there I may want to run some Monte Carlo simulations but honestly I have not felt the need to do that at this point because of all of the above analysis and by keeping my position sizes small, risking 0.5-2.0% of capital per trade.
