Welcome to NexusFi: the best trading community on the planet, with over 150,000 members Sign Up Now for Free
Genuine reviews from real traders, not fake reviews from stealth vendors
Quality education from leading professional traders
We are a friendly, helpful, and positive community
We do not tolerate rude behavior, trolling, or vendors advertising in posts
We are here to help, just let us know what you need
You'll need to register in order to view the content of the threads and start contributing to our community. It's free for basic access, or support us by becoming an Elite Member -- see if you qualify for a discount below.
-- Big Mike, Site Administrator
(If you already have an account, login at the top of the page)
Backtesting identical strategy tradestation and multicharts different results
I should start out by saying that my tradestation is the latest beta version with the grail walk forward testing. I have the same identical strategy literally copied and pasted. Tradestation is very slow to backtest (uses only 1 core of my cpu). Multicharts is a lot faster. However both give me completely different results and optimal parameters.
Can you help answer these questions from other members on NexusFi?
I guess I am wondering more what is the most accurate backtesting program and why? I am also a concerned taking parameters that Multicharts says are optimal and using those same parameters in tradestation to run the strategy especially since the same indicator and "optimal" parameters shows a profit in Multicharts but a loss in Tradestation. Since it is literally the same indicator, same symbol, same duration I guess I want to explore more of the differences between tradestations backtesting and multicharts.
Maybe you can post your exact settings you used for the 2 backtests for the 2 programs. Start first with comparing backtests before comparing optimizations since some optimization engines are stochastic in nature (i.e., random aspects used in Genetic engines and I believe also in Genetic Grail's Walk Forward tester).
Example of things to ensure across backtests across different platforms, ensure same start/end dates, same timeframes, same IOG settings, turn off bar magnifier in MC because TS probably doesn't have, etc. Also, make sure to look at individual trades (especially first one and make sure that is in sync - sometimes buffers are different in the beginning and they start trading on different days). Maybe you can post the settings for each backtest for each program and then post the results of each and possibly screenshots too of the trades, etc.
My potential interest in the MC lies in using a tool designed to optimize portfolios - particularly to minimize drawdowns.
I noticed a difference in the results of a portfolio optimization between Multicharts (MC) and TradeStation (TS) or NinjaTrader (NT).
With MC in trial, my first test on a proprietary strategy resulted in identifying a discrepancy with the results obtained with the other trading platforms.
The results are obtained with the genetic optimization with 100 generations of 100 individuals on TS (1 symbol at a time), NT (where I optimized a portfolio) and finally MC (portfolio and individual). All tests performed on TS data (my broker/platform).
Here is a summary of the net profit obtained using the net profit as fitness function:
TS individual EU: $113,810 GU: $135,629
NT portfolio EU: $118,810 GU: $117,720 MC portfolio EU: $117,799 GU: $49,121
MC individual EU: $157,635 GU: $130,341
One thing is for sure, in MC there is always a difference between Portfolio backtesting and MC individual backtesting results.
Reason for this is bar magnifier, in portfolio backtesting they don't use it and with manual inspection I have found that MC
individual backtester is more accurate.
Open the list of trades report and see if one particular signal got repeated for no reason, because that happen some times.