NexusFi: Find Your Edge


Home Menu

 





Backtesting identical strategy tradestation and multicharts different results


Discussion in MultiCharts

Updated
      Top Posters
    1. looks_one crbucks with 2 posts (0 thanks)
    2. looks_two philippe07024 with 2 posts (0 thanks)
    3. looks_3 jdfagan with 1 posts (0 thanks)
    4. looks_4 Alexlaxmikant with 1 posts (0 thanks)
    1. trending_up 7,814 views
    2. thumb_up 0 thanks given
    3. group 5 followers
    1. forum 7 posts
    2. attach_file 0 attachments




 
Search this Thread

Backtesting identical strategy tradestation and multicharts different results

  #1 (permalink)
crbucks
chicago
 
Posts: 10 since Mar 2011
Thanks Given: 0
Thanks Received: 0

I should start out by saying that my tradestation is the latest beta version with the grail walk forward testing. I have the same identical strategy literally copied and pasted. Tradestation is very slow to backtest (uses only 1 core of my cpu). Multicharts is a lot faster. However both give me completely different results and optimal parameters.

Reply With Quote

Can you help answer these questions
from other members on NexusFi?
NT7 Indicator Script Troubleshooting - Camarilla Pivots
NinjaTrader
My NT8 Volume Profile Split by Asian/Euro/Open
NinjaTrader
Online prop firm The Funded Trader (TFT) going under?
Traders Hideout
The space time continuum and the dynamics of a financial …
Emini and Emicro Index
Exit Strategy
NinjaTrader
 
Best Threads (Most Thanked)
in the last 7 days on NexusFi
Get funded firms 2023/2024 - Any recommendations or word …
59 thanks
Funded Trader platforms
37 thanks
NexusFi site changelog and issues/problem reporting
22 thanks
GFIs1 1 DAX trade per day journal
22 thanks
The Program
20 thanks
  #3 (permalink)
 
Big Mike's Avatar
 Big Mike 
Manta, Ecuador
Site Administrator
Developer
Swing Trader
 
Experience: Advanced
Platform: Custom solution
Broker: IBKR
Trading: Stocks & Futures
Frequency: Every few days
Duration: Weeks
Posts: 50,399 since Jun 2009
Thanks Given: 33,173
Thanks Received: 101,539



crbucks View Post
I should start out by saying that my tradestation is the latest beta version with the grail walk forward testing. I have the same identical strategy literally copied and pasted. Tradestation is very slow to backtest (uses only 1 core of my cpu). Multicharts is a lot faster. However both give me completely different results and optimal parameters.

If you are asking why, there are a huge number of possibilities.

Is this what you are asking?

Mike

We're here to help: just ask the community or contact our Help Desk

Quick Links: Change your Username or Register as a Vendor
Searching for trading reviews? Review this list
Lifetime Elite Membership: Sign-up for only $149 USD
Exclusive money saving offers from our Site Sponsors: Browse Offers
Report problems with the site: Using the NexusFi changelog thread
Follow me on Twitter Visit my NexusFi Trade Journal Reply With Quote
  #4 (permalink)
crbucks
chicago
 
Posts: 10 since Mar 2011
Thanks Given: 0
Thanks Received: 0

I guess I am wondering more what is the most accurate backtesting program and why? I am also a concerned taking parameters that Multicharts says are optimal and using those same parameters in tradestation to run the strategy especially since the same indicator and "optimal" parameters shows a profit in Multicharts but a loss in Tradestation. Since it is literally the same indicator, same symbol, same duration I guess I want to explore more of the differences between tradestations backtesting and multicharts.

Reply With Quote
  #5 (permalink)
 
jdfagan's Avatar
 jdfagan 
Pacifica, CA
 
Experience: Advanced
Platform: AmiBroker, TradeStation, NinjaTrader
Broker: Options House
Trading: Futures, Stocks
Posts: 34 since Jun 2009
Thanks Given: 68
Thanks Received: 44


crbucks View Post
I guess I am wondering more what is the most accurate backtesting program and why? I am also a concerned taking parameters that Multicharts says are optimal and using those same parameters in tradestation to run the strategy especially since the same indicator and "optimal" parameters shows a profit in Multicharts but a loss in Tradestation. Since it is literally the same indicator, same symbol, same duration I guess I want to explore more of the differences between tradestations backtesting and multicharts.

Maybe you can post your exact settings you used for the 2 backtests for the 2 programs. Start first with comparing backtests before comparing optimizations since some optimization engines are stochastic in nature (i.e., random aspects used in Genetic engines and I believe also in Genetic Grail's Walk Forward tester).

Example of things to ensure across backtests across different platforms, ensure same start/end dates, same timeframes, same IOG settings, turn off bar magnifier in MC because TS probably doesn't have, etc. Also, make sure to look at individual trades (especially first one and make sure that is in sync - sometimes buffers are different in the beginning and they start trading on different days). Maybe you can post the settings for each backtest for each program and then post the results of each and possibly screenshots too of the trades, etc.

JD

Follow me on Twitter Reply With Quote
  #6 (permalink)
 
philippe07024's Avatar
 philippe07024 
New York, NY
 
Experience: Intermediate
Platform: TradeStation
Broker: TradeStation
Trading: Forex
Posts: 33 since Feb 2011
Thanks Given: 16
Thanks Received: 16

My potential interest in the MC lies in using a tool designed to optimize portfolios - particularly to minimize drawdowns.

I noticed a difference in the results of a portfolio optimization between Multicharts (MC) and TradeStation (TS) or NinjaTrader (NT).

With MC in trial, my first test on a proprietary strategy resulted in identifying a discrepancy with the results obtained with the other trading platforms.

The results are obtained with the genetic optimization with 100 generations of 100 individuals on TS (1 symbol at a time), NT (where I optimized a portfolio) and finally MC (portfolio and individual). All tests performed on TS data (my broker/platform).

Here is a summary of the net profit obtained using the net profit as fitness function:

TS individual EU: $113,810 GU: $135,629
NT portfolio EU: $118,810 GU: $117,720
MC portfolio EU: $117,799 GU: $49,121
MC individual EU: $157,635 GU: $130,341

Did anyone run similar tests?

Reply With Quote
  #7 (permalink)
Alexlaxmikant
India
 
Posts: 18 since May 2010
Thanks Given: 39
Thanks Received: 8

One thing is for sure, in MC there is always a difference between Portfolio backtesting and MC individual backtesting results.
Reason for this is bar magnifier, in portfolio backtesting they don't use it and with manual inspection I have found that MC
individual backtester is more accurate.
Open the list of trades report and see if one particular signal got repeated for no reason, because that happen some times.

Alex

Reply With Quote
  #8 (permalink)
 
philippe07024's Avatar
 philippe07024 
New York, NY
 
Experience: Intermediate
Platform: TradeStation
Broker: TradeStation
Trading: Forex
Posts: 33 since Feb 2011
Thanks Given: 16
Thanks Received: 16


Alexlaxmikant View Post
One thing is for sure, in MC there is always a difference between Portfolio backtesting and MC individual backtesting results.
Reason for this is bar magnifier, in portfolio backtesting they don't use it and with manual inspection I have found that MC
individual backtester is more accurate.
Open the list of trades report and see if one particular signal got repeated for no reason, because that happen some times.

Alex

Actually, on the charts side I made sure not to use the bar magnifier.

Reply With Quote




Last Updated on April 26, 2012


© 2024 NexusFi™, s.a., All Rights Reserved.
Av Ricardo J. Alfaro, Century Tower, Panama City, Panama, Ph: +507 833-9432 (Panama and Intl), +1 888-312-3001 (USA and Canada)
All information is for educational use only and is not investment advice. There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
About Us - Contact Us - Site Rules, Acceptable Use, and Terms and Conditions - Privacy Policy - Downloads - Top
no new posts