Welcome to NexusFi: the best trading community on the planet, with over 150,000 members Sign Up Now for Free
Genuine reviews from real traders, not fake reviews from stealth vendors
Quality education from leading professional traders
We are a friendly, helpful, and positive community
We do not tolerate rude behavior, trolling, or vendors advertising in posts
We are here to help, just let us know what you need
You'll need to register in order to view the content of the threads and start contributing to our community. It's free for basic access, or support us by becoming an Elite Member -- see if you qualify for a discount below.
-- Big Mike, Site Administrator
(If you already have an account, login at the top of the page)
in the past when I develop a strategy I first run the optimizer on a range of historical data to find best settings for parameters. Then I enter the settings the optimizer found as best and run backtest on same time period with same settings and ensure that I do get the value returned by the optimizer (sort of a sanity check on making sure I have settings entered correctly) then I run backtests on out of sample time periods. Only after that do I move on to sim trading.
I have a strategy now though, that when I optimize and get some results and settings, and enter those settings to do backtest... it does not match the results the optimizer claimed to have gotten with the same settings. I have meticulously checked every setting and they are set the same on backtest as on best run from optimizer. The results aren't even close.
Any ideas?
Can you help answer these questions from other members on NexusFi?
I am using the Ninjatrader built-in genetic optimizer. I would have thought that each step of the genetic algorithm it would internally run a backtest with selected values... then score value and run genetic routine to choose next generation of parameters. So I am stumped as to why running a backtest with the same settings the optimizer found doesn't match the results it showed. I have verified and re-verified that all settings are the same... including slippage & commisions, etc.
I can optimize other strats and then backtest on same time period with same values and get the same results. At first I thought that it was because this strat used an indicator that repainted past values. I tried telling the backtester and optimizer to not run multithreaded and a few other ideas... no go. I then wrote my own version of the indicator that did not repaint past values and still the same thing.
Broker: TD Ameritrade (soon to be Schwab) & NinjaTrader
Trading: The indexes, ES, YM, NQ & RTY
Posts: 171 since Aug 2012
Thanks Given: 216
Thanks Received: 157
I had problems using Ranko bars. I had a tested strategy that was giving me very good results, then I ran that strategy against live data on the sim account and got a totally different return.
I don't know for sure, but what I think happens is that the Renko bar historical data is truncated so that the bar high/low values are used for the optimization. When running live, it appears that there are ticks outside of the Renko bars, those wide range ticks were hitting my stop loss and changing the results. What I would see is the execution of orders outside of the Renko bar.
Not sure if that is your problem, but when I tried the same strategy with Range bars, I couldn't duplicate the great results in the Analyzer but they matched the results better when I had a live data feed. Because of this, I won't use Renko bars on a strategy.
Yes. Ninja does not use any data outside the bars, so you need bars that show true high low or better anther bar in the strat (tick 1) and then issue trades against this one.
I have also had similar experiences with the NinjaTrader built in Genetic Optimizer. In my case, I used a plain old 5 minute chart, ran the optimizer, then backtested the winner and received wildly different results.
Upon inspecting the actual trades in the NinjaTrader Strategy Analyzer, it looks as if the optimizer wasn't using all of the price data available. What I mean is that if the strategy had a rule to close a position on a MA crossover, the optimized strategy did not close the position whereas the backtested strategy (with identical settings) did close the position on the crossover.
