Welcome to NexusFi: the best trading community on the planet, with over 150,000 members Sign Up Now for Free
Genuine reviews from real traders, not fake reviews from stealth vendors
Quality education from leading professional traders
We are a friendly, helpful, and positive community
We do not tolerate rude behavior, trolling, or vendors advertising in posts
We are here to help, just let us know what you need
You'll need to register in order to view the content of the threads and start contributing to our community. It's free for basic access, or support us by becoming an Elite Member -- see if you qualify for a discount below.
-- Big Mike, Site Administrator
(If you already have an account, login at the top of the page)
If you have limited data available a Monte Carlo Simulation (fancy word) can be used to increase the reliability of your backtest.
This is what the Monte-Carlo-Simulation does: Let us assume that your backtest includes 200 trades. Then the Monte-Carlo-Simulation will pick 200 trades out of the sample at random (this means that typically some trades will be picked several times and some not at all) and then repeats the exercise N times. This will result in N different equity curves, which will give you an indication of the robustness of your backtest.
Advantages:
If your strategy is curve-fitted, it is likely that it will not pass the Monte-Carlo-Simulation very well, as some of the N equity curves will not include the (probably few large) trades that the strategy has been fitted to. So it is a simple, but effective tool to avoid curve-fitting.
Disadvantage:
If your trades are positively or negatively correlated - this could be the case if you use pyramiding and each of the consecutive trades is counted as a separate trade - the Monte-Carlo-Simulation should not be used. It is a tool that only can be used if the outcome of a trade is independent from the outcome of the prior trade(s).
There is an excellent - and quite simple to understand - book on backtesting, which includes the use of Monte-Carlo-Simulation to avoid curve fitting:
It is all real printed books. In earlier times, when I left home, I took some food with me. This is no longer necessary, as I am already carrying some weight around. Also food is available everywhere. But there is a real risk that you starve from boredom, when not at home. A good book is the best insurance. Currently reading
OK, thanks for the replies. If I have a 1000 trade backtest which is curve fitted, and a 1000 trade backtest which isn't. Let's say for the sake of argument, that they have identical result profiles, and let's say each trade's P&L is the same absolute amount. Monte Carlo is going to give pretty much identical results between them right?
If I have a profitable strategy with 30% success because it hangs on to the few big wins very well, Monte Carlo is going to be all over the place right?
I guess I'm asking if there is a class of performance types for which Monte Carlo will tell you useful info, otherwise it's not so useful?
If there are no outliers in terms of P&L, it is difficult to do curve fitting, you will need a lot of parameters.
The 30% success rate is not yet the problem, but let us assume that you do not have 1000 trades, but just 100. And let us further assume that the 30 profitable trades are dispersed and that a few of them make up for more than half of all profits, which could be the equivalent of the net profits. Then a Monte-Carlo-Simulation may well pick a combination of trades which is not profitable at all. The dispersion of the equity curves will show that the strategy is not robust, but that the P&L has relied on a few good trades, which may well have been random.
Thanks. I see what you mean. I can see its use in that case.
I tend to scan through my trades anyway, since you get a better feel for what's going on, and I suppose I've always been wary of a few big trades in testing.
You get more information out of your sample without increasing sample size, so you can actually save part of the sample for the walk-forward-analysis.
What Monte-Carlo can not do:
If your entire sample has been collected in a trending market, and your real trading hits a range bound market, you may suffer from a losing streak, even if the sample size exceeded 1,000. And obviously Monte-Carlo could not predict this, but would have indicated robustness. So choosing the right sample period is not only a question of sample size and curve fitting, but also a question of common sense.
