Welcome to NexusFi: the best trading community on the planet, with over 150,000 members Sign Up Now for Free
Genuine reviews from real traders, not fake reviews from stealth vendors
Quality education from leading professional traders
We are a friendly, helpful, and positive community
We do not tolerate rude behavior, trolling, or vendors advertising in posts
We are here to help, just let us know what you need
You'll need to register in order to view the content of the threads and start contributing to our community. It's free for basic access, or support us by becoming an Elite Member -- see if you qualify for a discount below.
-- Big Mike, Site Administrator
(If you already have an account, login at the top of the page)
NinjaTrader with Neural Networks using Encog framework
kk240 --now that you mention them I recall a number of the heuristics you listed. It's been almost 2 decades since I was into this sort of thing and hopefully the (re)learning curve won't be as steep this time around.
gregid -- I have almost no experience with recurrent networks but plan to do some research. On the one hand it seems if trading is (mostly?) about pattern recognition one might naively suppose it should be possible to come up with a network that is adequate in the first instance with the right training set (able to cope with whatever a particular instrument might throw at it, or at least know enough to bail when things start going sideways), in which case the trick might be developing the right training set & the necessary failsafes. On the other hand I've read recurrent networks produce better results with sequences having certain characteristics, which might include financial market order flow sequences--trust time will tell.
In any case I'm still trying to assure myself of what the NT optimizer does, from a mathematical perspective, versus e.g., how AI techniques in general find solutions, and from past experience my bottom line is to avoid getting bogged down by theory at the expense of the practical--a bad habit of mine.
Can you help answer these questions from other members on NexusFi?
I have found (from my limited experience) that reducing the training set size and the number of hidden neurons has helped enormously. I'm using a MLP feedforward with back prop, but the idea should help any neural net.
Reducing the training set size from around 4000 items to 250 both speeds up training and helps with generalization. I have tried random sampling and what I call normalized random sampling. This is where I sample the data in a way that generates a training set with a normal distribution. As I have been trying to predict the X Day ROC the raw data has a fat tailed distribution. So using a purely random sampling technique produces a training set that still has a fat tailed distribution. This type of distro causes the neural net to fit to the data I care about least.
With the hidden layer I'm starting with 2/3 the number of input neurons (e.g. max( 0.66 * input count, 1)) and only having one hidden layer. I have read where people start with more hidden neuron a then start removing them until there is a big increase in the error as well as vice versa, starting with fewer and adding them in.
As mentioned by kk240 you also need a good fitness function. For the prediction I'm using r squared. I'm also training for one iteration and then testing. I stop once the error or fitness measure is not improving.
I have no formal background in math or computer science so take this with a grain of salt.
I have read lot of articles of neural nets and finance and read forums where people are using neuro shell.
example you have to stop noxa indicator optimazation by hand that it will not overlearn. Silly human helping software, it should be technology is helping humans.
They are complaining they neural networks are over learned and curve fitting. I think they have feed data to neural network which just not solve the problem what they try to solve or they network and structure of network is not good.or fitness function wrong.
Have to do lot of work for finding good setup for neural network.
We have to remember when start creating system, do we want, system generates, buy sell signalls, or do it try predict future values and how far, and what time frames we are using.
There is so many diffrent ways to do it, and there is no allready chew it setup. sometimes different type of sampling is good for one problem and not to other.
example if you fill your network 10000 values of RSI and try to predict price change tommorow, offcourse that network is curve fitted if its random sampled or whatever sampled.
I will try to use enqoc to squeeze several inputs to one value which helps me to intrept where we are in current market.
it won't predict anything.
Neural networks are just type of regression analysis where hidden layers add non linearity
I've been experimenting with the sample application included with version 2.6 of the Encog core applied to EUR:USD 240 minute forex data. For the record the version 2.6 package comprises the following:
So far what I've done is
- add the EncogStreamWriter indicator to the 240 minute EUR:USD chart keeping the default indicator and parameters (MACD(12,26,9)) to generate data for the NinjaIndicators C# app
- run NinjaIndicators in Visual Studio with the data generated by EncogStreamWriter to construct a network trained according to the default hypothesis (BestReturn)
- add EncogMACDBased indicator to the 240 minute EUR:USD to run with the NinjaIndicators output net
- observe the resulting plot
While a useful exercise it turns out I need to learn more about the EncogMACDBased.BestReturn predictor used in the example before potentially writing a strategy to exploit it.
An approach might be to repeat the exercise using an indicator derived from an existing (successful) NT strategy with the following intent:
1. at this stage, to avoid having to design a network capable of resolving nuances and optimizing upon features of certain indicators' raw output that I already know can produce an effective strategy, including 1st- and 2nd-order rates of change; in other words, to use the existing strategy as an input preprocessor.
2. to try to write a new NN-based strategy to generate orders from any signals observed in the BestReturn output by comparing bar by bar with existing strategy behavior, and thereby learn more about it--specifically whether it's of actual use in its present form and if so, e.g. whether it might improve performance of the existing strategy.
Subsequently the plan is to try to relax pre-processing, putting more of the load on a purpose-built network, to avoid dependence on the built-in predictors (which appear to me at the moment at least to be in the same category as NT's strategy optimizer, if not equivalent in operation).
As an aside, BestReturn is defined in the encog-core-cs.App.Quant.Indicators.Predictive object of encog-core-cs.dll but unfortunately encog-core-cs.App is not included in the source distribution (encog-core-dotnet-2.5.3.zip).
That said, I'm not sure yet if it would make sense to expose encog-core-cs.App to NT.
In any event it's not a show-stopper since the entire project, including BestReturn code, is available on the encog-cs.googlecode.com subversion trunk.
I may download a read-only version of the project once I get SVN up and running (again), bearing in mind the code appears still to be in a state of flux and therefore likely not a good idea (and it may not be permissible) to patch the release version of the source--still have to find out for sure what the guidance is in that department, if there are any beyond the Apache license.
At this point I think it could be worthwhile to buy one of Jeff's books (e.g., "Programming Neural Networks with Encog2 in C#")
(Edited to add: I don't mean to shill for Jeff, just that there appears to be sufficient substance here and being disposed by nature to doing things the hard way I think for once it may be ok to RTFM to see if it helps, since the idea is to prove or disprove ASAP whether I can use Encog to improve my trades)
Thanks for that JonnoB. My problem has to do more with becoming oriented after abandoning the topic years ago than reverse engineering DLLs since the Encog framework is open source. In fact largely because of your suggestion I finally bit the bullet and (re)installed Tortoise SVN to create a local Encog repository--now have everything required in that respect.
kk240 pretty much summed up my feelings at the time I left off working with AI ("Neural networks are just type of regression analysis where hidden layers add non linearity"). The only reason I'm back at it is I reviewed a demo of a SharpNEAT implementation of the double pole balancing NN solution recently that reminded me of a Forex scalping strategy I have yet to implement and (now that I'm into Forex and have a little time to waste and a strong desire to make money) figure there may be something to it I've been missing.
I have to admit to kk240's take on things because the first thing I did after modifying Encog's complementary data export indicator ("EncogStreamWriter") to export NN input data from the candidate strategy was run a simple spreadsheet cluster analysis. Anyone who has ever run a cluster analysis will appreciate the optimism, and as one might expect the results were disappointing. It (my chosen input to any neural net) appears at first glance to lend itself to a temporal (recurrent) solution, as gredid predicted--in other words it doesn't cluster.
After giving up on the complementary Encog BestReturn approach mentioned earlier (something of a red herring in the grand scheme) set out to run the Encog Market demo to learn what Encog's concept of temporal nets is. It failed with an IO error, very likely a consequence of the way I've got it configured (perhaps trying to follow an absolute path).
Edited to add: for anyone wanting to work with the framework, as expected the error in the Market example program turned out to be a hardwired directory path in Encog.Examples.Market.Config (line 39)
Edited to add: following 2 lines added at line 277 in encog-cs.encog-core.encog-core-cs.App.Quant.Normalize.EncogNormalize.cs to fix an error occurring the Market Predict example:
if (this.Stats == null)
this.Stats = new NormalizationStats(list.Count);
i.e., just before the following line
this.Stats.Data = list.ToArray<NormalizedFieldStats>();
Encog is on the back burner for the time being after developing the required StreamWriter indicators for 2 NT ("conventional") strategies to input data into the Encog framework, trying to develop an appropriate neural net (strictly Feedforward / Resilient propagation for all trials--Note 1 **) and strategies to implement the resulting nets.
Encog Workbench (EW) was used for all trials.
70 - 80 nets were created with a half dozen training sets. Some converged to 1% (or better error), some did not converge (as expected). No (EW) strategy exceeded the performance of any parent NT strategy that employed the conventional heuristics available to NT.
** Note 1:
The full NEAT application in the Encog framework apparently has a bug in it (it can't find the training set) and I'm not the only one to report this. As we all know, NEAT is what one appeals to after one has exhausted time available to test topologies & initial conditions manually. Don't have time to debug the EW source.
Summary so far:
- Encog is a great platform for developing basic neural nets--best open source framework I've ever seen, but I'm not a big follower of neural net platform development.
- it seems to have at least one bug I haven't managed to resolve yet (NEAT training sets) in the Workbench, but implementing your session directly in code may be the work around.
- the good news is, after returning to one original, "conventional" NT strategy to try to prune it for better net performance, the conventional NT strategy's performance increased 50% in backtesting, and so far looks great in live data simulation under NT
Edited to add: I'm guessing the NEAT bug is in the Workbench, since the programming is excellent (and GUI programmers are to application programmers as drummers are to musicians, I've heard)
JonnoB: you advise excellent heuristics given my experience last week with non-predictive neural nets--namely, more input is not better and/but one likely needs a complete representation of the data in some statistical sense.
For me the acid test is whether I can beat the parent NT strategy right now--any results there?
I am currently using metatrader with the FANN library. It is not as fully featured as Encog, but does the job. I plan to use Ninja Trader when I eventually move to Interactive Brokers. So I cant comment on the Ninja Trader optimizer as all I have done so far is port a few indicators.
I have recently purchased the book "Trading on the Edge" by Guido Deboeck. The second chapter deals with pre and post processing and was where I got the idea of a reduced training set. I had a look on google books and unfortunately not all of chapter two is shown. The good thing is while I was looking through that book I found a reference to another, "Cybernetic trading strategies" by Murray A. Ruggiero. I have been wanting to predict turning points in the market and couldn't get past thinking of it as a classification problem. On pages 245 and 246 he creates a forward looking oscillator that you use as your target for prediction. Brilliant
Whereas I tend to look at every neural net ("NN") application as a clustering problem
On that topic while trying to improve MAEs and MFEs of trades generated by my best performing NT (i.e., non-NN-based) strategy I noticed the MAEs and MFEs do happen to cluster for this particular strategy at least, so once the modded strategy is online and generating income I may try to develop an NN-based strategy for that purpose--again, mostly for the learning experience, not because I have high hopes.
Edited to add: Having tried to see or influence the future as well as the past any way I know how, and failing for the most part, I suspect the "forward looking oscillator" may not be everything it's reputed to be, but as time & priority permit will check it out. Please let me know if you have good results, although it's been my experience good results are not widely advertised
As an aside, as one might surmise the reason for focusing on MAE & MFE is to exit or reverse existing orders that are wandering into the "losing trade" MAE cluster and potentially to pile into orders while they are still in "winning trade" MFE territory.
It may be that the time-trajectory of an excursion matters (i.e., before price crystallizes into an MAE/MFE when the order is covered), in which case the exercise might lend itself to prediction.
In any event and practically speaking, this has meant introducing order update and execution event handling into the strategy over the last few days. While the worst of the conversion is over it greatly increased the complexity of the code (2600 lines or so including debugging) and if there are undetected bugs in the way orders are being handled I fear the transition to live trading from real time simulation. Should know for sure later this week.
