Welcome to NexusFi: the best trading community on the planet, with over 150,000 members Sign Up Now for Free
Genuine reviews from real traders, not fake reviews from stealth vendors
Quality education from leading professional traders
We are a friendly, helpful, and positive community
We do not tolerate rude behavior, trolling, or vendors advertising in posts
We are here to help, just let us know what you need
You'll need to register in order to view the content of the threads and start contributing to our community. It's free for basic access, or support us by becoming an Elite Member -- see if you qualify for a discount below.
-- Big Mike, Site Administrator
(If you already have an account, login at the top of the page)
How to measure data feed latencies between continents?
I don't want to discuss the pro + con of using VPS here - thats obvious and it's mentioned many times at BM. It's a thread about measuring latencies with the emphasis of getting the most accurate wall time on a windows system.
Getting the time from an local attached stratum 1 source can be done with sub ms precision. Getting the time from within Windows (7) is a more complex issue. But with the combined use of the system timer and the performance counter this can be solved.
Starting with Windows 8 there is no need for that, because some kernel functions were added and the function GetSystemTimePreciseAsFileTime makes all of the stuff for us.
You can also using Linux to get an accurate system time.
Thanks to artemiso, to flash that topic. Using an hosted OS brings more timing problems ahead, because the guest can't share the hosts timing system 1:1.
The following latency test's are planned (each test is running a few days):
Common: Local attached stratum 1 (GPS) based time reference with NTP Server (+-500us)
* Test on Win7 (clock controlled by Microsecond Resolution Time Services for Windows solution)
* Test on Win8
* Test on Linux
* Test on Win7 + VMWare Win7
* Test on Win Server 2012 (currently no license for that)
EDIT: Some test are dispensable, because i have an better idea to grab the OS differences (See Post #48). With my main Setup i will run the R|API test to measure the Overall latencies. To get a better feeling about the plattform dependend latencies (just for interest - not for this project) i apply the local stack test.
I make some more tests to see if it is easier to get the current time direct from the stratum 1 source (without utilizing the clock control). An optimized UDP path the the stratum device should have a sub ms RTT.
If anyone want to buy me a SolarFlare NIC, i include this in my tests!
With regard to timing anything in these areas it is definitely worth being aware of scheduling issues, these two PDF's are worth a lunchtime read.
As a multi-decade veteran of OS scheduling I would be interested to hear how people handle this on current VPS boxes, e.g. by tweaking settings, limiting access, etcetera.
If it's too off-topic then please accept my apologies.
You are welcome. For my tests i can only do the following to reduce context switches:
1. Stripe down / disable OS Services
2. Test program is attached to one cpu core and thread
3. Switch cpu power management off
4. Run test program at higher priority
If anybody has other recommendations, feel free to explain. BUT the thread belongs to get precise wall time and not to reduce lag within the system. The cost of implementing sophisticated solutions (there are many of them) doesn't fit to the possible gains in my trading account.
I am getting anywhere between 2 to 26 milliseconds on my VPS, averaging about 8 ms, as predicted. This is a fresh install with practically no load, but these two factors are not the focal point. I expect it's another order of magnitude larger on EC2.
EDIT> The moral of the story is that most people don't need to make any kind of sub-second latency measurement. Here's an algorithm: They just need to determine their budget based on their returns, find options with a steady internet connection and power supply (it's true that a server at a data center generally qualifies), then do the elementary ping and bandwidth tests to choose the best option amongst those available at that budget. At the end of the day, 99% (I'm generalizing) of people extract no measurable value from their $150/month dedicated servers and are actually better off with getting a better internet service for their home computers.
