Welcome to NexusFi: the best trading community on the planet, with over 150,000 members Sign Up Now for Free
Genuine reviews from real traders, not fake reviews from stealth vendors
Quality education from leading professional traders
We are a friendly, helpful, and positive community
We do not tolerate rude behavior, trolling, or vendors advertising in posts
We are here to help, just let us know what you need
You'll need to register in order to view the content of the threads and start contributing to our community. It's free for basic access, or support us by becoming an Elite Member -- see if you qualify for a discount below.
-- Big Mike, Site Administrator
(If you already have an account, login at the top of the page)
How to measure data feed latencies between continents?
NTP was my first idea ("Internet clock adjustment"), but Fat Tails mentioned that there will be still an 100ms latency. But i think if i use the nearest NTP server the time is pretty precise. I don't know if the protocol implementation take the trip time to and from the NTP Server into account. If yes, everything is fine - i will check that too. And of course i will be utilizing SAM's systems for that test. For my strategy controlling a latency of 100-500 ms is sufficient, the real strategy execution will be aligned to latency vs. costs considerations. So for the running systems there will also be SAM in my boat.
But i'm really surprised that no one has some valid, proofable statistics about that topic. As an european trader i want to know if the data feed currently lagging 10, 100, 1000 or 30000 MS behind the exchange time. It's an relevant information, because if the feed lags, my system lags and i don't trade until the lag is gone. Many threads at BigMike are talking about high latency, but now longer term statistics can be found.
Koepisch
Can you help answer these questions from other members on NexusFi?
I strongly disagree on 100ms latency with NTP. Perhaps he was talking about Windows Time, which is not very efficient and loosely based on NTP.
NTP can easily obtain 1ms precision, usually in fact much less.
The reason most people don't have the info you are requesting is because most people on the forum are using retail platforms which don't provide that info. And if you are working with an API it is trivial to compute the latency, no real need for a discussion I don't think on "how" to do it.
As for a discussion on the benefits of lower latency, I think it's been discussed many times. It's all relevant to your application. If you aren't already co-located at Aurora then I don't believe you are anywhere near the "serious" level of most people that want to get into specifics about microsecond (or better) latency like @artemiso. Just guessing of course.
For the "professional" home trader concerned about latency, at the very least you would use a Chicago based server, like at the Equinix data center at 350 E Cermak. It is relatively cheap. As mentioned previously @sam028 offers a few options of VPS or dedicated at that location for reasonable price, or you could go direct to Steadfast if you have the manpower to manage everything.
I have several servers at Equinix. I do not care about gaining that last mile 1ms advantage and therefore do not spend the extra money to be at Aurora. As I spend more and more time developing my custom platform, that may change and it's a simple cost/benefit analysis. There are some latency and real world $$ benefits of being inside that 1ms barrier even for non-HFT, but how much of a benefit will again depend on your strategy.
If you are referring to post #5 in this thread, the 100ms he is talking about is the speed of light delay between your location and Chicago. Nothing to do with NTP.
NTP will use server pools nearby your physical location (determined based on ping response) automatically, and it also automatically prioritizes based on the quality of those clock sources (ie GPS or etc) from those servers.
I think you misunderstood @Fat Tails comment, an NTP synchronized clock will be accurate with a few ms precision, far below 100ms. And whatever the NTP servers are, that the magic of NTP .
Depending on the PC hardware clock, the drift can be high, some chips are more reliable than other. There are alos few tricks to know: CPU with power-efficiency frequency adjustment will be less precise than CPU running always at 100% of their nominal frequency.
Virtual Machines are also less precise, as the physical CPUs are shared between multiple VMs. On my VPS the NTP sync is done every minute, which is not useful on a physical server which an accurate clock.
I think a simple way would be to have a NinjaTrader strategy, comparing using OnMarketData(), and compare times.
That article also mentions PTP (Precision Time Protocol) as an alternative to NTP.
Anyway I fear I may have got off topic with your thread. The easiest answer to your question remains to simply use a data feed supplied exchange time marker and compare it to local time.
You should remove NT7 from the equation as it doesn't support better than 1 second resolution anyway.
The easiest answer is to just use ping. Ping your broker/data feed. That will probably get you 98% there in terms of latency and takes about 1 second of your time to implement it.
This would work best in the case of a broker supplied data feed and not IQFeed which is in a very different physical location to CME, whereas most brokers are co-located next to CME. IQFeed is not a latency sensitive feed but instead is focused on accuracy, whereas broker feeds are focused on latency and thus give up accuracy (such as dropping L2 events not important to execution) to get there.
@Koepisch: It is not possible to synchronize your PC to the level of milliseconds, because the data exchange and transmission itself takes several milliseconds. There are a few problems you need to be aware of if you wish to compare the time stamps of the data feed with your PC time.
Reference time being used - UTC, GPS or TAI
There are three available reference times, UTC, GPS and TAI (Temps Atomique International). TAI is about 19 seconds (=19000 milliseconds) ahead of GPS and GPS is about 16 sec (16000 milliseconds) ahead of UTC. This suggests that you better use the same reference time, which is used by exchange and data provider, if you want to find out about a lag which is less than 16000 milliseconds.
I think that the CME trade engine is synchronized to NTP, an interesting discussion of time-stamps can be found here:
The correct synchronization time is NTP. To avoid any issues with lag of the time signal, you should use
- the nearest available NTP time server (which is PTB)
- not connect to that time server via internet (approx. lag 25 msec) but via radio clock receiver (approx. lag 3 msec)
There are some products available from Meinberg that can be connected via USB and that will allow accurate synchronization of your PC. I do not know the accuracy level though.
You can now compare the time stamps of the data feed to the synchronized time of your PC. The result will still suffer from a few errors
-> due to the lag of the radio signal from the PTB atomic clock
-> due to data processing
-> due to inaccurate synchronization of your PC
-> due inaccurate synchronization of the internal clock of the CME maching engine
However, these errors should only add up to something like 10 - 20 msec, which is good enough for monitoring the data feed lag, which is about 50 msec (100 msec referred to the round trip) or larger. You should even be able to show the current lag on your NinjaTrader tick-built chart as an indicator.
I am not one to usually disagree with @Fat Tails... but...
1) Yes you can measure latency with millisecond and even microsecond accuracy. It is trivial when using any API that sends the Exchange timestamp. The difference between your PC time and the Exchange time is your latency, or how old that update is when you receive it. The transmission time does not distort the results, it will simply be part of the results.
2) If you are in a data center, you will find NTP <1ms away. If you aren't in a data center, why bother with latency at all?
3) You don't need a hardware device when in a data center. Steadfast already has an NTP server and there are many others in Chicago that do as well. NTP will sync with any of them and the precision can be measured as less than 1ms.
Pretty much all of this is irrelevant, I think the simplest answer to the original question is just to use ping.
Output from ntpq peers, using Steadfast NTP, from one of my Equinix datacenter servers:
Shows 189 microseconds and 310 microseconds respectively (<1ms), this is round trip time. Also shows accuracy/precision of 6 microseconds and 1 microsecond.
The third entry is another stratum 1 server in Chicago that is public.
.
