NexusFi: Find Your Edge


Home Menu

 





How to measure data feed latencies between continents?


Discussion in Traders Hideout

Updated
      Top Posters
    1. looks_one Koepisch with 22 posts (5 thanks)
    2. looks_two Big Mike with 18 posts (6 thanks)
    3. looks_3 artemiso with 11 posts (1 thanks)
    4. looks_4 gomi with 7 posts (21 thanks)
      Best Posters
    1. looks_one gomi with 3 thanks per post
    2. looks_two Fat Tails with 3 thanks per post
    3. looks_3 Big Mike with 0.3 thanks per post
    4. looks_4 Koepisch with 0.2 thanks per post
    1. trending_up 28,002 views
    2. thumb_up 48 thanks given
    3. group 8 followers
    1. forum 66 posts
    2. attach_file 15 attachments




 
Search this Thread

How to measure data feed latencies between continents?

  #61 (permalink)
 Koepisch 
@ Germany
 
Experience: Beginner
Platform: NinjaTrader
Broker: Mirus Futures/Zen-Fire
Trading: FDAX
Posts: 569 since Nov 2011
Thanks Given: 440
Thanks Received: 518


gomi View Post
Suppose you have a constant latency of 150 ms.
Then the measured latency should be equally distributed between 0.150 s (ticks happening at mm:ss.000, timestamped mm:ss ), and 1.150 s (ticks happening at mm:ss.99999999999, also timestamped mm:ss).

The idea is you take the histogram of all latencies, and you cumulate it using a sliding window of 1 second length.
So you calculate how much ticks you have between
0 and 1
0.10 and 1.10
0.20 and 1.20
etc....

The window that catches the maximum of ticks will be the one that represents the latency.
Here's a chart, the blue reactangle is where the measured tick latencies are.
Attachment 137538

Thanks, now i've got it!

Started this thread Reply With Quote

Can you help answer these questions
from other members on NexusFi?
Exit Strategy
NinjaTrader
Are there any eval firms that allow you to sink to your …
Traders Hideout
Build trailing stop for micro index(s)
Psychology and Money Management
NexusFi Journal Challenge - April 2024
Feedback and Announcements
Deepmoney LLM
Elite Quantitative GenAI/LLM
 
  #62 (permalink)
 Koepisch 
@ Germany
 
Experience: Beginner
Platform: NinjaTrader
Broker: Mirus Futures/Zen-Fire
Trading: FDAX
Posts: 569 since Nov 2011
Thanks Given: 440
Thanks Received: 518

@gomi: You wrote "I you choose to go with the hardware way, it's better to use a legacy serial port, which has a direct IRQ access to the CPU, than going through the full serial over usb stack."

My current implementation idea was to send one UDP packet per second. I want to avoid to utilize a new serial port driver (did you need one?) and i think the network stack adds only very little latency. Was this an option within your solution? I can attach my stratum source to the serial port too, but i don't know if this is a legacy port (have to look into the motherboard spec first). I have more knowledge to handle IP data then serial port access.

Started this thread Reply With Quote
  #63 (permalink)
 gomi 
Paris
Market Wizard
 
Experience: None
Platform: NinjaTrader
Posts: 1,270 since Oct 2009
Thanks Given: 282
Thanks Received: 4,505


If I understand you will be building a dedicated NTP server on PI box ?
In this case, you will need some way to discipline your local PC clock (make it slew faster/slower)

Since you're on a LAN (constant network latency), I would leave this to a local NTP client..

Reply With Quote
  #64 (permalink)
 Koepisch 
@ Germany
 
Experience: Beginner
Platform: NinjaTrader
Broker: Mirus Futures/Zen-Fire
Trading: FDAX
Posts: 569 since Nov 2011
Thanks Given: 440
Thanks Received: 518


gomi View Post
If I understand you will be building a dedicated NTP server on PI box ?
In this case, you will need some way to discipline your local PC clock (make it slew faster/slower)

Since you're on a LAN (constant network latency), I would leave this to a local NTP client..

No not anymore. NTP is out of the game. I can achieve much more precision with doing it myself and i can keep control. Adjusting the timer frequency is one part of the game. I want to send a UDP packet (not NTP) to my PC and the (my) client handles the windows time sync (like yours) OR use the time information directly.

Using the PC serial port instead of using ethernet can reduce the hardware design a lot. The PI box is only for fast development. Later i can attach a small atmel avr. That's why i asked about the serial port.

Started this thread Reply With Quote
  #65 (permalink)
 gomi 
Paris
Market Wizard
 
Experience: None
Platform: NinjaTrader
Posts: 1,270 since Oct 2009
Thanks Given: 282
Thanks Received: 4,505

I didn't really think to use a network stack as my GPS card was serial ;-)

Using a serial port is easy, you just set up a mask
SetCommMask(hPort, EV_RLSD | EV_ERR | EV_RXCHAR);

An wait for the interrupt
WaitCommEvent(hPort, &dwCommModemStatus, 0);

FYI here's my code, you need Windows 8 because of GetSystemTimePreciseAsFileTime
But you can replace with GetSystemTimeAsFileTime if needed to compile on pre-Win8.

I also have some Win 7 code that will do interpolation with performance counters if you need.


Code will set thread affinity on one core only and set real time priority.

Attached Files
Elite Membership required to download: main.cpp
Reply With Quote
  #66 (permalink)
 Koepisch 
@ Germany
 
Experience: Beginner
Platform: NinjaTrader
Broker: Mirus Futures/Zen-Fire
Trading: FDAX
Posts: 569 since Nov 2011
Thanks Given: 440
Thanks Received: 518


gomi View Post
I didn't really think to use a network stack as my GPS card was serial ;-)

Using a serial port is easy, you just set up a mask
SetCommMask(hPort, EV_RLSD | EV_ERR | EV_RXCHAR);

An wait for the interrupt
WaitCommEvent(hPort, &dwCommModemStatus, 0);

FYI here's my code, you need Windows 8 because of GetSystemTimePreciseAsFileTime
But you can replace with GetSystemTimeAsFileTime if needed to compile on pre-Win8.

I also have some Win 7 code that will do interpolation with performance counters if you need.


Code will set thread affinity on one core only and set real time priority.

Thanks a lot gomi. You can help me out with the Win7 code, if you are willing to share it. Now it's family time, so i will look into the code at later time. But i'm looking forward to do it. I think we had the same needs and it resolves in the same solution. Awesome!

Started this thread Reply With Quote
  #67 (permalink)
 gomi 
Paris
Market Wizard
 
Experience: None
Platform: NinjaTrader
Posts: 1,270 since Oct 2009
Thanks Given: 282
Thanks Received: 4,505

Here's the Win7 code. To make it work with good precision, the code will compute a corrective scale factor for performance counter frequency, using an EMA(100) of measured values.

Be prepared to see some ugly stuff, never planned to make it public ;-)

Attached Files
Elite Membership required to download: main.cpp
Reply With Quote




Last Updated on February 17, 2014


© 2024 NexusFi™, s.a., All Rights Reserved.
Av Ricardo J. Alfaro, Century Tower, Panama City, Panama, Ph: +507 833-9432 (Panama and Intl), +1 888-312-3001 (USA and Canada)
All information is for educational use only and is not investment advice. There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
About Us - Contact Us - Site Rules, Acceptable Use, and Terms and Conditions - Privacy Policy - Downloads - Top
no new posts