Support Board
Support Board
Search Board
Control Panel
View My Posts / Threads
Direct MessagesDate/Time: Sat, 27 Apr 2024 05:08:42 +0000
[Locked] - Information: Denali Data Feed Lagging/Stopping Issues for Some Users
View Count: 2037
| [2020-09-08 17:49:41] |
| |
|
We have had a few reports recently of users reporting the Denali data feed lags or stops for them in particular around the market open. We have believed, that these are public Internet related issues. Once the public Internet is involved, there can be all kinds of problems. And when the market activity is very high at the open, there is a lot of data to be transmitted, and some amount of lag can be expected. We have been looking over these issues carefully and seeing what performance improvements can be made, and making those improvements, and analyzing the source of the problem. What we have determined, is that recent performance improvements are working well but not essential. There will still be more performance improvements to be made, but at best, these are reducing latency at the microsecond/small millisecond level which would go unnoticed by users. Most importantly, there is no lag or delay or any stopping issues coming out of the Denali data feed servers. None at all. These servers are very high-performance, and have a lot of excess capacity. Lagging data, and stopping data it is the result of public Internet connectivity. If the data is feed stops, it is the TCP protocol at the operating system level, which is stopping the transmission of data on specific connections due to remote acknowledgments not being received timely. This also can result in a lag because the TCP protocol will start to slow transmission of data. You can actually monitor the data related to this by referring to the information here: Prices / Data Falling Behind: 4.16 - Monitoring Sierra Chart Data Feed Performance from Perspective of Server (4.16) We are going to add a new setting to allow users to control the buffer timing. We will have controllable between 20 ms to 2000 seconds. The fewer packets that go out and the fewer remote acknowledgments that are needed, should reduce lagging/stopping we would hope. We will need more feedback on this. We will try to get this out before the end of the week. The other possibility is for us to compress the real-time data in some way. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2020-09-09 05:36:09
|
| [2020-09-09 08:42:16] |
| nosast - Posts: 290 |
|
Could it be that this issue is somehow related to the ms-timestamping? I never had any issues with Denali before upgrading to the new version. Even in very high vola markets (like March 20) the feed was rock solid and not a single lag occurred. Notable: The current issue was only visible right at the open and the feed stalled for 10-15 seconds while Bookmap with Rithmic was up-to-date. Also I did see this lag last week a few times but not on Tuesday this week. |
| [2020-09-09 13:32:11] |
| nosast - Posts: 290 |
|
Again today. No data right at RTH open for 10-15 seconds. All other feeds where running without any lag.
|
| [2020-09-09 14:18:36] |
| |
|
Millisecond time stamping will use higher bandwidth. So yes that is definitely going to contribute to it in a small way. This issue absolutely has to do with TCP flow control. It is not a problem with the Denali feed in of itself. There simply is no lag at all with the data feed itself at the source or from remote locations, with reliable two-way connectivity. We have verified that. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2020-09-09 14:20:32
|
| [2020-09-09 14:54:42] |
| nosast - Posts: 290 |
|
How to deactivate ms timestamping? Want to see if that helps. Maybe the issue is at the carrier side distributing the feed in times of high volume burst. But why is it then only ocurring right at the open and not intraday when market gets fast? What would you suggest to get rid of this issue? Running the feed on UDP instead of TCP? Date Time Of Last Edit: 2020-09-09 15:16:51
|
| [2020-09-10 23:25:47] |
| |
|
We have to add an option to disable millisecond time stamping. We will do that. But really we have to look at a more comprehensive fast compression method. UDP At a low level would be the best solution and using an alternative to TCP so there are no missed packets. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing |
| [2020-09-11 00:33:11] |
| ertrader - Posts: 644 |
|
I'm on v 2146 and this also happened to me today. It has occurred a couple other times in the last week and I will watch closer. Also, I use Wine/Linux with both Denali and Rithmic on one computer with two instances of SC. Disconnecting and reconnecting Denali worked to restore the connection. Interestingly, Rithmic kept going real-time while Denali stopped. Date Time Of Last Edit: 2020-09-11 00:33:41
|
| [2020-09-11 01:29:39] |
| User100912 - Posts: 76 |
|
I'd report again but I don't have any other info to add from the heartbeat log. I can confirm it's still an issue and I'll need to change feeds for October if it isn't resolved. Can't have a trading feed which repeatedly locks up, sorry. You guys do great engineering which is appreciated, so I hope this is resolved soon. |
| [2020-09-11 02:17:08] |
| User753428 - Posts: 158 |
|
just wanted to add a datapoint b/c I had these same problems with denali all the way since March. these were support posts created by me back in March and May: denali feed froze for more than 10 seconds at 9:30 est market open today feeds stops for 3-5 seconds multiple times during the day the symptoms i describe exactly match those mentioned recently (delay -> catches up to recent price action via superspeed -> normalizes to streaming real-time data at 1x speed) which suggests the underlying cause may be similar. for whatever reason, these never got resolved (i guess it wasn't affecting a high enough % of denali users to warrant serious attention at the time) and i had to switch back to cqg. i love sierracharts b/c you guys do great engineering but any kind of datafeed lag or freezing, no matter how small, cannot be tolerated for daytraders. i wish you guys luck in finding a solution b/c i would love to go back to the denali feed and use Sierrachart's TT order routing one day. |
| [2020-09-11 05:11:12] |
| |
|
For users who are having an issue, these are connection specific. It is not a performance issue on the server. We think in your case what is happening is that the TCP implementation at the operating system level is failing to get the acknowledgments of received packets from your side timely and it stops the transmission of data. There could be other causes as well. We have made a change which we think will help and we will want to know how the data feed is working as of this post. We do not want to say what we did at this time, we just want to see if there is an improvement. More information here: https://en.wikipedia.org/wiki/Transmission_Control_Protocol#Flow_control Also we have never noticed any stopping of the data feed in our testing. We have notice a delay of a few seconds at the open depending upon the monitoring location and quality of connectivity. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2020-09-11 05:11:34
|
| [2020-09-11 05:12:49] |
| |
|
but any kind of datafeed lag or freezing, no matter how small, cannot be tolerated for daytraders. Therefore, you must therefore get a server in close proximity to our servers to avoid this issue consistently. And use a remote desktop connection to it. That is the only consistent definitive solution especially when dealing with high data rates. Otherwise you are vulnerable to issues on the public Internet or your own networking.In the case of the Denali data feed, that would be within the Chicago area. You can use a server within the Aurora data center or Equinix Cermak center. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2020-09-11 05:13:51
|
| [2020-09-11 05:17:34] |
| User753428 - Posts: 158 |
|
Therefore, you must therefore get a server in close proximity to our servers to avoid this issue consistently. And use a remote desktop connection to it. That is the only consistent definitive solution especially when dealing with high data rates. Otherwise you are vulnerable to issues on the public Internet or your own networking.
That would be the case if not for the fact that I have never experienced these issues with CQG and Rithmic. I have mentioned often previously that if this was simply a problem with my public internet or networking, I'd be experiencing similar problems with other datafeeds too. But it only happens with Denali. |
| [2020-09-11 06:07:59] |
| |
|
That would be the case if not for the fact that I have never experienced these issues with CQG and Rithmic. This is not believable. We have plenty of reports, overwhelmingly numerous, of issues like this, with these services as well. To say that you never have a problem with these services when it comes to lagging data or disconnects (which results in stopping data) or stopping data is nonsense.You are making yourself very clear here: but any kind of datafeed lag or freezing, no matter how small, cannot be tolerated for daytraders.
If you want absolute perfection, you do need a connection in close proximity.And we are being very clear in return. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2020-09-11 06:32:14
|
| [2020-09-11 07:13:03] |
| User753428 - Posts: 158 |
|
To say that you never have a problem with these services when it comes to lagging data or disconnects (which results in stopping data) or stopping data is nonsense.
If you put it that way, then sure. No feed is perfect and there will always be unforeseeable problems with any feed. The question is whether these problems happen too frequently enough to pose repetitive disruption. Once or twice a year, when there's breaking news and volume explodes in a single candle, I've experienced a 2-3 second delay with CQG. I've once experienced the CQG singapore feed server go down and had to reroute to the US feed server. With Rithmic I've never had any problems besides those related to your bridge program with Rithmic but that's irrelevant here. In all of these cases, 1) the cause was evident 2) it happened so infrequently or was an one-off event 3) they had nothing to do with my personal internet connection. The lag and freezing we've experienced with Denali is nothing like these cases: 1) cause isn't evident and you simply blame us for having poor internet 2) it happens daily or weekly, sometimes even multiple times a day 3) the manner in which the lag and freezing occurs is very clearly different than that experienced (if at all) with other data/trading services. how is it different? 1) delay -> catches up to recent price action via superspeed -> normalizes to streaming real-time data at 1x speed 2) it tends to occur right at cash market open consistently 3) there is no disconnection with the data/trade service (vs problems with CQG/Rithmic feed have usually resulted in disconnection). We have plenty of reports, overwhelmingly numerous, of issues like this, with these services as well.
You seem to think lag/freezing is common as well with other services like CQG/Rithmic but I can assure you that's b/c you're constantly responding to support requests. given the sheer number of CQG/Rithmic users, the % of actual lag/freezing experiences is much lower than you think. If you want absolute perfection, you do need a connection in close proximity.
That is true. What I should have said is "any kind of datafeed lag or freezing, within the tolerances acceptable to a discretionary futures daytrader." Most of us aren't trading 100% algorithmically through a co-located server next to the exchange. As discretionary retail traders, we just need a datafeed whose lag is marginally unnoticeable to the naked eye. SierraChart is extremely proud of their work and for good reason. But for that same reason, you guys are frequently too defensive. Look at my old posts bringing up these problems in March/May. I remember very clearly of other posts posting similar issues with the Denali Feed back around when I posted them. But back then, you made it a point to express that it was OUR fault, our internet's fault. Well, fast forward 3-5 months now, there are clearly more and more people experiencing the same exact problem. Are you going to continue doubling down and blaming users for having bad internet or are you going to do something about it? |
| [2020-09-11 17:43:46] |
| User100912 - Posts: 76 |
|
Problems again today, reporting as it isn't the open or a particularly active time. SC Futures Order Routing/Data | Heartbeat from server | ServerReceivedClientHeartbeatSecondsAgo=17, NumberOfOutstandingSendBuffers=0, TransmissionDelayInMilliseconds=0, ServerSendBufferSizeInBytes=0, ActualMessageDelay=1.7 seconds | 2020-09-11 12:41:15.904
SC Data - All Services | Heartbeat from server | ServerReceivedClientHeartbeatSecondsAgo=8, NumberOfOutstandingSendBuffers=0, TransmissionDelayInMilliseconds=0, ServerSendBufferSizeInBytes=0, ActualMessageDelay=0.8 seconds | 2020-09-11 12:41:20.510 SC DTC Data with Market Depth | Heartbeat from server | ServerReceivedClientHeartbeatSecondsAgo=2, NumberOfOutstandingSendBuffers=0, TransmissionDelayInMilliseconds=0, ServerSendBufferSizeInBytes=0, ActualMessageDelay=1.3 seconds | 2020-09-11 12:41:23.487 SC Data - All Services | Heartbeat from server | ServerReceivedClientHeartbeatSecondsAgo=0, NumberOfOutstandingSendBuffers=0, TransmissionDelayInMilliseconds=0, ServerSendBufferSizeInBytes=3936, ActualMessageDelay=1.7 seconds | 2020-09-11 12:41:28.419 SC Futures Order Routing/Data | Heartbeat from server | ServerReceivedClientHeartbeatSecondsAgo=16, NumberOfOutstandingSendBuffers=0, TransmissionDelayInMilliseconds=0, ServerSendBufferSizeInBytes=0, ActualMessageDelay=1.7 seconds | 2020-09-11 12:41:35.892 SC Data - All Services | Heartbeat from server | ServerReceivedClientHeartbeatSecondsAgo=8, NumberOfOutstandingSendBuffers=1, TransmissionDelayInMilliseconds=15, ServerSendBufferSizeInBytes=29, ActualMessageDelay=0.8 seconds | 2020-09-11 12:41:40.551 SC DTC Data with Market Depth | Heartbeat from server | ServerReceivedClientHeartbeatSecondsAgo=2, NumberOfOutstandingSendBuffers=0, TransmissionDelayInMilliseconds=0, ServerSendBufferSizeInBytes=0, ActualMessageDelay=1.3 seconds | 2020-09-11 12:41:43.472 SC Data - All Services | Heartbeat from server | ServerReceivedClientHeartbeatSecondsAgo=0, NumberOfOutstandingSendBuffers=1, TransmissionDelayInMilliseconds=25, ServerSendBufferSizeInBytes=9393, ActualMessageDelay=1.7 seconds | 2020-09-11 12:41:48.446 SC Futures Order Routing/Data | Heartbeat from server | ServerReceivedClientHeartbeatSecondsAgo=15, NumberOfOutstandingSendBuffers=0, TransmissionDelayInMilliseconds=0, ServerSendBufferSizeInBytes=0, ActualMessageDelay=1.7 seconds | 2020-09-11 12:41:55.889 SC Data - All Services | Heartbeat from server | ServerReceivedClientHeartbeatSecondsAgo=8, NumberOfOutstandingSendBuffers=3, TransmissionDelayInMilliseconds=45, ServerSendBufferSizeInBytes=1106, ActualMessageDelay=0.9 seconds | 2020-09-11 12:42:00.557 SC DTC Data with Market Depth | Heartbeat from server | ServerReceivedClientHeartbeatSecondsAgo=1, NumberOfOutstandingSendBuffers=1, TransmissionDelayInMilliseconds=15, ServerSendBufferSizeInBytes=32, ActualMessageDelay=1.3 seconds | 2020-09-11 12:42:03.477 SC Data - All Services | Heartbeat from server | ServerReceivedClientHeartbeatSecondsAgo=0, NumberOfOutstandingSendBuffers=1, TransmissionDelayInMilliseconds=25, ServerSendBufferSizeInBytes=10003, ActualMessageDelay=1.7 seconds | 2020-09-11 12:42:08.432 Updating to say that I'm getting the issue on an EC2 box (Oregon). Haven't tried colocating at Aurora yet. Has anyone tested that? SC Data - All Services | Heartbeat from server | ServerReceivedClientHeartbeatSecondsAgo=18, NumberOfOutstandingSendBuffers=0, TransmissionDelayInMilliseconds=0, ServerSendBufferSizeInBytes=4471, ActualMessageDelay=0.6 seconds | 2020-09-11 12:55:14.783
Date Time Of Last Edit: 2020-09-11 17:58:26
|
| [2020-09-11 18:48:20] |
| User220914 - Posts: 239 |
|
i thiink this is a great topic and needs to be addressed. how can we test actual price from the server(denali) to the price on our computers? i dont mean the ping i mean the actual delay from the server to it hitting our computer and being drawn. We can see how long it takes in millseconds to build indicators and studies why dont we have a built in feed delay on the system this way we can see in real time if there is an issue and quite possibly set an alert based on a user or sierra chart threshold. The problem i am seeing right now is can i TRUST this FEED anymore? |
| [2020-09-11 19:18:58] |
| |
|
Are you going to continue doubling down and blaming users for having bad internet or are you going to do something about it? Absolutely because this is a basic verified fact. There are things that we can do that can help to some extent and we are working on them but the simple fact is that the Denali feed is going to transmit more data than CQG with the extended depth. And we have verified that these are user specific connectivity issues.We monitor the Denali data feed from the other side of the world and we never notice the feed ever stop. Never. Unless there is a complete loss of connectivity but that is not common and that is due to connectivity issues. You also have to realize there are so many variables between data feeds including the particular ports being used the data being transmitted and how the networks are handling that. Take for example this case from Sweden: Stalling charts They claim no other trading platform has a problem and somehow this is a problem with the Sierra Chart Delayed Exchange Data Feed. This is complete nonsense. There is no problem like this the Delayed Exchange Data Feed but this is due to a connectivity issue. That is very clear. Unless there is something else going on that we are not aware of. But it is not a problem with the data feed at the source. So yes we are going to double down, triple down, quadruple down. As many times as we need but at the same time we are looking to see what we can do to help with these issues and we are currently doing that and making some improvements. Here is another example, of an issue reported that is not related to the data feed itself: https://www.sierrachart.com/SupportBoard.php?ThreadID=56265 Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2020-09-11 19:39:23
|
| [2020-09-11 20:14:01] |
| |
|
In our discussion today with our infrastructure provider, they said that there were hundreds of pairs of fiber cut the prior Thursday near a railroad somewhere in the East Coast which was affecting connectivity between Chicago and New York increasing latency. This only got fixed today. Depending upon your location if your connectivity was affected by this, this would increase latency. This would affect users in the east of the US. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing |
| [2020-09-11 20:21:06] |
| |
|
Post #15 claiming problems again today, you are noticing a slight delay not only with the Denali data feed, but our other feeds from our other services. Once again this is an Internet connectivity issue. 1 to 1.5 seconds delay at times is not serious. If you want to solve the problem you need to get a server somewhere preferably within the Chicago area and use a Remote Desktop connection to it. We can help you with that. Just start a new thread that you want assistance with a remote server to run Sierra Chart. This is why we have locked this thread because users are jumping onto any little problem and saying it is a problem with our data feed. Complete nonsense. If you have a problem with the public Internet, there are solutions to bypass it. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2020-09-11 20:30:10
|
| [2020-09-11 20:27:21] |
| |
|
Prior post has been corrected.
Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing |
To post a message in this thread, you need to log in with your Sierra Chart account: