Support Board
Date/Time: Sat, 13 Jun 2026 23:18:21 +0000
[Sticky] [Locked] - Information: Data Feed and Connectivity Issues. Week of June 7, 2026
View Count: 38
| [2026-06-13 20:17:12] |
| Sierra_Chart Engineering - Posts: 24017 |
|
We now have information on the connectivity problems, on Friday, June 12 , 2026 affecting Sierra Chart data feeds. And probably this may have been the case on other days as well but that is not clear to us at this time. The first incident we were aware of, occurred around 9:35 AM Eastern time June 12. There may have been problems earlier but we just had not been aware of it. We experienced these connectivity problems ourselves, on completely isolated networks. And with servers that are fully isolated from the Denali Exchange Data Feed, and with servers not used by the Denali Exchange Data Feed. We have our own path of connectivity into our network that uses dedicated routers, and switches, and dedicated communication lines, and dedicated servers. We do not want to say anything more than that. We also lost connectivity through those isolated routes. But we still have enough redundancy, that we were still able to get in any way. Significant loss or total loss of connectivity occurred around 4:40 PM and went on intermittently for about an hour. And thereafter, there were various connectivity issues but much less of a problem. We are not noticing any problems at this time. We were told by one of the major ISPs today, that we use in the data center what the cause of the problem was. We finally got the information today. They were experiencing a distributed denial of service attack and they were taking steps to mitigate the problem. So obviously this was happening during the day. And it intensified after 4:40 PM Eastern time. We also had a problem, earlier in the week briefly affecting Teton order routing. Our only reasonable determination, was that this was also a carrier issue. That service uses Zayo. And there is backup connectivity through Lumen and the backup connectivity worked. So that caused a short interruption of about 30 seconds for connectivity until Sierra Chart could reconnect through our backup connection. That problem, created some condition on the server which then started to lead to packet loss. Our steps initially to solve that problem were not successful and the packet loss was ongoing on one of those servers, not all just one of the servers, throughout the next day, and then we had to restart the server, that evening. Which we did. That server uses Windows. And that is why we have long had a plan to move the data feed processing to Linux and we are going to be accelerating that plan. We have already developed the software for that. We have not been saying too much about the problems this past week, because we have been working on the issues rather than just posting here or responding to user questions. One thing we also just want to briefly say, is that when you have problems, with lagging data or stopping data due to the public Internet, or your local network hardware or your local system there is nothing we can do about that. Inevitably these problems are going to happen from time to time. and these problems can be happening at any point. They are not issues most of the time at our side. This Friday, our CME market data processing was perfect. There was zero packet loss, and there were no problems at all, but we did see some delays on some channels, further down the distribution path, of up to 1 to 2 seconds but this is minor. Since we moved out, a few months back, processing of CME stock index futures onto separate processes, this has helped significantly. One thing that might help users, is for us to reduce the frequency of market depth data updates. We have never done that though. We are aware of competitive data feeds, that throttle the data feed. Sierra Chart does not provide any throttling. Other feeds like CQG (we think), and dxfeed (they openly express this) throttle their data feeds. Sierra Chart does not do that. But we have always throttled back NASDAQ TotalView since the very beginning. There is no other way. We have to increase the throttling rate of market update updates with NASDAQ TotalView. We have to do that. There were other problems, involving our 100 Gb wavelength circuit from Michigan to Chicago this past week. There was a fiber cut, along a railroad, that interrupted that circuit for more than a day. We had to rely on the backup circuit and during that time, we had some packet loss Issues with US equities but that does not cause any obvious problems. That wavelength is used for US equities predominantly. Not CME data. We also had another, background connectivity issue, with the CME, that only lasted 50 seconds, during the CME one hour downtime which we do not want to go into. There was no detrimental effect about that, but it was an incident that got the attention of the CME, and was reported to us. This was related to port reconfiguration to mitigate the packet loss issue. That issue only involved the CME iLink backup server. Not the primary server. So clearly it was not a good week, but we are aware of the issues, and we have taken all of the steps to mitigate the problems. Most likely we will be adding another carrier to our Michigan location. We are considering AT&T. Normally we have relied on the data center to provide carrier diversity and connectivity because they connect into many many carriers something like 8 or more. We have to check on the number. But obviously when there is a denial of service attack, then that service becomes not good. We also had serious problems with NASDAQ TotalView on June 12. We are implementing a solution to that by increasing the throttling, of market depth updates. We have no other choice. But part of that problem was some faulty or unexpected data, at 11:45 AM. There is another thread about this. We do not like to place blame, with all of the above, but none of this, was any problem with Sierra Chart software or systems. But we can do better by moving Multicast data feed processing to Linux. The problem has been with Internet carriers, and fiber carriers, and Microsoft, and Windows. And NASDAQ TotalView architecture. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2026-06-13 21:32:56
|
To post a message in this thread, you need to log in with your Sierra Chart account: