Support Board
Date/Time: Sun, 08 Feb 2026 23:20:41 +0000
Post From: Notice: CME Denali Data Feed Intermittent Data Delay January 26-February 6, 2026
| [2026-02-08 03:00:45] |
| Sierra_Chart Engineering - Posts: 22739 |
|
This thread is to explain, the intermittent, data delay issues, during the time range, in the subject line. You may or may not have noticed, any delays during this timeframe and we are covering a larger timeframe, than probably the delays occurred in but just to be comprehensive. We want to, preface our comments, that we do not know definitively why this problem occurred. We have not had a problem specifically like this previously, at least not in the last 10 years or so. Although we did have a problem like this with EUREX about a month ago. The cause is probably the same. Update: The problem with EUREX about a month ago actually was different but probably had the same underlying cause. The data feed would pause for about three seconds and then resume. Not a significant lag like a one minute lag. It would just be pausing every 3 to 4 seconds. And then, the data would be transmitted/received, instantly thereafter. This was due to a high processing load, due to apparent inefficient processing of the data. While users, from time to time, here and there, for different reasons may notice delay data issues, the cause of those more often than not , is related to Internet connectivity. Our real-time data distribution has been greatly improved upon over the years. While there could be a problem there, that is not our focus at this time. We do not believe, that is the source of any problem at this time. The most recent issue is not due to Internet connectivity. We thought it was on our side, but that is unlikely. In general, we understand that the issue, is that with a large amount of data that needs to be processed, from the exchange feeds, at some point, there is some inefficiency with the processing of that data. The problem is tentatively resolved, because in general we understand, where there are issues and what can be done to resolve this. Potential reasons for the problem: An operating system update, which is causing some performance issue which gets worse over time within the process. This is not likely but we have to consider it. Changing from a map to an unordered map for one of the primary lookup containers, in the core, market data manager of the data feed process. Although, unordered maps, also known as hash maps, have been used, from the very beginning to process MBO data. So we have never been aware of any problem with that. Although, CME market data processing does not use this particular map. It uses a combination of a vector, and multiple STL maps (one for each channel). Our data feed handlers processing CME data, use multiple threads but for CME channels processed within the same process, there is a single critical section, which protects access to symbol data objects across threads. We are going to change this to a separate critical section for each symbol. The reason we have to do this is because although there are multiple threads, one for each multicast channel processing the channel, there is a lot of contention between them, when updating the data for a particular symbol. We have known about this issue for a long time, but it is something we just have not spent any time making changes to it since there really has not been any obvious or measurable, impact from it. But we will be doing it, over the next 30 days. A similar contention is also a problem after the data feed is normalized and placed into a single socket stream. There is a way to mitigate this issue and that has been done. Another potential problem is that this channel, we have noticed packet loss on it for the last two weeks: Exchange CME. Channel 312 -> CME Globex Interest Rate Futures Packet loss with CME data is an issue that we rarely have and when it does occur it is extremely minor. We wonder, if there is more data on that channel as compared to in the past. We have separated out, some CME channels, into separate processes, this is going to be a huge performance improvement for the time being. It is specifically the equity index channels which are now running on their own dedicated processes. So what we describe above regarding a critical section is effectively already applied in these cases at this time. The most likely problem is that the processes processing CME data run for a long time, weeks and months at a time and the STL containers that we use, are not actually getting cleared, at every new connection to the data feed. They should be but technically they probably are not. For the CME feeds, the connections occur once a week. For example a vector does not actually release memory when we clear it: https://www.tutorialpedia.org/blog/force-a-std-vector-to-free-its-memory/ We believe, that this is actually the core reason for the problem and why a process restart, is what actually needs to be done. We are now doing automatic weekly process restarts. So we think, the problem is an accumulation of worsening performance, over time, due to the behavior of STL containers. This is our tentative notes at this time. Anyway, we expect everything to be fine, this coming week and the following weeks, because we have done enough, and will be doing more, to manage the issue. In the end, we will have greatly improved data feed processing related to performance within Sierra Chart. This also affects the Sierra Chart program that you use, for the better because the same program is used to process the CME feeds. We expect complete and full resolution, over the next 30 days. During this time, we do not expect anyone to observe a further issue. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2026-02-08 17:41:18
|