Home >> Support Board

Support Board

Date/Time: Sat, 28 Mar 2026 14:47:13 +0000

How to Post a Message for Support

[Locked] - Notice: CME Denali Data Feed Intermittent Data Delay January 26-February 6, 2026

View Count: 1175

[2026-02-08 03:00:45] #1 [Go To First Post]
Sierra_Chart Engineering - Posts: 23359
This thread is to explain, the intermittent, data delay issues, during the time range, in the subject line. You may or may not have noticed, any delays during this timeframe and we are covering a larger timeframe, than probably the delays occurred in but just to be comprehensive. We want to, preface our comments, that we do not know definitively why this problem occurred. We have not had a problem specifically like this previously, at least not in the last 10 years or so. Although we did have a problem like this with EUREX about a month ago. The cause is probably the same. Update: The problem with EUREX about a month ago actually was different but probably had the same underlying cause. The data feed would pause for about three seconds and then resume. Not a significant lag like a one minute lag. It would just be pausing every 3 to 4 seconds. And then, the data would be transmitted/received, instantly thereafter. This was due to a high processing load, due to apparent inefficient processing of the data. While users, from time to time, here and there, for different reasons may notice delay data issues, the cause of those more often than not , is related to Internet connectivity. Our real-time data distribution has been greatly improved upon over the years. While there could be a problem there, that is not our focus at this time. We do not believe, that is the source of any problem at this time. The most recent issue is not due to Internet connectivity. We thought it was on our side, but that is unlikely. In general, we understand that the issue, is that with a large amount of data that needs to be processed, from the exchange feeds, at some point, there is some inefficiency with the processing of that data. The problem is tentatively resolved, because in general we understand, where there are issues and what can be done to resolve this. Potential reasons for the problem: An operating system update, which is causing some performance issue which gets worse over time within the process. This is not likely but we have to consider it. Changing from a map to an unordered map for one of the primary lookup containers, in the core, market data manager of the data feed process. Although, unordered maps, also known as hash maps, have been used, from the very beginning to process MBO data. So we have never been aware of any problem with that. Although, CME market data processing does not use this particular map. It uses a combination of a vector, and multiple STL maps (one for each channel). Our data feed handlers processing CME data, use multiple threads but for CME channels processed within the same process, there is a single critical section, which protects access to symbol data objects across threads. We are going to change this to a separate critical section for each symbol. The reason we have to do this is because although there are multiple threads, one for each multicast channel processing the channel, there is a lot of contention between them, when updating the data for a particular symbol. We have known about this issue for a long time, but it is something we just have not spent any time making changes to it since there really has not been any obvious or measurable, impact from it. But we will be doing it, over the next 30 days. A similar contention is also a problem after the data feed is normalized and placed into a single socket stream. There is a way to mitigate this issue and that has been done. Another potential problem is that this channel, we have noticed packet loss on it for the last two weeks: Exchange CME. Channel 312 -> CME Globex Interest Rate Futures Packet loss with CME data is an issue that we rarely have and when it does occur it is extremely minor. We wonder, if there is more data on that channel as compared to in the past. We have separated out, some CME channels, into separate processes, this is going to be a huge performance improvement for the time being. It is specifically the equity index channels which are now running on their own dedicated processes. So what we describe above regarding a critical section is effectively already applied in these cases at this time. The most likely problem is that the processes processing CME data run for a long time, weeks and months at a time and the STL containers that we use, are not actually getting cleared, at every new connection to the data feed. They should be but technically they probably are not. For the CME feeds, the connections occur once a week. For example a vector does not actually release memory when we clear it: https://www.tutorialpedia.org/blog/force-a-std-vector-to-free-its-memory/ We believe, that this is actually the core reason for the problem and why a process restart, is what actually needs to be done. We are now doing automatic weekly process restarts. So we think, the problem is an accumulation of worsening performance, over time, due to the behavior of STL containers. This is our tentative notes at this time. Anyway, we expect everything to be fine, this coming week and the following weeks, because we have done enough, and will be doing more, to manage the issue. In the end, we will have greatly improved data feed processing related to performance within Sierra Chart. This also affects the Sierra Chart program that you use, for the better because the same program is used to process the CME feeds. We expect complete and full resolution, over the next 30 days. During this time, we do not expect anyone to observe a further issue. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2026-02-08 17:41:18

[2026-02-08 05:14:56] #2 [Go To First Post]
Sierra_Chart Engineering - Posts: 23359
We have now implemented a solution for this but it is not yet deployed: The most likely problem is that the processes processing CME data run for a long time, weeks and months at a time and the STL containers that we use, are not actually getting cleared, Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing

[2026-02-08 19:04:53] #4 [Go To First Post]
Sierra_Chart Engineering - Posts: 23359
One thing we want to point out, is that when Sierra Chart is being used by users, normally market data is received by the underlying DTC Service, and the lookup of what we call a symbol data object as market data is received is through a vector and an index. It is extremely efficient. In the case of CME market data processing and other exchange feeds, it is not that simple because the symbol identifiers, are usually large integers. They are not one based, incrementing integer values. They are more like a security identifier. This is the why a STL map is being used for symbol data lookups. In the case of CME it was a design way back in 2013, where we use multiple maps, one for each channel. So there is much smaller containers and the lookups are fast. We are going to be doing some testing, to see how to optimize this further if we can. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2026-02-08 19:07:11

[2026-02-08 20:28:31] #5 [Go To First Post]
Sierra_Chart Engineering - Posts: 23359
We are going to be replacing, the use of STL maps in the core market data manager of Sierra Chart, with vectors, using sorted integer symbol identifiers, and a binary search. Our DTC based services do not use this method. They use a very efficient method of a vector and a offset. This is already extremely efficient. What we describe in the first paragraph is the same concept that is used in the: c_VAPContainer Which is used for Volume at Price data. This will be a major performance improvement, with our server side market data processing and eliminate this problem assuming the core problem does relate to these maps and we think that likely is the case. In any case, with weekly restarts of the process, to guarantee memory releasing and get a fresh memory layout, is a good idea, and also separating out more CME channels into separate processes is all good as well. The other thing we want to mention is that Sierra Chart is not managed code like .net. It is native C++ code and memory management is very good and performed ourselves. When memory is not used, it is released immediately. We recognize there are cases with these STL containers where data is not always getting cleared. In many cases they will get cleared because they are contained within objects that get deleted. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing

[2026-02-09 14:53:14] #7 [Go To First Post]
Sierra_Chart Engineering - Posts: 23359
Regarding post #6 this conclusion is actually not correct. The intermittent delay we are seeing, it appears to have a cause different than we are thinking. It is not related to the CME market data processing. The server of concern, is also feeding our server in Japan (ds45.sierracharts.com), and we are watching data from that server, with no interruptions at all. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2026-02-09 14:53:48

[2026-02-09 15:32:45] #8 [Go To First Post]
Sierra_Chart Engineering - Posts: 23359
Regarding post #6, the issue we are seeing is local to our system we are monitoring from. There is no further issue that we are observing at all today with the Denali Exchange Data Feed for CME data. The data feed is being delivered with very low latency, with no interruptions and everything is working very well. This only goes to show, that when users have problems often these are issues specific to them based on, their system or Internet connectivity or local network. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2026-02-09 15:33:30

[2026-02-09 18:20:57] #9 [Go To First Post]
Sierra_Chart Engineering - Posts: 23359
We do not want to come to any definite conclusions at this point in time, but we believe, the source of the issue, which was a processing inefficiency in one or more locations, is the fact that STL containers, do not actually release memory when cleared. Thinking about it, the very fact that we call a clear function on a STL container and it does not actually release memory, is a design flaw. Not releasing memory can have its advantages. Therefore, the clear function needs to have a parameter, specifying whether or not to release memory. The fact that this does not exist is dumb. And the fact that these containers also throw exceptions, due to memory allocation failures is another dumb thing. This only goes to show, why we just cannot trust anyone. What we have learned over the decades is to do everything ourselves and trust no one. The fault has been STL. Well we have learned something. And we are not going to be using STL any longer, for lookup of symbol data objects. This further, just simply emphasizes that the concept of managed code, and higher-level languages like C#, really put you in a category of substandard software. This is why Sierra Chart is and will always be unmanaged native C++ code. And why programs like ninja trader, have a massive disadvantage and will never be able to overcome it. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2026-02-09 19:45:19

[2026-02-11 14:46:54] #10 [Go To First Post]
Sierra_Chart Engineering - Posts: 23359
Latency and performance of Sierra Chart Denali Exchange Data Feed for CME data from the Tokyo Japan server: ------------------------------------------------- Attached is a video, from the morning February 11, 2026, at 8:30 AM US Eastern time. This is during the employment economic data release. Attached is also a screenshot showing the trace route from the server, to the Aurora CME data center. This video was taken from our Tokyo Japan server. You can see the latency to the exchange timestamp in the "DD:" field at the top of the chart. This field is measured by taking the difference between the time on the computer, which is synchronized to NTP servers in Asia, and comparing it to the timestamp, of the trades or bid and ask data using the timestamps from the CME Globex system. There is precision to the millisecond in this calculation. As you can see, latency remains low, closer to 120 ms, and the updating is fast and continuous. At times the delay goes higher, but only briefly. The higher momentary latency is simply the result of the immense amount of processing that has to be done on the servers related primarily to MBO data. If MBO data is removed, and only 10 levels of depth is needed, we could deliver data with even lower latency. This measurement of latency is not a network latency check. Certainly network latency absolutely does increase the time but this is not what it is directly measuring. One thing that we notice, which is really irrelevant, is users will check the ping time to a particular server and think they have a better low latency connection if the latency is low. This is just simply not true and has no validity. Users may be, usually, much better off connecting directly to the United States servers. What matters, is a latency calculation like this: This field is measured by taking the difference between the time on the computer, which is synchronized to NTP servers in Asia, and comparing it to the timestamp, of the trades or bid and ask data using the timestamps from the CME Globex system. There is precision to the millisecond in this calculation. This of course, is a calculation being done in Asia. Your data delay calculation within your installed installation of Sierra Chart, will be done based upon your location. This calculation, is done within Sierra Chart. Here is information about the Data Delay (DD) field: https://www.sierrachart.com/index.php?page=doc/WorkingWithCharts.html#DataDelayHeaderField And one more thing, we connected to the Tokyo Japan server, from a location in New Zealand, and while the Data Delay increased about another 100 milliseconds, due to network latency, the update speed is the same as when we connect directly to the United States servers. It is fast and continuous and no delays. So any user connecting in Asia to the Sierra Chart Tokyo Japan server, absolutely is not going to have an issue as long as the Internet connection they have, is working well. So we have shown, there are no problems receiving the Denali Exchange Data Feed in Asia and we are confirming there are no problems, receiving the Denali Exchange Data Feed, as far as way as New Zealand. New Zealand is even further from the the Midwest of the United States as compared to Thailand to the Midwest of the United States. Clearly there are no problems, with the Denali Exchange Data Feed in Asia at all. Why we chose, Tokyo Japan versus Hong Kong for the Asia server for the Denali Exchange Data Feed. Hong Kong is part of China. It is a special administrative region of China. Japan is an actual country and when it comes to data centers, we would believe the infrastructure in Japan would be above Hong Kong. Differences are documented here: https://xlc.com/technology-insights/hong-kong-vs-japan-dedicated-server/ Clearly, Japan is a better choice. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2026-02-13 07:41:29


Video_2026-02-11_0830_EmploymentData.mp4 - Attached On 2026-02-11 14:11:36 UTC - Size: 40.93 MB - 99 views DS45_TraceRouteToAurora_CME_DC.png / V - Attached On 2026-02-13 07:40:36 UTC - Size: 16.06 KB - 98 views

[2026-02-14 05:13:18] #11 [Go To First Post]
Sierra_Chart Engineering - Posts: 23359
The solution for China, is what is described here: https://www.dataplugs.com/en/company/cn2-direct-china/ They are not the only provider of that service. Numerous co-location and server providers, provide the identical CN2 China direct service. The problem is it is limited to 50 Mbs. And we saw some pricing for higher data rates and it goes up astronomically more. For the small number of Chinese customers we have, 50 Mb is probably fine for the time being. The question we have, and we never could get a straight answer on this from the provider, is what international bandwidth (For outside of China) is provided to the server. Since we must have, the direct China connection, and also international bandwidth everywhere else to be able to access the CME data from the USA Midwest. They never give us a clear answer on this. They say international bandwidth is not included but in another answer they say indicate that it is. But we doubt the international bandwidth that we get now. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing

[2026-02-15 03:05:11] #12 [Go To First Post]
Sierra_Chart Engineering - Posts: 23359
Some additional notes: We have given all users 5 days of additional usage time related to this incident. The changes we have made in Sierra Chart on the server side to solve this problem are also applicable to the client side of Sierra Chart as well. So updating Sierra Chart to the current version 2882, is beneficial but not likely to make any real difference unless you leave Sierra Chart running for weeks or months. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2026-02-15 03:05:49

To post a message in this thread, you need to log in with your Sierra Chart account: