Support Board
Date/Time: Fri, 16 Jan 2026 06:46:00 +0000
RESOLVED - Emergency Situation: CME Overheating. We Are Powering DOWN Our CME Systems
View Count: 2037
| [2025-11-28 15:06:42] |
| JBTFD - Posts: 38 |
|
Gotcha, so then in these rare instances when the markets re-open price could gap up or down and skip your stop entirely (if you had a position on BEFORE the halt)
|
| [2025-11-28 22:57:17] |
| Sierra_Chart Engineering - Posts: 22390 |
|
We are always very concerned, with high temperatures for our servers, and we always run our servers with configurations, where they run at low temperatures. For example, we try to keep our CPUs not running more than 115°F. Often they run lower. This is why we have never experienced a CPU failure. One of our servers in Aurora may very well have suffered a partial hardware failure of the network adapter due to this overheating situation. A network adapter failure due to the high temperatures. We do not know for sure but it is not functioning right. And we have done everything that should have made it work but it does not work. We are going to be decommissioning one of the servers. Not a problem as we have others. But there is going to be some downtime, during this work over the weekend. This work began as of about one hour ago. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2025-11-28 22:57:30
|
| [2025-11-29 00:35:14] |
| Sierra_Chart Engineering - Posts: 22390 |
|
Gradually over the next 6 hours the connectivity will come back for Teton order routing. Most of the work is done moving to a new server.
Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing |
| [2025-11-29 01:06:57] |
| TonyCipriani - Posts: 73 |
|
If such an event like this happened during regular trading hours, where a technical glitch caused a sudden market halt, there's a situation where a random black swan event during the long downtime could severely jeopardize one's account. A stop loss in such situation is useless. Would be good for high-performance options trading functionality as it seems like a more robust speculative trading vehicle over futures given the technical risks. Something like cash-settled options would be the only way to truly protect traders. |
| [2025-11-29 01:09:51] |
| Sierra_Chart Engineering - Posts: 22390 |
|
The cooling situation is not completely resolved in the Cyrus One data center where the CME is located. The temperatures we are seeing, are 10°F higher than usual. We are seeing an inlet temperature of about 71-72°F. The temperatures the equipment in that data center were exposed to, were very high. We would expect, that there has been, some degradation of equipment in some cases. Electronic components cannot necessarily be exposed to high temperatures, without some degradation that may show as a failure at a later point in time. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2025-11-29 08:32:16
|
| [2025-11-29 08:31:18] |
| Sierra_Chart Engineering - Posts: 22390 |
|
Temperatures are coming down: CPU1 Temp 41 °C (105.8 °F)
CPU2 Temp 45 °C (113 °F) System Board Inlet Temp 18 °C (64.4 °F) System Board Exhaust Temp 29 °C (84.2 °F) Max DIMM Temperature 32 °C (89.6 °F) Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing |
| [2025-11-29 08:49:08] |
| Sierra_Chart Engineering - Posts: 22390 |
|
The Michigan data center we use, at this time of year because it is cold outside does not use chillers. As we understand it uses this method: https://www2.kyotocooling.com/how-kyotocooling-works/ So as long as there is power, there could not be of failure like occurred at the Cyrus One data center in Aurora. The question is how many wheels do they have. We will try to find out. When it is cold outside, logically the outside air would be used to cool the data center. And using a heat transfer mechanism, like this wheel that is continuously turning makes a lot of sense. Isolating the inside air from the outside air. So why is this not more widely used. Good question. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2025-11-29 08:50:23
|
| [2025-11-29 16:56:49] |
| Sierra_Chart Engineering - Posts: 22390 |
|
Now: System Board Inlet Temp 17 °C (62.6 °F)
Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing |
| [2025-11-29 20:02:38] |
| Sierra_Chart Engineering - Posts: 22390 |
|
This overheating situation really has us concerned. Will it happen again? Why did it happen. The data center has to explain in detail, why this has happened, and why there is a failure of redundancy with cooling. None of this is making any sense. Are they just using watercooled chillers? There should be backup fans to utilize the cool outside air. Even through a heat exchanger process to create isolation between the inside and outside air. What we are very concerned about, is degradation of the electronic components of our servers, which leads to a failure at a future point in time. Anyway, we are going to be diversifying away from this data center which we thought was a more trustworthy data center but apparently not. We will still keep equipment there, but have another location as well. Even though, this is not going to help with the outage that has occurred, at least, we do not want to be burdened, with a damaging situation like this. We will leave that to others. Modern data centers are remarkably technologically complex, and keeping them running safely and efficiently requires continual close monitoring and management. Among the most important various tasks facing data center managers is maintaining the right temperature. Should the temperature and humidity rise to excessive levels inside the data center, condensation can start to form - thereby damaging the machines within. It should go without saying that this can cause massive damage and disruption, and that it therefore has to be avoided at all costs. Fortunately, there are various technologies on hand which can help to keep data center temperatures at the right level.
Who is going to pay for the damage that has occurred. Are we going to be given a new server? We should. Of course, we will pay for our own replacement equipment and get everything replaced that has failed or anything of concern. We do a good job with maintaining our equipment. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2025-11-29 20:15:32
|
| [2025-11-30 18:17:43] |
| Sierra_Chart Engineering - Posts: 22390 |
|
Latest from CME: The Colo Data Halls and Carrier Rooms continue to operate within target temperature ranges. Below is the latest information from CyrusOne: "CyrusOne’s chiller plant remains stable with additional redundancy in place to ensure reliability. All data halls are within normal operating range and temperatures remain stable. On-site and remote engineering teams continue proactive monitoring and engagement to maintain a stable and secure environment." Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing |
| [2025-11-30 18:34:00] |
| Sierra_Chart Engineering - Posts: 22390 |
|
We are reconfiguring one of the servers used by Teton order routing. Teton order routing will be unavailable for a few minutes.
Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing |
| [2025-11-30 18:56:46] |
| Sierra_Chart Engineering - Posts: 22390 |
|
This reconfiguration is complete.
Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing |
| [2025-11-30 22:56:07] |
| Sierra_Chart Engineering - Posts: 22390 |
|
Order routing is working fine on the new server. Latency is below 1 ms as usual. But we cannot help but be concerned, if there is any damage to equipment that might show up later due to the high temperatures. Not likely anytime soon though. We wish, we shut the servers down sooner. We were not monitoring because it was Thanksgiving evening, thinking everything would be fine and quiet. We have no alerts for high temperatures and the systems were still running fine. Maybe we should add high temperature alerts. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2025-11-30 23:49:23
|
To post a message in this thread, you need to log in with your Sierra Chart account: