Support Board
Support Board
Search Board
Control Panel
View My Posts / Threads
Direct MessagesDate/Time: Sun, 28 Apr 2024 10:35:41 +0000
[Locked] - Denali Data Feed Compression Available
View Count: 6789
| [2020-09-22 09:36:54] |
| |
|
More information is here: Data Feed Stopping/Lagging Issue (non-Denali specific) | Post: 234601 Update with Help >> Download Prerelease to take advantage of this. There still is more work we are doing related to this, to optimize the whole setup of this on our servers. We should have that done by the end of the day Tuesday. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2020-09-22 09:47:35
|
| [2020-09-22 10:07:14] |
| User100912 - Posts: 77 |
|
SC Data - All Services | Heartbeat from server | ServerReceivedClientHeartbeatSecondsAgo=10, NumberOfOutstandingSendBuffers=0, TransmissionDelayInMilliseconds=0, ServerSendBufferSizeInBytes=895, ActualMessageDelay=0.9 seconds, DataCompRatio=2.24, UncompressedBytes=40429, CompressionTime=0.000270, NumCompressions=61 | 2020-09-22 05:06:31.877
Running this now... |
| [2020-09-22 10:25:17] |
| |
|
Yes that message does confirm you are on a compressed feed. Also for reference, the CompressionTime is in seconds. So to the right of the decimal place is effectively microseconds. .000001 is a microsecond. And when the market becomes more active, the buffer sizes become larger and the compression ratio increases. But you may not see the compression ratio immediately reflect that because it is the compression ratio across the entire connected session. So gradually you will begin to see it go down towards 2 and then if there is a higher amount of activity, gradually it will start to go up. So long as we can compress data at under 20 µs we should be able to take the compression up even higher. Like 4. But right now typically it is going to be 2 to 3. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2020-09-22 10:27:22
|
| [2020-09-22 19:09:00] |
| |
|
We did notice a stability issue with one of the server processes which is supporting compression which resulted in an abnormal shutdown. We are looking into the cause and we do have a dump file. This would have caused disconnected users at the time. Otherwise, the compression is working very well. Also understand that if you notice a lag during very busy times, there can be different reasons for this. Our testing of the data feed in recent weeks and today is that there is no lag coming out of the servers. Instead, lagging data would be due to other issues elsewhere. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2020-09-22 19:17:30
|
| [2020-09-23 08:57:41] |
| |
|
Regarding post #4, the stability issue with our real-time server process has been found and is being fixed now. This fix has now been released, and we are making further performance improvements. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2020-09-23 10:44:22
|
| [2020-09-23 20:49:59] |
| |
|
Regarding data feed compression, in version 2173 to be released later this evening, there is an option to enable and disable this. It is: Global Settings >> Data/Trade Service Settings >> SC Server Settings >> Use Real-time Data Compression. After changing the setting you need to reconnect to the data feed: File Menu: Procedure to Reconnect to the Data and Trade Servers There is also the Remote Buffer Delay Send Time in Milliseconds setting. Normally keep this at the default of 0 but it allows you to buffer data on the server before data is compressed. It can be set up to 5000 ms. The main purpose of using the Remote Buffer Delay Send Time in Milliseconds setting is to get a better compression ratio. This is useful for low bandwidth Internet connections like a satellite connection or a terrestrial microwave connection with limited bandwidth. So now users have the ability of receiving a high-quality data feed, even with limited bandwidth in remote areas. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2020-09-23 22:06:49
|
| [2020-09-24 05:14:52] |
| Sierra_Chart Engineering - Posts: 14111 |
|
We now have a data server located in the Pacific Northwest of the US. If you would like to have access to the server let us know. It can be used also as a Denali data feed relay. The main reason for this location is the international connectivity. But it may serve useful for users on the West Coast of the US. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2020-09-24 10:30:28
|
| [2020-09-24 05:17:19] |
| User100912 - Posts: 77 |
|
As you may know, I am on the Pacific coast and would like to be set on this server please.
|
| [2020-09-24 06:50:45] |
| Sierra_Chart Engineering - Posts: 14111 |
|
This has been done. The server is one of the servers available to be used for your Sierra Chart account. It is not the only one. So it does not necessarily mean that when you are connected to data feed you are using that server. You can tell if you are by looking at the Message Log when you see a connection to DS13.SierraCharts.com.
Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing |
| [2020-09-24 13:34:43] |
| Tooth Fairy - Posts: 79 |
|
Does it have ID like server N under the Data/Trade Service Setting or need to be set up by request? If it's the 2nd case, I'd like to be setup on that server too. Thanks.
|
| [2020-09-24 14:31:47] |
| |
|
Regarding post #10, no you cannot manually select the server, we need to set that up for your Sierra Chart account and we will do that. Update: This is done. You need to restart Sierra Chart. The plan is we will automatically make this West Coast server available among the pool of servers, for users logging in, in US Western states and from the Asia-Pacific region. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2020-09-24 14:38:13
|
| [2020-09-24 15:06:10] |
| red23 - Posts: 38 |
|
Can you set me up with the west coast server also. Thanks
|
| [2020-09-24 15:12:04] |
| USER0007 - Posts: 21 |
|
Could you also set me up with the west coast server, thank you.
|
| [2020-09-24 15:14:07] |
| User833286 - Posts: 14 |
|
could you please set me up also for west coast server. thankyou.
|
| [2020-09-24 15:33:06] |
| |
|
In regards to the above requests. This is all done. You need to restart Sierra Chart to have access to that server. But also please take note of this post: Denali Data Feed Compression Available | Post: 234892 We also monitored the data transmission from the West Coast server down to Auckland, and the performance was excellent. No lags, and no stopping. What this confirms, is that all the way from the Denali data feed in the CME Aurora data center, everything is working properly and optimally. We will try to take a video of this, tomorrow morning. Of course, the entire link is 100% fiber-optic across high-capacity multi-terabit links and monitoring within a data center. And we are very very impressed, by the real-time data compression. Here is the most recent heartbeat message related to it: SC Data - All Services | Heartbeat from server | ServerReceivedClientHeartbeatSecondsAgo=1, NumberOfOutstandingSendBuffers=2, TransmissionDelayInMilliseconds=40, ServerSendBufferSizeInBytes=1194, ActualMessageDelay=-0.1 seconds, DataCompRatio=2.12, UncompressedBytes=151321072, CompressionTime=0.500447, NumCompressions=29588 | 2020-09-24 11:29:32.899
So as you can see 151 MB was compressed in 500 milliseconds, and that is 29,588 separate compressions. This is is very impressive. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2020-09-24 15:43:55
|
| [2020-09-24 17:51:43] |
| USER0007 - Posts: 21 |
|
I don't see it yet. My logs are still showing connection to the following servers. ds23.sierracharts.com port 10149 ds21.sierracharts.com port 10150 Regarding post #10, no you cannot manually select the server, we need to set that up for your Sierra Chart account and we will do that. Update: This is done. You need to restart Sierra Chart.
|
| [2020-09-24 18:26:12] |
| User100912 - Posts: 77 |
|
I don't appear to be set to the West Coast server yet. Have done an update to prerelease and restart. TCP 192.168.0.113:61601 216.1.90.130:10149 TIME_WAIT TCP 192.168.0.113:61602 216.1.90.129:10150 TIME_WAIT TCP 192.168.0.113:61603 216.1.90.130:10149 TIME_WAIT TCP 192.168.0.113:61604 216.1.90.129:10150 TIME_WAIT TCP 192.168.0.113:61605 8.18.161.133:10149 TIME_WAIT TCP 192.168.0.113:61606 216.1.90.129:10150 TIME_WAIT DS3 is 65.182.172.42 |
| [2020-09-24 20:07:30] |
| |
|
This server is DS13.SierraCharts.com. You can ping that address to determine its IP. You may have to reconnect about four times to actually get it: File Menu: Procedure to Reconnect to the Data and Trade Servers We will set up better management of this so it is more easily used. We just set up an interim solution for the time being. These address port combinations are for historical data: ds23.sierracharts.com port 10149
ds21.sierracharts.com port 10150 This is what you want to look for: SC Data - All Services | Connecting to the server ds13.sierracharts.com. Port 10059 | 2020-09-24 10:30:01.921
But we really do not want to put too much emphasis on this particular server. For users in the US who are in the western US, the primary advantage is that it is another server with lots of processing capacity. That is really the main advantage. So you would be on a server, which would have a small number of users and a lot of available bandwidth. It will give you a little lower latency, but in many cases the overall path of latency is going to be a little worse than going direct to Chicago (but this would not be relevant though if you are at a great distance like in the Asia-Pacific region). Although if your Internet connection is not good, you will have a faster return path for IP acknowledgments which would definitely be helpful. So it is important to focus on the availability of processing capacity and bandwidth (probably more important) which we normally have lots of on the other servers anyway, and faster return path for IP acknowledgments. And maybe just based upon routing in your area you just have better connectivity to this server. This last item is really quite important as well. Our provider for the colocation for this server tells us they use a mix of six Internet carriers whereas out of Chicago, each location just has one carrier. But we want to once again say, if you are continuing to notice a problem, you really have to focus on your Internet connection. The data compression makes a huge difference, and the amount of data sent has been cut down more than half. That is dramatic. And we have thoroughly examined all issues at this point and we will be releasing some additional fine-tuning this evening. We will continue to monitor but if we identify no other areas of improvement or issues within our control you have to look at your Internet connection at this point. Really, it just bewilders us, why some users in the US, have poor quality Internet and other users at such a great distance on other continents have superior consistent low latency connectivity. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2020-09-24 20:17:55
|
| [2020-09-24 20:18:28] |
| User100912 - Posts: 77 |
|
SC Data - All Services | Connecting to the server ds13.sierracharts.com. Port 10059 | 2020-09-24 15:11:40.632
This seems to be working for me.My last mile is a 100mbit fiber line here in Mexico - could be a peering issue or bufferbloat. |
| [2020-09-24 20:24:04] |
| |
|
In regards to post #20 let us know if you notice an improvement being on that server. And make sure you do have compression enabled as we explained further above (Post #6). Latest compression data: SC Data - All Services | Heartbeat from server | ServerReceivedClientHeartbeatSecondsAgo=6, NumberOfOutstandingSendBuffers=2, TransmissionDelayInMilliseconds=40, ServerSendBufferSizeInBytes=208, ActualMessageDelay=0.0 seconds, DataCompRatio=2.09, UncompressedBytes=614928397, CompressionTime=2.116531, NumCompressions=173507 | 2020-09-24 16:22:37.679
~615 MB compressed. Total compression time 2.11 seconds. Very impressive. The way that we handle compression, is as follows: The individual client connection in our real-time server program fills in a buffer with data to send to the user. That buffer, is then processed on one of multiple threads. Currently we are running 4 threads but this is tunable. We can increase as needed. The data is compressed, and then passed to the operating system to go out to the user. This is an overlapped I/O operation with no intermediate operating system buffer. The buffer goes straight out to the network adapter. Extremely efficient nonblocking process. We can easily scale this up by adding more CPU cores, and more threads. But with very low compression time, is there is already ample capacity. And even during the busiest of times, there is no latency added which is perceivable. The average compression time is about 12 µs. This is microseconds. A tiny fraction of a millisecond. Another advantage, as the market becomes busy and there is more data to transmit to the user, the compression ratio increases. It goes up to 3 or 4. Therefore the actual output data, is not much more. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2020-09-24 20:42:48
|
| [2020-09-24 23:59:14] |
| USER0007 - Posts: 21 |
|
This is what you want to look for:
SC Data - All Services | Connecting to the server ds13.sierracharts.com. Port 10059 | 2020-09-24 10:30:01.921 Still not working at my end, I see the following in the logs: SC Data - All Services | Connecting to the server ds23.sierracharts.com. Port 10059 | 2020-09-24 19:36:55.767 Date Time Of Last Edit: 2020-09-25 00:01:50
|
 2020-09-24_17h00_51.png / V - Attached On 2020-09-25 00:01:18 UTC - Size: 27.6 KB - 348 views |
| [2020-09-25 01:00:46] |
| Acro - Posts: 436 |
|
Can you also please set me up with the west coast server. Now the Singapore server has gone I believe this will be my next best option |
| [2020-09-25 03:25:48] |
| |
|
Regarding post #22 and #23 we are working on making providing this server address automatically based on your location. We should have this done in about an hour. After that just restart Sierra Chart and perform a Forced Login: Sierra Chart - Software Login Instructions: Forced Login We never had a Singapore server for market data and we removed it for trading, because we did not have confidence in that server or the routing to TT. This was done for safety reasons. It is not right for us to put out a server, that has reliability issues. But what we will do is we will set up a TT order routing process on the US West server. We will get that done as soon as possible. And that really is going to be much better all around especially if you are trading US markets. You will have much lower and stable order routing latency. That actually was one of our plans anyway for that server. And as we said, that location was chosen due to the subsea cable connectivity to the Asia-Pacific region which terminate in that very same building (the Westin building: https://www.westinbldg.com/). Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2020-09-25 03:28:50
|
| [2020-09-25 04:04:54] |
| ertrader - Posts: 645 |
|
For the southern coast Houston area, what do you recommend for TT routing/Futures data? Current primary is set to Server 1 with backup Server 1 as Server 4 and backup server 2 as server 4. These are defaults I have not changed.
Date Time Of Last Edit: 2020-09-25 04:15:51
|
To post a message in this thread, you need to log in with your Sierra Chart account: