Home >> Support Board

Support Board

Date/Time: Sat, 11 May 2024 18:52:01 +0000

How to Post a Message for Support

Receiving bad SecurityDefinitionResponse messages from DTC server

View Count: 647

[2020-06-09 08:44:08] #1 [Go To First Post]
User442651 - Posts: 25
After sending a SymbolsForExchangeRequest to the server, I start receiving malformed messages amongst the SecurityDefinitionResponse replies, and importantly I never get the final response. Also an excessive number of bytes are being received from the server with each of these messages. Furthermore, when encryption is enabled the client crashes after it is unable to decrypt the server response. Here is the output from my program. This is using Binary encoding. ---------------------------------------------- Received 388 bytes from server: message of type: GeneralLogMessage and size: 132 ---------------------------------------------- ---------------------------------------------- Received 76 bytes from server: message of type: ExchangeListResponse and size: 76 ---------------------------------------------- ---------------------------------------------- Received 348 bytes from server: message of type: SecurityDefinitionResponse and size: 348 security definition for symbol: AUDCAD, description: Australian Dollar / Canadian Dollar ---------------------------------------------- ---------------------------------------------- Received 32768 bytes from server: message of type: SecurityDefinitionResponse and size: 348 security definition for symbol: AUDCHF, description: Australian Dollar / Swiss Franc ---------------------------------------------- ---------------------------------------------- Received 65536 bytes from server: message of type: Unknown and size: 0 ---------------------------------------------- ---------------------------------------------- Received 33296 bytes from server: message of type: Unknown and size: 0 ---------------------------------------------- ---------------------------------------------- Received 2032 bytes from server: message of type: Unknown and size: 0 ---------------------------------------------- ---------------------------------------------- Received 24708 bytes from server: message of type: SecurityDefinitionResponse and size: 348 security definition for symbol: $MYHH, description: YTD Highs Price gt $10 ---------------------------------------------- ---------------------------------------------- Received 65536 bytes from server: message of type: SecurityDefinitionResponse and size: 348 security definition for symbol: $DSDT, description: U.S. Diversified Reits Index ---------------------------------------------- ---------------------------------------------- Received 65536 bytes from server: message of type: 28271 and size: 17184 ---------------------------------------------- ---------------------------------------------- Received 65536 bytes from server: message of type: Unknown and size: 0 ---------------------------------------------- ---------------------------------------------- Received 65536 bytes from server: message of type: Unknown and size: 0 ---------------------------------------------- ---------------------------------------------- Received 65271 bytes from server: message of type: 25441 and size: 28787 ---------------------------------------------- ---------------------------------------------- Received 65536 bytes from server: message of type: Unknown and size: 0 ---------------------------------------------- ---------------------------------------------- Received 65536 bytes from server: message of type: Unknown and size: 0 ---------------------------------------------- ---------------------------------------------- Received 65536 bytes from server: message of type: 32768 and size: 0 ---------------------------------------------- ---------------------------------------------- Received 65536 bytes from server: message of type: Unknown and size: 0 ---------------------------------------------- ---------------------------------------------- Received 65536 bytes from server: message of type: Unknown and size: 0 ---------------------------------------------- ---------------------------------------------- Received 65536 bytes from server: message of type: Unknown and size: 572 ---------------------------------------------- ---------------------------------------------- Received 65536 bytes from server: message of type: Unknown and size: 0 ---------------------------------------------- ---------------------------------------------- Received 65536 bytes from server: message of type: Unknown and size: 0 ---------------------------------------------- ---------------------------------------------- Received 65536 bytes from server: message of type: Unknown and size: 0 ---------------------------------------------- ---------------------------------------------- Received 65536 bytes from server: message of type: 21845 and size: 20031 ---------------------------------------------- ---------------------------------------------- Received 65536 bytes from server: message of type: Unknown and size: 0 ---------------------------------------------- ---------------------------------------------- Received 65536 bytes from server: message of type: Unknown and size: 0 ---------------------------------------------- ---------------------------------------------- Received 65536 bytes from server: message of type: Unknown and size: 0 ---------------------------------------------- ---------------------------------------------- Received 64794 bytes from server: message of type: Unknown and size: 12338 ---------------------------------------------- ---------------------------------------------- Received 65483 bytes from server: message of type: Unknown and size: 0 ---------------------------------------------- ---------------------------------------------- Received 65536 bytes from server: message of type: Unknown and size: 0 ---------------------------------------------- ---------------------------------------------- Received 65430 bytes from server: message of type: Unknown and size: 0 ---------------------------------------------- ---------------------------------------------- Received 65483 bytes from server: message of type: Unknown and size: 0 ---------------------------------------------- ---------------------------------------------- Received 65483 bytes from server: message of type: Unknown and size: 0 ---------------------------------------------- ---------------------------------------------- Received 65483 bytes from server: message of type: Unknown and size: 0 ---------------------------------------------- ---------------------------------------------- Received 65483 bytes from server: message of type: Unknown and size: 0 ---------------------------------------------- ---------------------------------------------- Received 65483 bytes from server: message of type: Unknown and size: 88 ---------------------------------------------- ---------------------------------------------- Received 34251 bytes from server: message of type: Unknown and size: 0 ---------------------------------------------- ---------------------------------------------- Received 26448 bytes from server: message of type: SecurityDefinitionResponse and size: 348 security definition for symbol: N8J21, description: UK Peak Electricity Gregorian Futures (UPL) - LIFFE ---------------------------------------------- ---------------------------------------------- Received 7656 bytes from server: message of type: SecurityDefinitionResponse and size: 348 security definition for symbol: MAN21, description: Milling Wheat Futures (WA) - WCE ---------------------------------------------- ---------------------------------------------- Received 65483 bytes from server: message of type: SecurityDefinitionResponse and size: 348 security definition for symbol: CXU20, description: Containerised White Sugar (CS) - ICE EU / LIFFE ---------------------------------------------- ---------------------------------------------- Received 65483 bytes from server: message of type: Unknown and size: 0 ---------------------------------------------- ---------------------------------------------- Received 65483 bytes from server: message of type: 25710 and size: 18720 ---------------------------------------------- ---------------------------------------------- Received 65483 bytes from server: message of type: 63 and size: 32768 ---------------------------------------------- ---------------------------------------------- Received 65483 bytes from server: message of type: Unknown and size: 0 ---------------------------------------------- ---------------------------------------------- Received 65483 bytes from server: message of type: Unknown and size: 0 ---------------------------------------------- ---------------------------------------------- Received 65483 bytes from server: message of type: 21320 and size: 0 ---------------------------------------------- ---------------------------------------------- Received 65483 bytes from server: message of type: Unknown and size: 0 ---------------------------------------------- ---------------------------------------------- Received 65483 bytes from server: message of type: Unknown and size: 0 ---------------------------------------------- ---------------------------------------------- Received 65483 bytes from server: message of type: Unknown and size: 0 ---------------------------------------------- ---------------------------------------------- Received 65483 bytes from server: message of type: Unknown and size: 0 ---------------------------------------------- ---------------------------------------------- Received 65483 bytes from server: message of type: Unknown and size: 0 ---------------------------------------------- ---------------------------------------------- Received 65483 bytes from server: message of type: 71 and size: 21576 ---------------------------------------------- ---------------------------------------------- Received 65483 bytes from server: message of type: Unknown and size: 0 ---------------------------------------------- ---------------------------------------------- Received 65483 bytes from server: message of type: Unknown and size: 0 ---------------------------------------------- ---------------------------------------------- Received 65483 bytes from server: message of type: Unknown and size: 0 ---------------------------------------------- ---------------------------------------------- Received 43600 bytes from server: message of type: Unknown and size: 0 ---------------------------------------------- Sending heartbeat ---------------------------------------------- Received 16 bytes from server: message of type: Heartbeat and size: 16 ----------------------------------------------

[2020-06-09 09:13:53] #2 [Go To First Post]
Sierra Chart Engineering - Posts: 104368
You need to check for the problem on your side. That is where the issue is. Make sure you are using the latest header files and data structures. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, change to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2020-06-09 09:14:10

[2020-06-09 15:41:32] #3 [Go To First Post]
User442651 - Posts: 25
Well, is it normal to be receiving 60,000+ bytes from the server at once (for a message that is supposed to 348 bytes)? This is a single statement stream read, which can't be "checked" any further. Also, the message type is at the start of the message header, where I am sometimes getting blocks of zeros. Having out of date structure definitions would not affect that.

[2020-06-09 17:57:02] #4 [Go To First Post]
Sierra Chart Engineering - Posts: 104368
Your determination of the message size is incorrect because there is some problem on your side with reading the stream of data. There could not be anything wrong with this functionality. It is used every day by us handling multiple tens of thousands of security definition messages in seconds. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, change to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2020-06-09 17:57:41

[2020-06-09 18:59:48] #5 [Go To First Post]
User442651 - Posts: 25
OK. So should I be receiving exactly 348 bytes from the data stream for a SecurityDefinitionResponse? How about for the GeneralLogMessage? My code determined that 388 bytes were received from server, but the message size encoded in the header is 132. Is it possible the server really sent 388 bytes for this message?

[2020-06-10 00:07:24] #6 [Go To First Post]
Sierra_Chart Engineering - Posts: 14315
Are you using Binary encoding and not Binary encoding with Variable Length Strings? With binary encoding the size of messages are fixed and you can use the sizeof operator in the C++ language to check the structure sizes. There are so many fundamental problems with this question: Is it possible the server really sent 388 bytes for this message? It should not even be asked. One problem is, there is no way for us to validate that your conclusion is correct. That is essentially advanced programming help. Which we do not provide. The second problem is that looking at this question in isolation, the server can be sending more data because there are other messages sent at the same time. The reason it should not be asked is because you should already know that there is no way we can provide any type of proper answer. . Furthermore, when encryption is enabled the client crashes after it is unable to decrypt the server response. This indicates a problem in your code. This should not happen. Furthermore, what is the purpose of telling us this. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, use the Teton service: Sierra Chart Teton Futures Order Routing

[2020-06-10 10:36:43] #7 [Go To First Post]
norvik_ - Posts: 106
Hi. 100% You have a mistake in your code, receiving buffer overflowing..

[2020-06-11 16:39:00] #8 [Go To First Post]
User442651 - Posts: 25
Are you using Binary encoding and not Binary encoding with Variable Length Strings? With binary encoding the size of messages are fixed and you can use the sizeof operator in the C++ language to check the structure sizes. Yes, binary encoding. I am using C#, but yes I know how to calculate structure sizes (so if my size #'s are off then there is a problem with the structure definition). ...The second problem is that looking at this question in isolation, the server can be sending more data because there are other messages sent at the same time. What I meant is that the call to Stream.ReadAsync returns 388 bytes in that example, but now you helped me understand that the stream is not necessarily returning one message per read. I will see if that fixes the issue with "bad" messages once I fix this. This indicates a problem in your code. This should not happen. Furthermore, what is the purpose of telling us this. This is not "in my code", per se. I simply wrap the socket in an SslStream and the .NET library handles encryption/decryption automatically from there. It is throwing a common exception that can apparently even be caused by hardware issues. The purpose of telling you this, was that I thought it was at least conceivable that a bad message from the DTC server could be the cause.

[2020-06-11 16:43:20] #9 [Go To First Post]
User442651 - Posts: 25
norvik_: Hi. 100% You have a mistake in your code, receiving buffer overflowing.. Yes, there was the error I just mentioned. But how do you know it is buffer overflow? I don't see how there could be buffer overflow, unless there is a bug in the .NET library (i.e. highly unlikely).

[2020-06-11 23:21:46] #10 [Go To First Post]
norvik_ - Posts: 106
Sorry, can't help with dotnet code. I'm using c++ for such things. I don't know exactly which values contains your output, but I had never received packet with size more than 4 kb from SC dtc server. Maybe, there are reason take a sniffer and check real size of packets your app receives?

[2020-06-16 07:27:15] #11 [Go To First Post]
User442651 - Posts: 25
Everything was fine with the SecurityDefinitionResponse messages once I made the code fix. I think the packet size is implementation dependent. I have a 65536 byte buffer and made a request to the stream reader accordingly. If my buffer was 4K then I would receive packets of size at most 4k. The encryption also works fine, and I don't think there was a problem with the code. I believe it was my stack: Either because I was in debug mode in Visual Studio Code, which I have seen cause strange behaviour (unlike MS Visual Studio), or else because before I was running SierraChart on an Ubuntu system using WINE.

To post a message in this thread, you need to log in with your Sierra Chart account: