Login Page - Create Account

Support Board


Date/Time: Tue, 16 Apr 2024 21:28:20 +0000



Post From: DTC Protocol

[2016-03-30 00:01:17]
User783015 - Posts: 7
Thanks, I have updated to the pre-release and I'm running Sierra Chart 1389 now.

I am playing with the Sierra Chart historical data service but I'm having some problems receiving the historical data. I'm using JSON encoding. I send the HISTORICAL_PRICE_DATA_REQUEST message to SC, receive the HISTORICAL_PRICE_DATA_RESPONSE_HEADER as expected but at that point when I should start receiving HISTORICAL_PRICE_DATA_RECORD_RESPONSE messages, Sierra starts throwing some binary data at me which I don't understand.

Here's the output from my script:

$ python -u DTCCliPy.py
DBG: DTCCliPy: Connected to localhost:11098
DBG: DTCCliPy: Sending encoding request (requested encoding: JSON)
DBG: DTCCliPy: RCV: 10 00 07 00 07 00 00 00 02 00 00 00 44 54 43 00
DBG: DTCCliPy: Binary encoding response received:
'(16, 7, 7, 2, 'D', 'T', 'C', '\x00')'
DBG: DTCCliPy: Sending logon request:
'{Type:1,ProtocolVersion:7,HeartbeatIntervalInSeconds:60,ClientName:"TestClient",Username:"user"}'
DBG: DTCCliPy: Logon response received:
'{Type:2,ProtocolVersion:7,Result:1,Integer_1:0,MarketDepthUpdatesBestBidAndAsk:0,TradingIsSupported:0,OCOOrdersSupported:0,OrderCancelReplaceSupported:1,
SecurityDefinitionsSupported:0,HistoricalPriceDataSupported:1,ResubscribeWhenMarketDataFeedAvailable:0,MarketDepthIsSupported:1,OneHistoricalPriceDataRequestPerConnection:1,
BracketOrdersSupported:0,UseIntegerPriceOrderMessages:0,UsesMultiplePositionsPerSymbolAndTradeAccount:0,MarketDataSupported:0,ResultText:"Logon successful.",
ReconnectAddress:"",ServerName:"SC HistoricalDataServer",SymbolExchangeDelimiter:""}'
DBG: DTCCliPy: SC HistoricalDataServer: Logon successful.
DBG: DTCCliPy: Sending historical data request:
'{Type:800,RequestID:1,Symbol:"ADSK",RecordInterval:86400,StartDateTime:1458430359,UseZLibCompression:0}'
DBG: DTCCliPy: Historical data response received:
'{Type:701,MessageText:"Data is being downloaded from a remote source. Download will start when this is done."}'
DBG: DTCCliPy: Ignoring response type 701
DBG: DTCCliPy: Historical data response received:
'{Type:801,RequestID:1,RecordInterval:86400,UseZLibCompression:0,NoRecordsToReturn:0,IntToFloatPriceDivisor:0}'
DBG: DTCCliPy: RCV: 58 00 23 03 01 00 00 00 80 39 ef 56 00 00 00 00 00 00 00 e0 51 d8 4c 40 00 00 00 00 29 1c 4d 40 00 00 00 c0 f5 88 4c 40 00 00 00 80 c2
15 4d 40 00 00 00 00 fc 20 35 41 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Traceback (most recent call last):
File "DTCCliPy.py", line 357, in <module>
dtc_client.historical_data_response( )
File "DTCCliPy.py", line 332, in historical_data_response
response = struct.unpack( "<2Hiq5dI2dB" , message )
struct.error: unpack requires a string argument of length 77

First I tried to parse the binary data I'm receiving
58 00 23 03 01 00 00 00 80 39 ...
as a binary HISTORICAL_PRICE_DATA_RECORD_RESPONSE message. But the binary data don't match that message - I'm expecting 77 bytes message but the first 2 bytes suggest the size of 88, the second 2 bytes suggest a message type 8963 which is invalid..

So I have two questions:
- why am I receiving binary stream instead of JSON data?
- what does the binary stream mean?

Am I by any chance receiving ZLib compressed data even though I didn't ask for them?

Any help appreciated, this doesn't look like something I will be able to figure out on my own.

This is what I see in the SC log:

HD Server | New server thread for client has been created for incoming connection. | 2016-03-30 01:32:39
HD Server | Received login. Requesting authorization. | Username: user. | 2016-03-30 01:32:39
HD Server | Login has been authorized. | Username: user. | 2016-03-30 01:32:39
HD Server | HD Request | Symbol: ADSK | ServiceCode: | RecordInterval: 86400 | MaximumDaysRequested: 0 | Start Date-Time: 2016-03-19 23:32:39 | ClientRequestID: 1 | Username: user. | 2016-03-30 01:32:39
HD Request # 17 | Downloading Historical Daily chart data for ADSK. Starting date: 2016-03-28. Service: equities.us | 2016-03-30 01:32:39
HD Request # 17 | Using user Barchart Historical data account. | 2016-03-30 01:32:39
HD Request # 17 | Requesting historical Daily data for ADSK starting at 2016-03-27 | 2016-03-30 01:32:39
HTTP connection to ds01.ddfplus.com:80 (0) | Resolved address ds01.ddfplus.com to IP 8.18.161.123. | 2016-03-30 01:32:39
HTTP connection to ds01.ddfplus.com:80 (127) | Connected. | 2016-03-30 01:32:40
HD Request # 17 | Receiving historical Daily data for ADSK starting at 2016-03-28 | 2016-03-30 01:32:40
HD Request # 17 | Writing historical Daily data to the file ADSK.dly | 2016-03-30 01:32:40
HD Request # 17 | Received 2 records from 2016-03-28 00:00:00 to 2016-03-29 00:00:00 (2.0 days) and wrote 2 records for ADSK | 2016-03-30 01:32:40
HD Request # 17 | Daily download COMPLETE for ADSK. Completion time: 0s. Unique request ID: 16 | 2016-03-30 01:32:40
Removed historical data download ID 16 | 2016-03-30 01:32:40

HD Server | HD Request # 1 | Symbol: ADSK | ServiceCode: | RecordInterval: 86400 | Records Served: 6 | ClientRequestID: 1 | Username: user. | 2016-03-30 01:32:40
Socket (0) | Closed. Windows error code 10053: An established connection was aborted by the software in your host machine. | 2016-03-30 01:32:40
Socket (0) | Shutdown and closed. | 2016-03-30 01:32:40

Btw. it seems to be using barchart. Is this expected? I thought that SC provides its own historical data service. The whole point of my exercise is to overcome the limitation IB have on their historical data (60 requests per 10 minutes). I need to download latest data snapshot for 500+ stock tickers before the end of regular trading hours which my system uses to make trading decisions. Is this something I can achieve with SC?

Thanks!
Petr
Date Time Of Last Edit: 2016-03-30 00:02:56