Login Page - Create Account

Support Board


Date/Time: Fri, 19 Aug 2022 04:25:47 +0000



Version 2198: Support For Historical Market Depth Data With Denali/Delayed Exchange Feeds

View Count: 5688

[2020-11-27 21:29:54]
Sierra Chart Engineering - Posts: 104220
Is the goal to have all products and spreads available?
We will set up what users need.

Is there a list somewhere of what is currently available?
No.

We are not maintaining historical market depth data for spreads.

Extended and historical market depth data for TN will go into effect over the weekend.
Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service:
Sierra Chart Teton Futures Order Routing
[2020-12-07 20:50:50]
nosast - Posts: 246
How to limit the days of automatic downloading historical market depth data? Looks like SC is always downloading the recent 15 days. I would like to limit it to the current day or current day -1
[2020-12-07 21:16:17]
Sierra Chart Engineering - Posts: 104220
We need to add a separate setting for this. We will do that. Right now it is controlled by these particular settings:
Data/Trade Service Settings: Maximum Historical Intraday Days to Download >> Non-Tick Data / 1-Tick Data (Global Settings >> Data/Trade Service Settings >> Common Settings)
Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service:
Sierra Chart Teton Futures Order Routing
[2020-12-07 22:12:27]
nosast - Posts: 246
A separate setting for depth download is perfect. Thanks!
[2020-12-14 14:27:19]
Chad - Posts: 213
Are historical depth files supposed to be stored within the \Data\MarketDepthData subdir? I followed steps listed in posts 1 and 12 above, restarted & all that, and scrolled back 15 days then R click and 'Download Depth Data from Date-Time'...no visible change, and no new .depth files in that subdir.

I'm looking at COMEX HG (Copper) futures, got the HGH21 intraday chart loaded, and have Denali-All Services with NYMEX and COMEX non-pro w/ depth enabled.
[2020-12-14 18:33:58]
Sierra Chart Engineering - Posts: 104220
The answer is yes. However, the most important question is whether you see the historical market depth data showing up on the chart when using the Market Depth Historical Graph study.:
https://www.sierrachart.com/index.php?page=doc/StudiesReference.php&ID=375#Market_Depth_Historical_Graph

If not, then follow these instructions:

Follow the instructions here to clear the Message Log:
Message Log: Clearing the Message Log

Right-click on the chart somewhere and select 'Download Depth Data from Date-Time'

After about 2 minutes provide a copy of the Message Log following these instructions:
https://www.sierrachart.com/index.php?page=PostingInformation.php#MessageLog
Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service:
Sierra Chart Teton Futures Order Routing
[2020-12-14 18:40:17]
Chad - Posts: 213
Re: seeing the historical depth w MDHG study, currently, no...I am on Service Package 3 (IIRC Pkg 5 is needed for this study, please confirm). I do have the 'Depth of Market Data' study applied.
[2020-12-14 20:31:09]
Sierra Chart Engineering - Posts: 104220
Yes you do need the Advanced Service Package for that study. But we can tell if the Historical market depth data is getting downloaded from the Message Log.
Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service:
Sierra Chart Teton Futures Order Routing
[2020-12-15 03:37:40]
Chad - Posts: 213
Message Log contents:

Software version: 2207 64-bit | 2020-12-14 21:27:00.269
Primary Thread ID: 60816 | 2020-12-14 21:27:00.269
Usage end date: 2021-09-08 | 2020-12-14 21:27:00.269
Enabled for: Sierra Chart Historical Data Service. | 2020-12-14 21:27:00.269
Enabled for: Denali Real-Time Exchange Data Feed. | 2020-12-14 21:27:00.269
Enabled for: Delayed Denali Real-Time Exchange Data Feed. | 2020-12-14 21:27:00.269
Enabled for exchange2: COMEX | 2020-12-14 21:27:00.269
Enabled for exchange2: NYMEX | 2020-12-14 21:27:00.269
Allow Support for Sierra Chart Data Feeds is enabled. | 2020-12-14 21:27:00.269
Current selected Data/Trading service: SC Data - All Services | 2020-12-14 21:27:00.269
Custom symbol settings values: enabled | 2020-12-14 21:27:00.269
Chart Update Interval: 10 | 2020-12-14 21:27:00.269
Intraday Data Storage Time Unit: 0 | 2020-12-14 21:27:00.269
Time Zone: -06:00:00 (CST-06CDT+01,M3.2.0/02:00,M11.1.0/02:00) | 2020-12-14 21:27:00.269
2020-12-15 03:27:00 Local computer time in UTC | 2020-12-14 21:27:00.269
2020-12-14 21:27:00 Local computer time in SC Time Zone | 2020-12-14 21:27:00.269
2020-12-15 03:26:36 Server time in UTC | 2020-12-14 21:27:00.269
Local computer UTC time and Server UTC time difference: 0 seconds. | 2020-12-14 21:27:00.269
Program path: C:\SierraChart\ | 2020-12-14 21:27:00.269
Data Files path: C:\SierraChart\Data\ | 2020-12-14 21:27:00.269
OS Version Number: 6.1 | 2020-12-14 21:27:00.270
Locale Setting: C | 2020-12-14 21:27:00.270
DLLs: [[i]I removed this for brevity[/i]]
Allowed protected custom studies: SST_FreeBundle | 2020-12-14 21:27:00.272

HGH21 1 Trades #1 | Reloading chart. | 2020-12-14 21:27:21.934
Mapping HGH21 to HGH21. Service code: comex | 2020-12-14 21:27:22.670
SC Data - All Services | Starting real-time market data updates for: HGH21 (HGH21). ID: 1 Service code: comex | 2020-12-14 21:27:22.694
SC Data - All Services | Connecting to the server ds12-2.sierracharts.com. Port 443 | 2020-12-14 21:27:22.712
SC Data - All Services | Requesting security definition data for: HGH21 (HGH21). ID: 1 | 2020-12-14 21:27:22.713
Requesting market depth updates for: HGH21 if supported. | 2020-12-14 21:27:22.713
Separate subscription for market depth updates not supported. HGH21 | 2020-12-14 21:27:22.713
Starting DelayedDownloadMonitoringTimer | 2020-12-14 21:27:22.713
Starting HDTimeoutTimer | 2020-12-14 21:27:22.713
Added historical Intraday data request for HGH21 to the queue. | 2020-12-14 21:27:22.713
Intraday data recording state for symbol HGH21 is set to download 'Pending'. | 2020-12-14 21:27:22.713
Triggering next historical data download in queue. | 2020-12-14 21:27:22.713
Delaying start of download for HGH21 | 2020-12-14 21:27:22.717
DTC Client socket (1) | Creating socket. | 2020-12-14 21:27:22.845
DTC Client socket (1) | New receive buffer size: 5242880 | 2020-12-14 21:27:22.845
DTC Client socket (1) | Connecting to IP: 204.15.197.13. | 2020-12-14 21:27:22.845
SC Data - All Services | Network connection to server complete. | 2020-12-14 21:27:22.899
SC Data - All Services | Starting socket receive thread. | 2020-12-14 21:27:22.920
SC Data - All Services | Sending encoding request to server: Binary VLS. Compression: standard | 2020-12-14 21:27:22.929
SC Data - All Services | Setting DTC encoding to Binary VLS | 2020-12-14 21:27:22.979
SC Data - All Services | Sending logon request message. | 2020-12-14 21:27:22.979
SC Data - All Services | Received logon response. | 2020-12-14 21:27:23.793
SC Data - All Services | Server Name: SC Realtime Server. | 2020-12-14 21:27:23.793
SC Data - All Services | Server protocol version: 8. Client protocol version: 8 | 2020-12-14 21:27:23.793
SC Data - All Services | Successfully connected. | 2020-12-14 21:27:23.793
SC Data - All Services | Trading is not supported. | 2020-12-14 21:27:23.793
Real-time market depth file updates started for HGH21 | 2020-12-14 21:27:23.808
Opened cached Depth file: C:\SierraChart\Data\MarketDepthData\HGH21.2020-12-15.depth | 2020-12-14 21:27:23.809
SC Data - All Services | Restarting real-time data updates for symbol: HGH21 -> HGH21. Market Depth data restarted. | 2020-12-14 21:27:23.809
SC Data - All Services | Clearing market depth for HGH21 | 2020-12-14 21:27:23.919
Enabling compression for file: C:\SierraChart\Data\MarketDepthData\HGH21.2020-12-15.depth. This may take a while. | 2020-12-14 21:27:24.049
File compression enabled. C:\SierraChart\Data\MarketDepthData\HGH21.2020-12-15.depth. CompressionState=2 | 2020-12-14 21:27:24.049
Opened cached Depth file: C:\SierraChart\Data\MarketDepthData\HGH21.2020-12-15.depth | 2020-12-14 21:27:24.049
SC Data - All Services | Received security definition for symbol HGH21 | 2020-12-14 21:27:24.049
Loading market depth data. | 2020-12-14 21:27:25.091
Loaded depth data records for HGH21 2020-12-11 from 2020-12-11 14:13:21.920000 to 2020-12-11 14:29:29.384270. | 2020-12-14 21:27:25.169
Loaded depth data records for HGH21 2020-12-14 from 2020-12-14 07:13:34.187000 to 2020-12-14 08:53:47.017000. | 2020-12-14 21:27:26.188
Market Depth Historical Graph | Current service package does not support this function. Select 'Help >> Account Control Panel' to change your package. | 2020-12-14 21:27:26.391
Market Depth Historical Graph | Current service package does not support this function. Select 'Help >> Account Control Panel' to change your package. | 2020-12-14 21:27:26.407
Market Depth Historical Graph | Current service package does not support this function. Select 'Help >> Account Control Panel' to change your package. | 2020-12-14 21:27:27.375
HD Request # 1 | Downloading Intraday chart data for HGH21 to the file HGH21.scid. Service: comex | 2020-12-14 21:27:28.793
HD Request # 1 | Download start date-time: 2020-12-14 08:53:47.000000. File last date-time: 2020-12-14 08:53:47.017002 | 2020-12-14 21:27:28.797
HD Request # 1 | Using server: ds23.sierracharts.com port 10150 | 2020-12-14 21:27:28.797
Socket (2) | Creating socket. | 2020-12-14 21:27:28.865
Socket (2) | New receive buffer size: 5242880 | 2020-12-14 21:27:28.865
Socket (2) | Connecting to IP: 216.1.90.130. | 2020-12-14 21:27:28.865
HD Request # 1 | Setting DTC encoding to Binary VLS | 2020-12-14 21:27:28.976
HD Request # 1 | Sending historical data logon request message. | 2020-12-14 21:27:28.976
HD Request # 1 | Requesting Intraday data. Start date-time: 2020-12-14 08:53:47. Record interval: 0. Symbol: HGH21 | 2020-12-14 21:27:29.475
HD Request # 1 | Decompressing data. | 2020-12-14 21:27:29.599
HD Request # 1 | Receiving Intraday data for HGH21 starting at 2020-12-14 08:53:47.017000 | 2020-12-14 21:27:29.740
Socket (2) | CloseSocket call. | 2020-12-14 21:27:29.853
Socket (2) | 0 bytes remaining in receive buffer when socket closed. | 2020-12-14 21:27:29.853
Socket (2) | Shutdown started. Waiting for graceful close. | 2020-12-14 21:27:29.853
Historical data download thread signaled to stop. | 2020-12-14 21:27:29.876
HD Request # 1 | Timestamp of first Intraday data file record written: 2020-12-14 08:53:49.662000 | 2020-12-14 21:27:29.877
HD Request # 1 | Received 19742 records from 2020-12-14 08:53:47.017000 to 2020-12-14 21:27:25.525000 (12.6 hours) and wrote 19739 records for HGH21 | 2020-12-14 21:27:29.877
HD Request # 1 | Add time&sales data | Remembered last file record date-time: 2020-12-15 03:27:25.525000 | 2020-12-14 21:27:29.893
Added 5 Time and Sales records to Intraday data file for HGH21 after download. | 2020-12-14 21:27:29.893
Socket (2) | Socket gracefully closed by remote side. | 2020-12-14 21:27:29.899
Socket (2) | Closed. | 2020-12-14 21:27:29.899
HD Request # 1 | Completion time: 1s | 2020-12-14 21:27:29.906
HD Request # 1 | Intraday data download complete for HGH21. Unique request ID: 1 | 2020-12-14 21:27:29.906
Removing historical data download ID 1. | 2020-12-14 21:27:29.906
Real-time Intraday chart data file updates started for HGH21 | 2020-12-14 21:27:29.906
Intraday chart data file opened for HGH21 | 2020-12-14 21:27:29.906
HD Request # 1 | Enabling Intraday chart updating for symbol. | 2020-12-14 21:27:29.906

Triggering next historical data download in queue. | 2020-12-14 21:27:29.906
Opened cached Intraday file: C:\SierraChart\Data\HGH21.scid. Thread ID: 60600 | 2020-12-14 21:27:35.086
The message log has been sent to support. | 2020-12-14 21:34:10.786
HTTPS connection ID:1 to www.sierrachart.com:443 for request ID:1 (2) | Creating socket. Using TLS 1.2. | 2020-12-14 21:34:10.786
HTTPS connection ID:1 to www.sierrachart.com:443 for request ID:1 (2) | New receive buffer size: 32768 | 2020-12-14 21:34:10.786
HTTPS connection ID:1 to www.sierrachart.com:443 for request ID:1 (2) | Connecting to IP: 46.4.97.203. | 2020-12-14 21:34:10.791
Sending of the Message Log is completed. The response was saved to C:\SierraChart\Data\MessageLogAnalysis.htm | 2020-12-14 21:34:11.846
HTTPS connection ID:1 to www.sierrachart.com:443 for request ID:1 (2) | CloseSocket call. | 2020-12-14 21:34:12.091
HTTPS connection ID:1 to www.sierrachart.com:443 for request ID:1 (2) | 0 bytes remaining in receive buffer when socket closed. | 2020-12-14 21:34:12.091
HTTPS connection ID:1 to www.sierrachart.com:443 for request ID:1 (2) | Shutdown started. Waiting for graceful close. | 2020-12-14 21:34:12.091
HTTPS connection ID:1 to www.sierrachart.com:443 for request ID:1 (2) | Socket gracefully closed by remote side. | 2020-12-14 21:34:12.231
HTTPS connection ID:1 to www.sierrachart.com:443 for request ID:1 (2) | Closed. | 2020-12-14 21:34:12.231

Steps I took before copying log:
1) launch SC & clear Message Log
2) open Chartbook with 1 Chart window - intraday tick chart of HGH21 (COMEX Copper Mar '21), 15 days to load per Chart Settings
3) scrolled to earliest time in Chart, R clicked and selected 'Download Depth Data from Date-Time'
4) waited 5 minutes to see if Message Log reported anything pertaining to Step (3)
Date Time Of Last Edit: 2020-12-15 03:38:18
[2020-12-15 04:43:33]
Sierra Chart Engineering - Posts: 104220
Ok we see what the problem is. You need to translate the symbols to the new format. Instructions:
Edit Menu: Edit >> Translate Symbols To Current Service

And you can rename the existing market depth files to the new symbol format. The files are located in:
C:\SierraChart\Data\MarketDepthData\
Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service:
Sierra Chart Teton Futures Order Routing
Date Time Of Last Edit: 2020-12-15 04:44:22
[2020-12-17 00:54:43]
nosast - Posts: 246
Any update on your post #27 regarding a separate setting to control the days of market depth download?
[2020-12-17 03:39:37]
Sierra Chart Engineering - Posts: 104220
Yes that setting will be in the new Data/Trade Service Settings window which we are working on. Should be out maybe in about a week or so.
Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service:
Sierra Chart Teton Futures Order Routing
[2020-12-17 19:19:11]
User502498 - Posts: 33
Will this ability to download market depth have any impact on having the pullstack data avail for backtesting? Right now all pullstack's are 0 if you do ANY modifications to the chart they are working on.
[2020-12-17 20:21:44]
Chad - Posts: 213
Supposing I run a Chartbook once per week, which contains several Charts/symbols that I want to keep accumulating historical depth data for - will all of the historical depth of the interim days be auto-downloaded/backfilled, or do I need to scroll back in each Chart to the last time the Chartbook was open, and select 'Download Depth Data from Date-Time'?

Perhaps this is in the same vein of inquiry as post #36...
[2020-12-17 22:30:49]
Sierra Chart Engineering - Posts: 104220
We need specific instructions to reproduce this:
Right now all pullstack's are 0 if you do ANY modifications to the chart they are working on.
We would not expect this to be the case.

will all of the historical depth of the interim days be auto-downloaded/backfilled,
Yes.
Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service:
Sierra Chart Teton Futures Order Routing
[2020-12-25 15:22:32]
User624476 - Posts: 30
Hello Sierra Chart Engineering,

Any idea when you will offer access to historical market depth data further back than 12/10/2020?
[2021-01-02 14:38:26]
nosast - Posts: 246
Regarding #36 and the settings to limit days of downloading market depth I can't find any valid setting therefore. Is this still in development?
[2021-01-03 22:27:10]
Sierra Chart Engineering - Posts: 104220
This has not yet been released. We will get this out as soon as we can.
Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service:
Sierra Chart Teton Futures Order Routing
[2021-01-31 01:53:57]
User100912 - Posts: 67
@ticinotrader I've fouled up and lost some historical depth data.

Do you perhaps have NYMEX or ICE crude oil futures from April 20, 2020 in SC market depth format?

I cannot download this as it is older than 6 months therefore deleted from SC servers.
[2021-06-13 16:32:42]
User257608 - Posts: 4
Hi, I had a similar issue - https://www.sierrachart.com/SupportBoard.php?ThreadID=64991#P269165

Do you allow this for XBT (bitcoin futures) from bitmex - I can't seem to download histroical data - only does it when Im connected to the feed
[2021-06-13 19:35:02]
Sierra Chart Engineering - Posts: 104220
There is no historical market depth data provided for BitMex symbols.
Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service:
Sierra Chart Teton Futures Order Routing
[2021-10-02 20:41:16]
User190920 - Posts: 1
Hello, would you please answer 2 questions for me?

1. Does historical depth data support nyse/nasdaq stocks?

2. Does DTC protocol support historical depth data download? Or plan to support?

Thanks!
[2021-10-03 17:15:55]
Sierra Chart Engineering - Posts: 104220
1. Not at this time but we will look at that functionality, for our new upcoming NASDAQ data feed.

2. Yes it does, but it is not yet implemented in Sierra Chart.
Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service:
Sierra Chart Teton Futures Order Routing
[2021-10-16 14:38:48]
User643553 - Posts: 1
Hello,

I found depth files are inconsistent between downloads.

Here's the details:

File: ESZ21-CME.2021-10-01.depth
First download date: Oct 2, 2021
Redownload again on: Oct 16, 2021

Let's call the first download v1 and the redownload v2.

Here's the difference:

File size: v1 = 335,496,712 bytes, v2 = 335,462,080 bytes

Are you updating those depth files after the trading date? and Why?

Thanks.
[2021-10-17 18:31:38]
Sierra_Chart Engineering - Posts: 3060
Potentially there can be differences compared to what is downloaded from the server as historical data and what is recorded in real time. Why there would be a difference, is not something we would look into but it would not surprise us. The files are not changed on the server.
Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

For the most reliable, advanced, and zero cost futures order routing, use the Teton service:
Sierra Chart Teton Futures Order Routing

To post a message in this thread, you need to log in with your Sierra Chart account:

Login

Login Page - Create Account