Login Page - Create Account

Support Board


Date/Time: Fri, 29 Mar 2024 13:47:38 +0000



Your Market Depth Studies have a MAJOR flaw!

View Count: 2220

[2022-05-08 18:44:22]
User839162 - Posts: 82
I’m not exactly sure what’s all involved but can we please get real-time continuously updating depth studies?

As of right now the Market Depth Historical Graph and Bid Ask Depth Bars do not display/update in a continuous fashion (like the DOM). They update when either a new bar is created or when an order is executed. Let me explain why this is a MAJOR issue. Let's say I'm trading a slow instrument (ZN). Price trades up and stalls. No orders are being executed and a new candle has not printed, BUT a few prices away someone puts 500 on the bid. IF that bid of 500 disappears before an order is executed or a new candle is created, it will never be shown on the Depth Graph. Therefor the Depth Graph is showing incorrect data. This data will also be missing from the bid ask depth bars.

I thought I could somewhat get around this problem by using 1 second bars BUT THOSE DON’T PRINT EVERY SECOND?? Your 1 second bars will not print if no orders are executing. Why?

Without the fluidity of depth in ZN it causes bursts of action and sudden changes in depth that can’t be seen before it up ticks or down ticks. All of your depth studies are inaccurate if they are not calculated in real-time and it especially hinders us who use these studies in slow instruments.

PLEASE fix this.
[2022-05-08 19:55:17]
Sierra Chart Engineering - Posts: 104368
Refer to the documentation here regarding this:
Market Depth Historical Graph: Market Depth Historical Graph Timing in Relation to Chart Bar Date-Times

This is not really true like you say:
They update when either a new bar is created or when an order is executed.

The specific behavior is documented here:
Market Depth Historical Graph: Market Depth Historical Graph Timing in Relation to Chart Bar Date-Times

IF that bid of 500 disappears before an order is executed or a new candle is created, it will never be shown on the Depth Graph.
This is not necessarily true. It depends upon the Input settings and whether this was a maximum depth value at the price level.

This is definitively not true at all:
This data will also be missing from the bid ask depth bars.

The data is absolutely not missing. It is processed and displayed but assigned to the appropriate chart bar when that bar is available.

The documentation explains all of this. And the solution.
Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service:
Sierra Chart Teton Futures Order Routing
[2022-05-08 22:59:47]
User839162 - Posts: 82
I followed that first link you posted and added the -BID_ASK_TRADE_SYNC to the ZN symbol. It looks exactly like this in my symbol settings (ZNM22-CBOT-BID_ASK_TRADE_SYNC). I then followed the instructions underneath it to create a 1t chart. I'm still seeing the depth graph not matching up with the DOM.

Please explain to me how I can see the Depth Graph update in real-time with every change of depth? Everything I've tried does not work and I see differences between the two.

I'm attaching an image so you can see that the Data Recording Suffixes do the same thing.
Private File
[2022-05-08 23:44:46]
User839162 - Posts: 82
I also don't understand why you would make this thread hidden? Plain and simple the Heatmap does not match the DOM.

This is not necessarily true. It depends upon the Input settings and whether this was a maximum depth value at the price level.

I've witnessed levels on the heatmap that are showing more than what the maximum depth value is and it doesn't update if that current depth value drops below before an order is executed or a candle is created. This is a problem.
Date Time Of Last Edit: 2022-05-08 23:47:19
[2022-05-09 14:13:43]
Sierra Chart Engineering - Posts: 104368
In the latest prerelease, we have made a change to -BID_ASK_TRADE_SYNC to also record data when there is a change in bid/ask quantities. This should solve the problem.

We have not yet tested this but will be testing it today.

We are also checking on another thing but we do not expect to find any further issue.
Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service:
Sierra Chart Teton Futures Order Routing
[2022-05-10 14:40:51]
Sierra_Chart Engineering - Posts: 13625
The functionality that you are referring to is fully documented here:
Market Depth Historical Graph: Market Depth Historical Graph Timing in Relation to Chart Bar Date-Times


There is a solution by using a special data recording which is documented in the section linked above.

Here is an example of this:
https://www.sierrachart.com/image.php?Image=1652192303728.png

You must be running the very latest release which is 2393. This recording mode has been enhanced for this specific purpose.

I also don't understand why you would make this thread hidden?
Since you are making statements which are actually not technically true in some cases and asserting them as if they are true.

The fundamental issue is you are using a very short timeframe per bar. If the timeframe per bar were like 10 minutes, you would not notice any problem. Other than at the end of 10 minutes for a short duration.

The release that we did previously, in 2392 related to this recording suffix for the Market Depth Historical Graph still did not work properly. It does now and you need to run 2393 or higher.
Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

For the most reliable, advanced, and zero cost futures order routing, use the Teton service:
Sierra Chart Teton Futures Order Routing
Date Time Of Last Edit: 2022-05-10 14:46:04
[2022-05-10 16:27:37]
User839162 - Posts: 82
It's working great now. I have one other question.

With this update it has made the depth bars look much different because it's recording the depth the entire time (which is good). But I did notice that when I Download Depth Data From Date-Time it will download the "old" depth that doesn't continuously update. Is there any way to download the depth for the -BID_ASK_TRADE_SYNC symbol that continuously updates?
[2022-05-10 17:21:30]
Sierra Chart Engineering - Posts: 104368
No this is not supported:
Is there any way to download the depth for the -BID_ASK_TRADE_SYNC symbol that continuously updates?

Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service:
Sierra Chart Teton Futures Order Routing
[2022-05-10 17:39:00]
User839162 - Posts: 82
No this is not supported:
Is there any way to download the depth for the -BID_ASK_TRADE_SYNC symbol that continuously updates?

Please consider this in the future. It would be nice to be able to backtest the correct depth data.
[2022-05-11 07:37:01]
User311344 - Posts: 136
How does this issue relate to Denali Delayed Data?

I get that when it was downloaded from Date-Time it doesnt have continous updates, but does it have continuous updates when being pulled directly from stream (as delayed data)?
Date Time Of Last Edit: 2022-05-11 08:40:10
[2022-05-11 17:14:23]
Sierra Chart Engineering - Posts: 104368
The Market Depth Historical Graph does work with the Delayed Exchange Data Feed:
Delayed Exchange Data Feed

Everything will be identical except the data will be delayed 10 to 15 minutes.

If you are seeing something different, we need to know.
Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service:
Sierra Chart Teton Futures Order Routing
[2022-05-12 11:16:08]
User311344 - Posts: 136
You said that BID_ASK_TRADE_SYNC switch is not supported when data being downloaded with Download Depth Data from Date/Time so Im asking if it's supported for Denali DELAYED Data or ONLY for LIVE Denali data?
Date Time Of Last Edit: 2022-05-12 11:17:10
[2022-05-12 11:25:24]
Sierra Chart Engineering - Posts: 104368
The historical data downloaded in no case is going to use this type of recording mode:
BID_ASK_TRADE_SYNC
Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service:
Sierra Chart Teton Futures Order Routing
[2022-05-13 06:39:01]
User311344 - Posts: 136
The historical data downloaded in no case is going to use this type of recording mode:
BID_ASK_TRADE_SYNC

Yes but what about DELAYED Denali data being pulled from datafeed? Would it use this type of recording and look EXACTLY the same as true LIVE Denali data?
[2022-05-13 07:17:16]
Sierra Chart Engineering - Posts: 104368
Yes.
Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service:
Sierra Chart Teton Futures Order Routing
[2023-01-17 03:14:23]
voided - Posts: 45
If I'm reading this thread correctly, not all state changes in the order book are written to the .depth files? I have recently been using the .depth files, and noticed that there are many missing records.
[2023-01-17 03:57:50]
Sierra_Chart Engineering - Posts: 13625
No that is not true:
not all state changes in the order book are written to the .depth files?

Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

For the most reliable, advanced, and zero cost futures order routing, use the Teton service:
Sierra Chart Teton Futures Order Routing
[2023-01-18 00:34:06]
voided - Posts: 45
I must be making a mistake, then. I have been testing my program with CLG23_FUT_CME on the .depth file for January 11th:

CLG23_FUT_CME.2023-01-11.depth

I read and parse the depth records. After reading your post, I checked the timestamps, printing each records timestamp, in the sequence read from disk, along with the duration (in seconds) since the previous record. Several had negative durations:

-----

3882564794899000  -0.393
3882573483952000  -1.700
3882579410123000  -0.026
3882580289013000  -0.644
3882595486811000  -0.027
3882596068603000  -0.129
3882597650570000  -0.877
3882599583655000  -0.137
3882601154027000  -0.063
3882602609175000  -0.165
3882604751565000  -0.019
3882605352708000  -0.081
3882606215738000  -0.026
3882607331439000  -0.030
3882608099468000  -0.008
3882609234477000  -0.022
3882612036599000  -0.094
3882612583541000  -0.023
3882613812842000  -0.003
3882617143484000  -0.006
3882617437276000  -0.008
3882619084757000  -0.137
3882621050867000  -0.555
3882621598414000  -0.237
3882622719117000  -0.007
3882623830650000  -0.024
3882625978142000  -0.014
3882627429297000  -0.198
3882628440203000  -0.069
3882628942626000  -0.617
3882629886540000  -0.415
3882631069988000  -0.427
3882631356149000  -0.110
3882633649364000  -0.099
3882634415297000  -3.419
3882634950445000  -2.342
3882636000080000  -305.156

-----

I also noticed some extremely large gaps, during which the intraday .scid files recorded many trades. A full list of depth records with more than 10 seconds since the prior record follows:

-----

3882614052330951 164.757
3882615577679513 1379.228
3882616627764529 974.451
3882617143489678 428.746
3882617437284224 206.422
3882619084893547 1565.991
3882620012736179 850.890
3882621051421557 963.978
3882621598650844 459.398
3882622719124114 1022.769
3882623830674489 1042.644
3882624178089245 262.180
3882625978155526 1717.477
3882627429495371 1366.029
3882628053722469 476.641
3882628440272371 310.200
3882628943243471 403.628
3882629886955481 856.914
3882631070415447 1100.475
3882631356258940 136.383
3882633649463388 2218.236
3882634418715692 668.591
3882634952786561 400.066
3882635862949969 832.561
3882636305235979 321.985
3882638700426000 2700.346
3882638772532000 48.775
3882638790394000 17.862
3882638830344000 39.950
3882638869399000 39.055
3882638893349000 21.514
3882638938351000 45.002
3882638963351000 25.000
3882638975707000 12.356
3882639000351000 18.000
3882639019352000 19.001
3882639037352000 18.000
3882639071353000 31.000
3882639104019000 26.665
3882639121203000 15.088
3882639166356000 17.001
3882639183356000 13.000
3882639195200000 11.844
3882639247357000 47.733
3882639262358000 15.001
3882639277358000 11.000
3882639298358000 16.000
3882639319021000 15.000
3882639332101000 13.080
3882639367022000 15.000
3882639582009000 19.769
3882640234350000 10.550
3882640949746000 12.867
3882641494432000 10.265
3882641965455000 12.540
3882642662214000 12.811
3882642742091000 10.002
3882642946160000 16.128
3882642960183000 14.023
3882642998780000 16.471

-----

As you can see, there are gaps of up to 2700 seconds (45 minutes) between depth records.

So my questions are:

1. Are the records not guaranteed to be in sequence?
2a. Why are there such long intervals between some of the records?
2b. If these intervals indicate missing data, is there any way to obtain it?

Regarding 2b, I tried right-clicking on an intraday chart for CLG23_FUT_CME from a point prior to Jan 11th but within the 30 day window (i.e., January 2nd 2023, 15:00) and selecting "Download Depth Data from Date-time," but nothing changed in my depth files. I had deleted files prior to January 6th, for example, and they were not recreated. None of the other existing depth files past that point changed, either.

Here are the entries in the message log subsequent to selecting "Download Depth Data from Date-time":

-----

Triggering next historical data download in queue. | 2023-01-17 17:06:57.787
No download requests in the queue to start downloads for. | 2023-01-17 17:06:57.787
CLG23_FUT_CME[M] 30 Min #18 | Reloading chart. | 2023-01-17 17:06:57.827
Socket (5) | Socket gracefully closed by remote side. | 2023-01-17 17:06:57.830
Socket (5) | Closed. | 2023-01-17 17:06:57.831
CLG23_FUT_CME[M] #2 | Reloading chart. | 2023-01-17 17:06:57.835
CLG23_FUT_CME[M] #2 | StartDateTimeForLoadingOrderFills: 00:00:00 | 2023-01-17 17:06:57.913
Opened cached Depth file: C:\SierraChart\Data\MarketDepthData\CLG23_FUT_CME.2023-01-18.depth | 2023-01-17 17:06:58.976
DTC Client socket (2) | Timed out waiting for shutdown. Performing an immediate socket close. | 2023-01-17 17:07:10.254
DTC Client socket (2) | Closed. | 2023-01-17 17:07:10.254
Opened cached Intraday file: C:\SierraChart\Data\NGH23-NGJ23.FUT_SPREAD.CME.scid. Thread ID: 1584 | 2023-01-17 17:07:24.332
DTC Client socket (2) | Deleting network socket now after delay. | 2023-01-17 17:07:40.273
Added historical Market depth data request for CLG23_FUT_CME to the queue. | 2023-01-17 17:08:21.672
Market depth data recording state for symbol CLG23_FUT_CME is set to download 'Pending'. | 2023-01-17 17:08:21.672
Triggering next historical data download in queue. | 2023-01-17 17:08:21.672
HD Request # 168 | Downloading Market depth data for CLG23_FUT_CME. Service: nymex.marketdepth. | 2023-01-17 17:08:21.673
HD Request # 168 | Download start date-time: 2023-01-12 17:08:21.000000. | 2023-01-17 17:08:21.682
HD Request # 168 | Using server: ds28.sierracharts.com port 10149 | 2023-01-17 17:08:21.683
Socket (2) | Creating socket. | 2023-01-17 17:08:21.683
Socket (2) | New receive buffer size: 5242880 | 2023-01-17 17:08:21.683
Socket (2) | Connecting to IP: 142.215.18.147. | 2023-01-17 17:08:21.683
HD Request # 168 | Setting DTC encoding to Binary VLS | 2023-01-17 17:08:21.786
HD Request # 168 | Sending historical data logon request message. | 2023-01-17 17:08:21.786
HD Request # 168 | Requesting market depth data. Start date-time: 2023-01-12 17:08:21. Symbol: CLG23 | 2023-01-17 17:08:22.365
HD Request # 168 | Decompressing data. | 2023-01-17 17:08:22.445
HD Request # 168 | Receiving market depth data for CLG23_FUT_CME starting at 2023-01-12 17:08:21.000000 | 2023-01-17 17:08:22.456
HD Request # 168 | Truncating file to starting date-time of 2023-01-12 17:08:21.000000. | 2023-01-17 17:08:22.456
HD Request # 168 | Timestamp of first market depth data file record written: 2023-01-12 17:08:21.000000. | 2023-01-17 17:08:22.667
Socket (2) | CloseSocket call. | 2023-01-17 17:08:54.955
Historical data download thread signaled to stop. | 2023-01-17 17:08:54.955
Socket (2) | Shutdown started. Waiting for graceful close. | 2023-01-17 17:08:54.955
HD Request # 168 | Received 16337566 Market depth data records from 2023-01-12 17:08:21.000000 to 2023-01-17 17:08:51.521000 (5.0 days) and wrote 16337566 records for CLG23_FUT_CME | 2023-01-17 17:08:54.955
HD Request # 168 | Completion time: 33s. | 2023-01-17 17:08:54.955
HD Request # 168 | Market depth data download complete for CLG23_FUT_CME. Unique request ID: 168 | 2023-01-17 17:08:54.955
Removing historical data download ID 168. | 2023-01-17 17:08:54.955
Real-time market depth file updates started for CLG23_FUT_CME | 2023-01-17 17:08:54.955
Opened cached Depth file: C:\SierraChart\Data\MarketDepthData\CLG23_FUT_CME.2023-01-18.depth | 2023-01-17 17:08:54.961
CLG23_FUT_CME[M] #2 | Flagging to reload Intraday data on market depth download completion. | 2023-01-17 17:08:54.961
CLG23_FUT_CME[M] 30 Min #18 | Flagging to reload Intraday data on market depth download completion. | 2023-01-17 17:08:54.961

Triggering next historical data download in queue. | 2023-01-17 17:08:54.961
No download requests in the queue to start downloads for. | 2023-01-17 17:08:54.961
Socket (2) | Socket gracefully closed by remote side. | 2023-01-17 17:08:55.005
Socket (2) | Closed. | 2023-01-17 17:08:55.005
CLG23_FUT_CME[M] #2 | Reloading chart. | 2023-01-17 17:08:55.009
CLG23_FUT_CME[M] 30 Min #18 | Reloading chart. | 2023-01-17 17:08:55.017
CLG23_FUT_CME[M] #2 | StartDateTimeForLoadingOrderFills: 00:00:00 | 2023-01-17 17:08:55.088
Opened cached Depth file: C:\SierraChart\Data\MarketDepthData\CLG23_FUT_CME.2023-01-18.depth | 2023-01-17 17:08:55.474
Opened cached Intraday file: C:\SierraChart\Data\CLQ23-CLU23.FUT_SPREAD.CME.scid. Thread ID: 1584 | 2023-01-17 17:10:20.304


-----

It appears to have only touched the depth files for 1-18 and 1-17, despite files from 1-6 through 1-18 existing.

Thanks for your help.

edit:

And one related question. In order to maintain the correct order book state, is it enough to parse the depth records alone? Or do trades need to be subtracted from the existing quantities at each level? I was under the assumption that every change to the quantity at each level would come from the add/mod/delete depth record commands, but if I also need to subtract the trades, that would be nice to know.
Date Time Of Last Edit: 2023-01-18 01:12:50
[2023-01-18 03:14:23]
Sierra_Chart Engineering - Posts: 13625
1. Yes they are in sequence.

2. We can only say to make sure there is no missing data by re-download the data:
Market Depth Historical Graph: Downloading Of Historical Market Depth Data

It appears to have only touched the depth files for 1-18 and 1-17, despite files from 1-6 through 1-18 existing.
Check this setting:
Sierra Chart Server Settings: Maximum Historical Market Depth Days to Download (Global Settings >> Sierra Chart Server Settings >> General >> General)


What actually is recorded is the market depth data stream to Sierra Chart in real time. And periodically there is also a snapshot recorded. Like every 10 or 15 minutes. That snapshot is not sent to Sierra Chart. Maybe you are seeing slightly out of order timestamps related to the snapshot. We would have to check on that. But just ignore the timestamps with the snapshot data.

Several had negative durations:
We do not see this at all. Those numbers you list are in ascending order. And how do you get a fractional number from an integer?


We checked this file across all three servers that provide this data and they have the exact data on all three:
CLG23_FUT_CME.2023-01-11.depth

Yes:
And one related question. In order to maintain the correct order book state, is it enough to parse the depth records alone?

Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

For the most reliable, advanced, and zero cost futures order routing, use the Teton service:
Sierra Chart Teton Futures Order Routing
Date Time Of Last Edit: 2023-01-18 03:16:41
[2023-01-18 05:43:32]
voided - Posts: 45
Thank you for the response. I will check out the links.


We do not see this at all. Those numbers you list are in ascending order. And how do you get a fractional number from an integer?

Sorry for the confusion. The timestamps I printed are only for the records that had negative duration since the prior record -- I omitted the rest of the stream. These records appear at various places throughout the file. As for the fractional durations (second column), these are in seconds. So, I divided the microsecond durations by one million.

Here is the context surrounding the record that appeared at 3882636000080000, 305 seconds before the previous record. Looking at the data again, it actually looks like the record at 3882636305235979 was the one out of sequence.

-----

3882635982385000 0.037
3882635982385000 0.000
3882635982385000 0.000
3882635982385000 0.000
3882635982385000 0.000
3882635982385000 0.000
3882635982385000 0.000
3882635982385000 0.000
3882635982385000 0.000
3882635982386000 0.001
3882635982386000 0.000
3882635982387000 0.001
3882635982457000 0.070
3882635983230000 0.773
3882635983251000 0.021
3882636305235979 321.985
3882636000080000 -305.156
3882636000080000 0.000
3882636000080000 0.000
3882636000080000 0.000
3882636000080000 0.000
3882636000080000 0.000
3882636000080000 0.000
3882636000080000 0.000
3882636000080000 0.000
3882636000080000 0.000
3882636000080000 0.000
3882636000080000 0.000
3882636000080000 0.000
3882636000080000 0.000
3882636000080000 0.000
3882636000080000 0.000
3882636000080000 0.000

-----

And here is a portion of the file showing a gap of 2700 seconds between two records:

-----

3882636000080000 0.000
3882636000080000 0.000
3882636000080000 0.000
3882636000080000 0.000
3882636000080000 0.000
3882636000080000 0.000
3882636000080000 0.000
3882636000080000 0.000
3882636000080000 0.000
3882636000080000 0.000
3882636000080000 0.000
3882636000080000 0.000
3882636000080000 0.000
3882636000080000 0.000
3882636000080000 0.000
3882638700426000 2700.346
3882638700426000 0.000
3882638700426000 0.000
3882638700426000 0.000
3882638700426000 0.000
3882638700426000 0.000
3882638700426000 0.000
3882638700426000 0.000
3882638700426000 0.000
3882638700426000 0.000
3882638700426000 0.000
3882638700426000 0.000
3882638700426000 0.000

-----

Perhaps this has to do with the snapshot record you mentioned. I will take a closer look at it.

If you would like to see the whole file, which shows every record in the sequence I find it in the file, with timestamp in the first column, and the duration in seconds since the last record in the second column, I've uploaded it here: https://mega.nz/file/MO8VUCiY#Cy2XkKFHWR7mFpdIGFSx1vTosSckzHynpIg8phbyp44

Edit:

Also, how do i identify a snapshot? Is it something to do with the FLAG_END_OF_BATCH field?
Date Time Of Last Edit: 2023-01-18 06:21:46
[2023-01-18 22:51:23]
voided - Posts: 45
Okay, after following your instructions to increase the "Max Historical Market Depth Days to Download" setting from its prior value (5) to 30, I was able to re-download the file for 2023-01-11. It increase from ~50 MB to ~180 MB. There are no more negative durations (records out of sequence). There aren't many > 10 second gaps either; however the 2700 second gap is still there.

It appears that is due to the trading halt:

-----

depth 3882636000080000 2023-01-11 22:00:00.080000 0.000
depth 3882636000080000 2023-01-11 22:00:00.080000 0.000
depth 3882636000080000 2023-01-11 22:00:00.080000 0.000
depth 3882636000080000 2023-01-11 22:00:00.080000 0.000
depth 3882636000080000 2023-01-11 22:00:00.080000 0.000
depth 3882638700426000 2023-01-11 22:45:00.426000 2700.346
depth 3882638700426000 2023-01-11 22:45:00.426000 0.000
depth 3882638700426000 2023-01-11 22:45:00.426000 0.000
depth 3882638700426000 2023-01-11 22:45:00.426000 0.000
depth 3882638700426000 2023-01-11 22:45:00.426000 0.000

-----

So, tentatively it looks like everything is good. I am still wondering why the file had missing / out of order records before, however, and how I can detect this and alert myself to re-download.

Scanning the file for out of sequence records seems like one way, since that is easy to check. But missing records is harder to tell.

I suppose for now my questions are all answered, except for how to detect records belonging to snapshot. Or any other hints about how to know records might be missing.

Thanks again for your help.
Date Time Of Last Edit: 2023-01-18 23:06:34
[2023-01-22 00:28:57]
voided - Posts: 45
The depth files all seem to be OK after a re-download. However, I have encountered a new problem. Right-clicking on an intraday chart and selecting "Download Depth Data From Date-Time" does not work for spreads. Furthermore, I have tested the depth files for various calendar spreads that my instance recorded this week, and they all have missing records.

I have confirmed that "Global Settings >> Symbol Settings >> Record Market Depth Data" is set to "yes" for both "CL?##_FUT_CME" and "CL?##-CL?##.FUT_SPREAD.CME," for example. When I initiate a re-download from an intraday chart for CL, the message log shows messages about downloading depth files, and the files are updated accordingly. However, when I initiate a re-download from an intraday chart for a spread, nothing appears in the message log, and no data is downloaded.

I have the same setup in my chartbook for both CL and the CL spread: a 30-minute intraday chart and a DOM for each. After reviewing the documentation, I don't see any relevant setting that is different between the two.

Given that the spread files are initially incomplete and I do not have a way to re-download them, it seems there is no way to get accurate depth files for spreads. I have tested this across several symbols.
[2023-01-26 00:25:00]
Sierra_Chart Engineering - Posts: 13625
We will write up documentation for how to identify a snapshot of market depth data.

There is no historical market depth data maintained for spreads.
Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

For the most reliable, advanced, and zero cost futures order routing, use the Teton service:
Sierra Chart Teton Futures Order Routing
[2023-01-26 04:25:49]
voided - Posts: 45
Thanks for the response. I am primarily a spread trader, and would really appreciate depth data for spreads.

If my only option is to download the data in real time, that might be okay. However, in my testing the spread files are always missing records. Is it necessary to keep my Sierra Chart process connected and open at all times (and PC awake) to ensure it records every record?

I dont need an enormous amount of data, but even an option to download just the last 48 hours or so would be very helpful.
Date Time Of Last Edit: 2023-01-26 04:26:59
[2023-01-27 03:51:35]
Sierra_Chart Engineering - Posts: 13625
Here is the documentation to identify snapshots:
Market Depth Data File Format: Identifying Full Market Depth Data Snapshot

Is it necessary to keep my Sierra Chart process connected and open at all times (and PC awake) to ensure it records every record?
Yes.
Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

For the most reliable, advanced, and zero cost futures order routing, use the Teton service:
Sierra Chart Teton Futures Order Routing

To post a message in this thread, you need to log in with your Sierra Chart account:

Login

Login Page - Create Account