Login Page - Create Account

Support Board


Date/Time: Sat, 28 Nov 2020 02:39:13 +0000



[Sticky] [Locked] - Changes to Historical Intraday Data for CQG and Interactive Brokers (January 1, 2021)

[2020-11-19 22:54:26]
Sierra Chart Engineering - Posts: 91599 | Ending Date: 2021-04-05
Effective January 1, 2021:

Sierra Chart will no longer be providing Sierra Chart provided historical Intraday data when using the CQG and Interactive Brokers trading services. Historical Daily Data will still be provided.

In the case of CQG, there is a special service that we have created which maintains historical data for CQG. We will be shutting that service down at the above date. For various reasons this is not a service we are going to be continuing with or maintaining.

At that time, you will need to rely upon the historical data provided by CQG which only provides data for current contracts, not expired contracts. This data is also not processed on a background thread or as efficiently as Sierra Chart provided historical data.

Also, some brokers will charge 15-20 USD a month for this CQG provided historical data. In the case of AMP, they do not charge for this historical data from what we understand, but they do have to activate it on your CQG account.

In the case of Interactive Brokers we merge in the Sierra Chart Historical Intraday data with data from Interactive Brokers and we will no longer be offering that with Interactive Brokers after the above date.


The best solution for both of these cases is to use the Denali Exchange Data Feed and/or the Sierra Chart Exchange Data Feed for historical Intraday data and real-time data when using the above services:
https://www.sierrachart.com/index.php?page=doc/RealTimeDataFeedsAvailableFromSierraChart.php


We are also going to soon be releasing our real-time EUREX extended order book data feed as part of the Denali feed which also can be used for historical Intraday data when using these services.
Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. If possible please keep your questions brief and to the point. Please be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

If your question/request has been answered and you do not have anything further, then it is easiest for us if you do not reply again to say thank you.
Date Time Of Last Edit: 2020-11-20 00:37:04
[2020-11-24 21:09:25]
Sierra Chart Engineering - Posts: 91599 | Ending Date: 2021-04-05
More information regarding CQG historical data.

If you are with AMP our understanding is you do not have to pay for the CQG historical data. This has been confirmed.

If you are not with AMP, then if you are just needing historical data for the CME Group or EUREX you definitely will want to use the Sierra Chart Data Feeds for data instead:
https://www.sierrachart.com/index.php?page=doc/RealTimeDataFeedsAvailableFromSierraChart.php

When using the Sierra Chart provided data feeds, you will have access to 10 years of tick by tick history (more in some cases), including expired contracts which CQG does not provide, and have very fast and reliable and smooth data delivery. We would not recommend paying for CQG data if you have to pay for it since there are several disadvantages to using CQG historical data.

Therefore, when you are using either the Denali Exchange Data Feed and/or the Sierra Chart Exchange Data Feed, you will no longer need market data from CQG and would need in this case to cancel through your broker, the exchange fees you are paying for market data.

We also plan to offer a new EUREX data feed with full market depth across all price levels as part of the Denali Exchange Data Feed. We expect this will become available in January.

And this will actually be a requirement to use, for all users and CQG users who require EUREX market data, at some later point anyway.
Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. If possible please keep your questions brief and to the point. Please be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

If your question/request has been answered and you do not have anything further, then it is easiest for us if you do not reply again to say thank you.
Date Time Of Last Edit: 2020-11-27 05:23:45
[2020-11-26 20:22:55]
Sierra Chart Engineering - Posts: 91599 | Ending Date: 2021-04-05
These logs show the timing comparison between historical Intraday data downloaded from the Denali Exchange Data Feed and CQG. You can see the Denali Exchange Data Feed delivers data and is processed in 20.8% of the time as compared to CQG. That is dramatically faster.

----
Denali data feed (And also the subsidized historical data we provide CQG users now and in the past for CME group and EUREX data):
HD Request # 4 | Received 5036503 records from 2020-11-16 15:12:33.374000 to 2020-11-26 12:59:59.477000 (9.9 days) and wrote 5036503 records for ESZ20-CME | 2020-11-26 15:12:58.829
HD Request # 4 | Add time&sales data | Remembered last file record date-time: 2020-11-26 17:59:59.477000 | 2020-11-26 15:12:58.829
HD Request # 4 | Completion time: 25s | 2020-11-26 15:12:58.829
HD Request # 4 | Intraday data download complete for ESZ20-CME. Unique request ID: 3 | 2020-11-26 15:12:58.829

----
CQG (Historical data from CQG servers, which is not currently used by SC users unless you are using Asian exchanges):
HD Request # 9 | Received 5030866 records from 2020-11-16 15:17:26.072000 to 2020-11-26 12:59:59.480002 (9.9 days) and wrote 5030866 records for F.US.EPZ20 | 2020-11-26 15:19:29.005
HD Request # 9 | Add time&sales data | Remembered last file record date-time: 2020-11-26 17:59:59.480002 | 2020-11-26 15:19:29.005
HD Request # 9 | Completion time: 2m | 2020-11-26 15:19:29.005
HD Request # 9 | Intraday data download complete for F.US.EPZ20. Unique request ID: 6 | 2020-11-26 15:19:29.005


Two other things to be aware of with CQG historical data. The processing of this data is done on the Sierra Chart primary thread rather than a background thread which is used for Sierra Chart historical data services, like the Denali data feed uses.

Processing of this historical data from CQG on the primary thread will cause Sierra Chart to freeze during the processing of that historical data. When downloading and processing 1 minute data, there really is not any noticeable freezing. In the case of tick by tick data, you would notice freezing during the processing of that data. Normally this would be just a few seconds.

So this is a reason to use the Denali Exchange Data Feed for data. To avoid this freezing.

Additionally, CQG provides tick by tick data in reverse order. From latest to earliest. So that data has to be held in memory until all of the data is received. This will cause a temporary increase in memory use of Sierra Chart during the processing of this data. And that can be very substantial. It easily could be 100-200 MB or more.

Another consequence to CQG providing data in reverse order is there is a risk for downloads that take a long time to complete like two minutes that you could end up missing some data just after the end of the historical data provided and result in a small time gap in the chart. This may or may not be an issue because Sierra Chart will try to fill in this missing data to the extent time and sales data is available for that time period.

So these are reasons to use the Denali Exchange Data Feed for data when using CQG.

However, if you are satisfied with the historical and real-time data from CQG, then that is perfectly fine. Just continue to use that data.

With the problems described above with CQG data, Sierra Chart at most will download 5 days of tick by tick data from CQG. The remaining prior time period will be 1 minute data. Also keep in mind that CQG only provides historical data for current futures contracts. Not expired futures contracts.
Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. If possible please keep your questions brief and to the point. Please be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

If your question/request has been answered and you do not have anything further, then it is easiest for us if you do not reply again to say thank you.
Date Time Of Last Edit: 2020-11-27 09:58:41

To post a message in this thread, you need to login with your Sierra Chart account:

Login

Login Page - Create Account