Home >> Support Board

Support Board

Date/Time: Fri, 26 Apr 2024 14:53:35 +0000

How to Post a Message for Support

Discrepancies In .scid Files

View Count: 542

[2021-10-07 13:02:03] #1 [Go To First Post]
Alberto Gauer - Posts: 121
Hi there guys at Sierra Engineering. I noticed something weird, perhaps you can explain what causes this. I have Sierra configured to store 1 tick data for intraday and 2000 days. I started to download spot forex data for USD group to perform some tests. I set the start date to 2017-01-01 in all major USD currency pairs, I know that the file size for tick data would be huge, but I noticed some discrepancies in the size of the files for the same period. For example NZDUSD has a 3GB file size and EURUSD has 15GB for the same period of time. I thought that the probable cause would be more activity on this pair, ok. But I noticed something weird on GBPUSD when I started monitoring the file size as Sierra was downloading the data. At one point GBPUSD was at 10GB, then the download got stuck and wouldn´t complete. I chose to "delete and download data" and it started from zero again. This time it completed the download and the file size I got is 4,3GB, that is 6GB less than the previous try! I am tempted now to delete and redownload the EURUSD and see what I get. Is there any explanation for this much of discrepancy?

[2021-10-07 14:29:53] #2 [Go To First Post]
Sierra Chart Engineering - Posts: 104368
There can be more than one explanation for this. But one thing we have to check on our servers is if there is a difference between the data on the servers. There could be. Some servers may have data compressed starting at a different date compared to another server. We have to make sure they are all consistent. We will be checking on this over the weekend. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, change to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2021-10-07 14:30:18

[2021-10-07 15:52:06] #3 [Go To First Post]
Alberto Gauer - Posts: 121
There can be more than one explanation for this. But one thing we have to check on our servers is if there is a difference between the data on the servers. There could be. Some servers may have data compressed starting at a different date compared to another server. We have to make sure they are all consistent. We will be checking on this over the weekend. I see, interesting. Wouldn´t be better if the data was compressed for all .scid files and servers?

[2021-10-08 14:35:05] #4 [Go To First Post]
Alberto Gauer - Posts: 121
Just wanted to show another crazy example GBPUSD has 10GB less than GBPJPY for the same period of 2017-01-01 til today. And you can see above that other minor crosses have more size than the major pair. Date Time Of Last Edit: 2021-10-08 14:37:55


Untitled.png / V - Attached On 2021-10-08 14:37:50 UTC - Size: 47.61 KB - 137 views

[2021-10-09 00:14:55] #5 [Go To First Post]
Alberto Gauer - Posts: 121
Craziest discrepancy so far, I thought you would find it interesting. A 20GB!!!! Difference in size for the same period in EUR group, check screenshot. EURJPY has 22GB size and it´s the biggest of all so far, and EURCHF 2GB. Both files tick data starting in 2017. By the way, all the EUR pairs in this screenshot have the exact same time period starting in 2017 so you can see each has a totally different size. Date Time Of Last Edit: 2021-10-09 00:16:36


Untitled.png / V - Attached On 2021-10-09 00:14:49 UTC - Size: 12.92 KB - 146 views

[2021-10-16 12:23:05] #6 [Go To First Post]
Alberto Gauer - Posts: 121
@Sierra Engineering Just wanted to check if you looked into this and discovered the reason of such big difference in the size of files for same time period.

[2021-10-19 02:32:12] #7 [Go To First Post]
Sierra Chart Engineering - Posts: 104368
One reason is there is a difference of the FXCM Forex/CFD data file sizes on the servers. We corrected that by applying compression to data older than a year on one of the servers. This was done this past weekend. Any differences you see between different pairs just have to do with the price update activity on them. Anything you point out in that regard is fully meaningless to us. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, change to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2021-10-19 02:33:03

To post a message in this thread, you need to log in with your Sierra Chart account: