Login Page - Create Account

Support Board


Date/Time: Wed, 08 May 2024 13:45:08 +0000



large scid file size

View Count: 1752

[2017-10-03 14:46:36]
Paul.M - Posts: 113
Hi sierra team,

I have not updated sierra for a while.
I run Sierra 1510 .

I store intra-day data as ticks,
but compress data older than 30 days to 1 minute data.

This keeps the scid files small.

Now - after an update to Sierra 1614 something has changed.
When I manually compress to 1 minute - new files xxxx.tempcompressed are created.
The original scid files are still very large.

How can I restore the behavior & small file sizes like in Version 1510 ?

Why compress anything if the overall Data size of sierra increases ?

TY for your help
Date Time Of Last Edit: 2017-10-03 14:47:09
[2017-10-03 17:52:02]
Sierra Chart Engineering - Posts: 104368
Make sure you are running the current version which is 1619. Follow the instructions here:
Software Download: Fast Update

In the version you are running, the recent development of the Intraday Data File Management may not have been complete.

During the compression, a temporary file is now used for performance and safety. But this file is copied back to the original Intraday data file and the temporary file is deleted. We verified this all does work correctly.
Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service:
Sierra Chart Teton Futures Order Routing
Date Time Of Last Edit: 2017-10-03 17:52:25
[2017-10-04 17:15:52]
Paul.M - Posts: 113
Hi Sierra Team,

I updated to 1619.

Now recent contracts are compressed as you describe it,
but older contracts are not.

For older contracts the xxx.tempcompressed files are not removed and the original scid files are still very large.

e.g.:
ESZ17.scid ok, small
ESU17.scid ok, small
ESM17.scid large, ESM17.scid.tempcompressed not removed
ESH17.scid large, ESH17.scid.tempcompressed not removed
...

Please have a look at it

TY
[2017-10-04 17:28:02]
Sierra Chart Engineering - Posts: 104368
OK we would not expect that, we will look into this.
Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service:
Sierra Chart Teton Futures Order Routing
[2017-10-04 19:27:04]
Sierra Chart Engineering - Posts: 104368
We have prepared this documentation for this issue:
https://www.sierrachart.com/index.php?page=doc/DataSourceSettings.php#IntradayDataFileManagement_ErrorsAndInterruptingCompression

Most likely there were errors encountered when compressing these files:

ESM17.scid large, ESM17.scid.tempcompressed not removed
ESH17.scid large, ESH17.scid.tempcompressed not removed


Try again as documented.
Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service:
Sierra Chart Teton Futures Order Routing
Date Time Of Last Edit: 2017-10-04 19:27:26
[2017-10-05 16:48:03]
Paul.M - Posts: 113
I followed your steps.

Yes, the log shows errors - please findĀ“them below:

---------
Software version: 1619 | 2017-10-05 18:31:53
Usage end date: 2018-03-09 | 2017-10-05 18:31:53
Enabled for: Advanced Features. | 2017-10-05 18:31:53
Enabled for: Sierra Chart Historical Data Service. | 2017-10-05 18:31:53
Allow Support for Sierra Chart Data Feeds is enabled. | 2017-10-05 18:31:53
Current selected Data/Trading service: SC Data - All Services | 2017-10-05 18:31:53
Chart Update Interval: 500 | 2017-10-05 18:31:53
Time Zone: +02:00:00 (CET+01CEST+01,M3.5.0/02:00,M10.5.0/03:00) | 2017-10-05 18:31:53
2017-10-05 18:31:53 Local computer time | 2017-10-05 18:31:53
2017-10-05 16:31:53 Local computer time in UTC | 2017-10-05 18:31:53
2017-10-05 18:31:53 Local computer time in SC Time Zone | 2017-10-05 18:31:53
2017-10-05 12:52:14 Server time in UTC | 2017-10-05 18:31:53
Local computer UTC time and Server UTC time difference: 2 seconds. | 2017-10-05 18:31:53
Program path: C:\SierraChart3\ | 2017-10-05 18:31:53
Data Files path: C:\SierraChart3\Data\ | 2017-10-05 18:31:53
OS Version Number: 6.1 | 2017-10-05 18:31:53
Locale Setting: C | 2017-10-05 18:31:53

Deleted 10 (*.tempcompressed) files from C:\SierraChart3\Data\ | 2017-10-05 18:32:03
Scanning intraday files in the data files folder to reduce file sizes by combining old data records. | 2017-10-05 18:32:03
SC Data - All Services | Disconnected. | 2017-10-05 18:32:04
Combining old data records in ESH17.scid. | 2017-10-05 18:32:04
Failed combining old data records in ESH17.scid. Failed to read from the original file. At the end of the file C:\SierraChart3\Data\ESH17.scid | 2017-10-05 18:33:16
SC Data - All Services | Disconnected. | 2017-10-05 18:33:16
Combining old data records in ESM17.scid. | 2017-10-05 18:33:16
Failed combining old data records in ESM17.scid. Failed to read from the original file. At the end of the file C:\SierraChart3\Data\ESM17.scid | 2017-10-05 18:34:50
SC Data - All Services | Disconnected. | 2017-10-05 18:34:50
Combining old data records in ESU16.scid. | 2017-10-05 18:34:50
Failed combining old data records in ESU16.scid. Failed to read from the original file. At the end of the file C:\SierraChart3\Data\ESU16.scid | 2017-10-05 18:36:20
SC Data - All Services | Disconnected. | 2017-10-05 18:36:20
Combining old data records in ESU17.scid. | 2017-10-05 18:36:20
Reduced the size of ESU17.scid by 12%. | 2017-10-05 18:36:21
SC Data - All Services | Disconnected. | 2017-10-05 18:36:21
Combining old data records in ESZ16.scid. | 2017-10-05 18:36:21
Failed combining old data records in ESZ16.scid. Failed to read from the original file. At the end of the file C:\SierraChart3\Data\ESZ16.scid | 2017-10-05 18:38:18
SC Data - All Services | Disconnected. | 2017-10-05 18:38:18
Combining old data records in ESZ17.scid. | 2017-10-05 18:38:18
Reduced the size of ESZ17.scid by 0%. | 2017-10-05 18:38:19
SC Data - All Services | Disconnected. | 2017-10-05 18:38:22
Combining old data records in NQH16.scid. | 2017-10-05 18:38:22
Failed combining old data records in NQH16.scid. Failed to read from the original file. At the end of the file C:\SierraChart3\Data\NQH16.scid | 2017-10-05 18:38:22
SC Data - All Services | Disconnected. | 2017-10-05 18:38:22
Combining old data records in NQH17.scid. | 2017-10-05 18:38:22
Failed combining old data records in NQH17.scid. Failed to read from the original file. At the end of the file C:\SierraChart3\Data\NQH17.scid | 2017-10-05 18:38:22
SC Data - All Services | Disconnected. | 2017-10-05 18:38:23
Combining old data records in NQM16.scid. | 2017-10-05 18:38:23
Failed combining old data records in NQM16.scid. Failed to read from the original file. At the end of the file C:\SierraChart3\Data\NQM16.scid | 2017-10-05 18:38:23
SC Data - All Services | Disconnected. | 2017-10-05 18:38:23
Combining old data records in NQM17.scid. | 2017-10-05 18:38:23
Failed combining old data records in NQM17.scid. Failed to read from the original file. At the end of the file C:\SierraChart3\Data\NQM17.scid | 2017-10-05 18:38:23
SC Data - All Services | Disconnected. | 2017-10-05 18:38:23
Combining old data records in NQU16.scid. | 2017-10-05 18:38:23
Failed combining old data records in NQU16.scid. Failed to read from the original file. At the end of the file C:\SierraChart3\Data\NQU16.scid | 2017-10-05 18:38:23
SC Data - All Services | Disconnected. | 2017-10-05 18:38:23
Combining old data records in NQZ16.scid. | 2017-10-05 18:38:23
Failed combining old data records in NQZ16.scid. Failed to read from the original file. At the end of the file C:\SierraChart3\Data\NQZ16.scid | 2017-10-05 18:38:23
Finished scanning intraday files in the data files folder. Completed 73 of 73 files. | 2017-10-05 18:38:26
[2017-10-05 18:40:42]
Sierra Chart Engineering - Posts: 104368
We are looking into this. We have patched this issue. But we will do more analysis on it.

We are putting out a new revision now.
Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service:
Sierra Chart Teton Futures Order Routing

To post a message in this thread, you need to log in with your Sierra Chart account:

Login

Login Page - Create Account