Support Board
Support Board
Search Board
Control Panel
View My Posts / Threads
Direct MessagesDate/Time: Thu, 02 May 2024 09:15:45 +0000
Disable NTFS compression on chart data files
View Count: 1469
| [2019-07-18 15:23:23] |
| hurleydood - Posts: 15 |
|
Can you add a checkbox to disable NTFS compression on chart data files?
|
| [2019-07-18 18:10:54] |
| |
|
This capability does exist but data files are quite large and we do not want to run the risk of compression being disabled. There really should not be a reason to disable compression. NTFS compression is efficient. And if you are using a dedicated solid-state drive for Sierra Chart, you should not have any trouble. Even on our servers, we always use compression even though we are working with hundreds of thousands of files and very large files. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2019-07-18 18:11:12
|
| [2019-07-18 19:17:19] |
| hurleydood - Posts: 15 |
|
The reason for it is performance reasons. Compression and decompression is happening over and over again without reason. When I unchecked the compression attribute on all the files manually I get lower CPU usage overall. However, every time I restart SC you turn compression attributes on my files back on. Check out the section "When To Use and When Not to Use NTFS Compression" https://www.howtogeek.com/133264/how-to-use-ntfs-compression-and-when-you-might-want-to/ NTFS compression is ideal for:
- Files you rarely access. (If you never access the files, the potential slow-down when accessing them is unnoticeable.) - Computers with fast CPUs and slow hard disks. NTFS compression should not be used for: - Windows system files and other program files. Using NTFS compression here can reduce your computer’s performance and potentially cause other errors. - Servers where the CPU is getting heavy use. On a modern desktop or laptop, the CPU sits in an idle state most of the time, which allows it to decompress the files quickly. If you use NTFS compression on a server with a high CPU load, the server’s CPU load will increase and it will take longer to access files. - Computers with slow CPUs, such as laptops with low-voltage power-saving chips. However, if the laptop has a very slow hard disk, it’s unclear whether compression would help or hurt performance. Also Microsoft has mentioned NTFS compression best practices, You can check out a summary here: https://superuser.com/a/1137127 Date Time Of Last Edit: 2019-07-18 21:55:57
|
| [2019-07-19 05:08:10] |
| |
|
Why does the CPU usage cause you a problem? This is not going to affect Sierra Chart and happens on another core with the way that Sierra Chart interacts with files. All file access in Sierra Chart is done on a background thread or out of memory caches. And we have never noticed a problem with this ourselves. If you have a noticeable CPU usage issue related to NTFS compression, then you need to look elsewhere rather than disabling it. there could be some driver hooking into the file system causing a problem and you should also be using Sierra Chart on a dedicated solid-state drive. Compression is just not an issue these days. We are not going to put this option in. It simply is out of the question because compression is so essential with the large amounts of market data which has to be stored. We are not going to put up with support issues, related to excessively large files. We process data rates far exceeding 200 Mb a second, and hundreds of thousands of files, and we have no issue with compression on a server. Also that information you are quoting in our opinion looks very dated and irrelevant these days with current hardware. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2019-07-19 05:11:47
|
| [2019-07-19 08:00:12] |
| hurleydood - Posts: 15 |
|
The compression doesn't affect SC in any way, the OS just pegs the CPU to the highest frequency, kicks the fans on and just drains my laptop battery faster. I don't have any issues with disk space on my system. I have 2TB Samsung 860 EVO SSD, 32GB RAM, and SC chart data is only at 30GB, which is nothing. I tried many optimizations and this is the only thing that worked. The bottleneck here is the processor and it's wasting CPU cycles. I'm not running SC on a server but server processors do have hardware-accelerated compression. Date Time Of Last Edit: 2019-07-19 08:00:57
|
| [2021-09-10 19:06:29] |
| WheresMoney - Posts: 56 |
|
Sorry to resurrect an old thread, but there is a legitimate reason to disable compression. I do understand that SC advises against using wine to run SC in linux, but many do successfully. If the linux file system in use already offers compression (zfs does out of the box), and the file system is not ntfs, then SC trying to compress it again is futile. Please tell me how is this functionality implemented in SC? Is SC just setting the compression property and letting windows take care of the actual compression? I think that is the case. Does SC check if the underlying fs is indeed ntfs? And for some reason the user perhaps is say running windows but using exfat for some reason? Can SC check it doesn't in fact have ntfs and not try to compress or does it already do that? Thanks. Date Time Of Last Edit: 2021-09-10 19:15:21
|
To post a message in this thread, you need to log in with your Sierra Chart account: