Support Board
Support Board
Search Board
Control Panel
View My Posts / Threads
Direct MessagesDate/Time: Fri, 03 May 2024 13:01:51 +0000
Trade Activity Log Hangs
View Count: 738
| [2019-11-11 19:37:16] |
| User41727 - Posts: 124 |
|
I have an issue with SC where the trade activity log fails to update and starts to hang after a while. New executions are not shown, and trades are not properly calculated. Hitting the "apply" button results in the message "Query Active" being shown. Sometimes this resolves itself after a few minutes, other times this state persists for hours with no resolution. Only a restart of SC seems to help. What is an even bigger issue with this is that open positions are also not being tracked if this happens. This means they don't show up on charts or anywhere else, and ACSIL code cannot access the position information. This is a severe problem while running automated strategies. It affects both simulated orders as well as live ones. The issue seems to occur relatively reliably after at most a few hours of uptime for SC with only a handful of trades. It seems to happen with all of the recent SC versions up to and including 2007. |
| [2019-11-11 20:22:32] |
| |
|
and starts to hang after a while. What exactly do you mean by this? No one has ever reported a problem like this.What is an even bigger issue with this is that open positions are also not being tracked if this happens. This cannot be true in the case of non-simulated trading.In general, what you are describing is not making technical sense or even legitimate from our perspective. For example, this cannot possibly be true: What is an even bigger issue with this is that open positions are also not being tracked if this happens. This means they don't show up on charts or anywhere else, and ACSIL code cannot access the position information. This is a severe problem while running automated strategies. It affects both simulated orders as well as live ones.
Unless it is a case where there are just outstanding Trade Activity data queries which are just not completing due to a performance problem on your side. But that would not affect non-simulated positions. Anyway, this issue is specific to what you are doing with Sierra Chart and your system. There is exactly 0% chance there is anything wrong on the Sierra Chart side whether you accept that or not. This is a basic fact. Long trade activity queries would be the result of a performance issue with your storage drive, and it cannot keep up with all of the trade activity data in the TradeActivityLogs subfolder. So you need to go through that folder and start deleting trade activity files which you do not need. So this is a problem you need to solve. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2019-11-11 20:24:06
|
| [2019-11-11 21:07:33] |
| User41727 - Posts: 124 |
|
What exactly do you mean by this? No one has ever reported a problem like this.
What I mean by that is that I had from set to Friday and to to "today" at the start of the day. Several live trades were made, and those seem to be stored in a respective trade activity log file. However, none of these trades show up in the "Trade Activity Log" window. I set the from date to today, hit the "Apply" like five hours ago, and the query is still pending. Unless it is a case where there are just outstanding Trade Activity data queries which are just not completing due to a performance problem on your side.
As I have been telling you, there is an outstanding query that has been pending literally for hours. My storage drive is a Samsung 970 Evo Plus NVMe SSD that can read around 3 GB/s. There are currently 335 trade activity log files with a total size of 69 MB in the trade activity log folder. It is literally impossible that this could be an issue with my hardware. Furthermore, since the "from" date had been set to Friday, only the trade activity log file from that day and today should even be touched; even if there were 10,000 files more, that should be irrelevant. The only explanation is that SC does something extremely stupid and inefficient while attempting to read these files. That being said, I will remove the older files to see if that helps remedy this issue, even though it makes no sense that it would. |
| [2019-11-12 04:28:52] |
| |
|
The most likely scenario is there are a lot of accumulated Trade Activity queries. Perhaps a chart maintaining a Trades List is continuously reloading causing an accumulation of queries. Restart Sierra Chart and do not open any charts, and use the Trade Activity Log independently and see if the problem still exists. For charts, check this setting and make sure it's not too far back in time: Chart Settings: Order Fills Start Date-Time (Chart >> Chart Settings >> Trading menu) The only explanation is that SC does something extremely stupid and inefficient while attempting to read these files.
No this is a false conclusion on your part.
Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2019-11-12 04:31:14
|
| [2019-11-12 16:09:43] |
| User41727 - Posts: 124 |
|
The loading was previously unrestricted. I have now changed that. I had also already removed all of the old activity files save for the last two. No issues so far today even without the aforementioned setting. So it stands to reason that this is indeed a problem with accumulated queries. This still begs why things would eventually hang completely. Either there is a deadlock or your query algorithm has exponential time complexity. |
| [2019-11-13 13:17:14] |
| |
|
No there definitively is not a deadlock. This is not possible. We are going to add logging to the Trade >> Trade Service Log with the amount of time it takes to perform a query, and the number of outstanding queries. And we will also display the current number of queries on the Trade Activity Log. query algorithm has exponential time complexity. No this is not the case at all. There there is a simple search of the files that specifically are within the date range of the query. And only those particular files are searched. It is a simple sequential search. The result is placed into an STL multi-map and as the multi-map grows, the insertion time might increase, but the elements are always added to the end of the multi-map if possible and that is where they likely need to be.
Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2019-11-13 13:17:28
|
| [2019-11-13 13:45:20] |
| |
|
We did some testing now of large queries. Here is one query: TradeActivity files query time: 0.779632 seconds. Entries: 16008. Outstanding queries: 1 The TradeActivityLogs folder contained: 60 MB in 200 files The 16008 is the resulting number of records. So you can see this is very fast. And then we did another query with 670 MB in 275 files in TradeActivityLogs: TradeActivity files query time: 93.799339 seconds. Entries: 1766483. Outstanding queries: 1 This one took a while, but there is a lot of data to search, and the result is almost 1,800,000 records. This is completely within reason. We recommend that you check your system for any performance issues. Like antivirus software. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2019-11-13 13:46:41
|
| [2019-11-13 16:03:24] |
| |
|
One thing though that will help is that we will start storing trade activity data not only by date, but also by trade account in individual files. This will make queries faster which are specific to a particular trade account.
Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing |
To post a message in this thread, you need to log in with your Sierra Chart account: