Support Board
Support Board
Search Board
Control Panel
View My Posts / Threads
Direct MessagesDate/Time: Fri, 26 Apr 2024 02:16:20 +0000
custom developed DTC Server and SC
View Count: 1945
| [2019-02-19 16:38:06] |
| User12089 - Posts: 350 |
|
I have a custom developed DTC Server which I use to integrate SC with currently unsupported market data vendor. My DTC Server supports both Market Depth and Time and Sale DTC Messages and during the majority of the time works fine and I can see and use all data in SC and know for a fact that it is correct - because I compared it with data from other market data vendors for which I am also subscribed However from time to time (e.g. once every 24 or 48 hours) SC freezes and I have to terminated and then restart the SC instance. It connect to my DTC Server and carries on without the need for me to restart the DTC Server ie the DTC Server itself doesn't get frozen Now I have been SC customer for 5 years in a row and secondly I am a professional software engineer and know for a fact that SC is a rock solid trading platform and it has never ever crashed or froze while I have been using it for 5 years .... So the issue is definitely not in SC, but in my DTC Server. The reason why I am asking you for help is to let me know under what conditions can SC freeze when communicating with a DTC Server and/or processing DTC Server messages (I am using the SC Google Protocol Buffers for the DTC Message Encoding) |
| [2019-02-19 20:14:08] |
| |
|
When you notice this freezing condition, check the CPU usage of the Sierra Chart process in the Windows task manager and let us know what it is. Is it 0%? There should not be a freeze condition at all. If it is 0% it would seem like there is a deadlock condition going on we would have to try to reproduce that. This does not sound like a problem with your server, rather within Sierra Chart. If we can get access to your server, we can then run a test and see why this is happening. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing |
| [2019-02-19 20:21:55] |
| User12089 - Posts: 350 |
|
ok thank you, will organize the collection of debug info. I get your point, however for 5 years of using SC with the market data adapters developed by SC e.g. for the SC Data Feed, IQ Feed, Rithmic etc, SC has not frozen or crashed on me even once, and again that's for 5 years period .... so far in my practice it has only happened now and with .... a DTC Server (used as market data adapter / feed handler so I can get the data in SC) developed by me |
| [2019-02-19 23:06:20] |
| |
|
This just should not happen under any condition, so we will await further information from you.
Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing |
| [2019-03-26 13:30:53] |
| User12089 - Posts: 350 |
|
"When you notice this freezing condition, check the CPU usage of the Sierra Chart process in the Windows task manager and let us know what it is. Is it 0%? There should not be a freeze condition at all. If it is 0% it would seem like there is a deadlock condition going on we would have to try to reproduce that. This does not sound like a problem with your server, rather within Sierra Chart. " I can report that the CPU utilization is normal e.g. 15% when the SC process freezes I have also enabled the writing of the Message Log to a file and will inspect its content after the next freeze/crash event Note that besides the connection to my custom developed DTC Server, the SC instance uses edited Global Symbol Settings file which I edited to include the extra symbols from my market data vendor - could the issue be related to the periodical parsing of that file by SC? I can confirm that SC starts ok, and works for several days in a row even a week without a freezing problem. But does it e.g. try to refresh that file somehow in the background while e.g. looking for some sort of URLs for the market data vendor? |
| [2019-03-26 22:58:03] |
| |
|
- could the issue be related to the periodical parsing of that file by SC? I No, that does not happen. Only when Sierra Chart is started.
Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing |
| [2019-04-01 18:55:57] |
| User12089 - Posts: 350 |
|
ok here we go we go - new important information available .... it seems that for each Detached chart, the main SC Process creates a dedicated Thread (all software dev folks know the difference between Process and Thread, that's not for non-technical readers of the forum) I have identified that there is a deadlock between the Thread of the main SC Process and the Thread of one Detached Chart - see attached the proof .... Let me describe the full setup SC version 1882 64 bit Windows 10 laptop with 32 GB RAM and 6 CPU cores SC is launched and then I open two different chart books, each with about 15 charts - that still results in only 1 Thread Then I create 1 detached chart and that seems to spawn a new Thread - so one of the chartbooks contained such detached chart and it was for the same symbol as for some of the non-detached charts in the chart book Then see the attached screenshot about the deadlock between the two Threads also see here https://docs.microsoft.com/en-us/windows/desktop/Debug/wait-chain-traversal Also note that both Threads seem to be exhibiting some meaningful CPU utilization e.g. about 14% in total, BUT at the same time their state in Windows Task Manager is shown as "Non Responding" .... I have been using chartbooks with gazillion of Detached charts (in fact with 100% detached charts) but so far have not encountered such deadlock It may be due to one of the following reasons: - Bug introduced in the latest versions of SC - Something to do with Detached and Non-Detached charts when they try to e.g. write to the same Historical Market Depth file (that study is on the charts and all charts are intraday) - Something to do with the custom DTC Server developed by me and how detached and non-detached charts use data from it Let me know what to do next In the meantime I will simply stop using chartbooks with mixed detached and attached charts and will use chartbooks with 100% NON-detached charts and see whether the problem with the deadlock will disappear. Then I will try with chartbooks with 100% Detached charts etc Date Time Of Last Edit: 2019-04-01 19:05:58
|
 4-1-2019 6-59-46 PM.png / V - Attached On 2019-04-01 18:41:43 UTC - Size: 69.45 KB - 321 views |
| [2019-04-01 21:52:30] |
| |
|
it seems that for each Detached chart, the main SC Process creates a dedicated Thread It is not Sierra Chart creating this thread. Not at all. There simply is no separate thread for a detached chart in any way.Also, if there is a deadlock condition going on, the CPU usage of the process will be 0. So if you are really seeing another tthread related to a detached chart and if you think the problem is with that thread in some way, then this is going to be an issue specific to your system. We recommend trying a different system. - Something to do with Detached and Non-Detached charts when they try to e.g. write to the same Historical Market Depth file (that study is on the charts and all charts are intraday) - Something to do with the custom DTC Server developed by me and how detached and non-detached charts use data from it There is no direct connection between a chart and the external service connection.
Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2019-04-01 21:58:08
|
| [2019-04-01 22:37:45] |
| User12089 - Posts: 350 |
|
let me start with the question did yiu see the screenshot and do you see there the name of sc exe and its two DEADLOCKED threads .... is that "what I see" or what the system is telling US also see here to interpret the screenshot https://docs.microsoft.com/en-us/windows/desktop/Debug/wait-chain-traversal also open the Window Task Manager and under the main SC process you will see a number of child instances - definitely one for every detached chart ... that happens on both new and old SC versions - why the the Task Manager doesn't show just one record for e.g. sc.exe ...? there is nothing wrong with the system - standard Intel and standard MS Win 10, sprinkled for taste with the MS Computer Sciences of the owner and his 20 years of industry experience as a software architect and developer (although mainly on Linux/Unix not Win) Date Time Of Last Edit: 2019-04-01 22:39:08
|
| [2019-04-01 22:52:22] |
| |
|
That screenshot you provided is not helpful to us. What is a child instance? And if you see another thread for a detached chart, this is not from Sierra Chart. Not at all. If you do not want to believe us we cannot help with that. There certainly is something unusual going on if you see another thread for a detached chart. Maybe Windows is doing that for a particular reason. We do not know. The only way we can analyze this problem, is to reproduce it under our debugger and see what is happening. There is nothing you are providing here which is helpful to us at all. We are only becoming more convinced that this is an external problem which does not involve Sierra Chart. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2019-04-01 23:34:42
|
| [2019-04-01 23:33:45] |
| |
|
Thinking about this, because you are mentioning the Market Depth Historical Graph study we do think we know what the problem is. There are several other reports of a problem like this and we have yet been able to reproduce it in part because no one has provided us a Chartbook, even though we have asked for that multiple times. Provide us the Chartbook you are using with your detached charts and we will test it for as long as is needed to reproduce the problem. Based upon the information we have this is not a deadlock condition but appears to be some kind of an endless loop type condition. So please provide us the Chartbook so we can get this resolved definitively. We do not know why anyone has not been helpful in this regard yet. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2019-04-01 23:34:03
|
| [2019-04-02 00:18:35] |
| User12089 - Posts: 350 |
|
well I guess they don't provide the chartbooks because they contain trading knowhow .... and in my case there is the added complication of my "super secret" custom developed DTC Server haha which is market adapter to unsupported market data source anyway - for the next freeze I will be prepared even better this time will be attaching MS Visual Studio Debugger to the SC process to see what I can figure out in the meantime let me just tell you that I am running exactly the same chartbooks on 6 to 8 month older SC versions and no problems there wahsoever .... I may also try to downgrade the version here and see whether the problem would disappear one other thing to check with you is whether "unorthodox" characters in the symbol names like "/", "\", ":" etc may be messing something - I can see that when creating intraday tick data files and market depth files, SC replaces them with characters allowed to part of file names on Win but just asking anyway |
| [2019-04-02 00:57:14] |
| |
|
It does not matter what data feed you are using. We will just update the symbols to match our data feed we will be using for testing. contain trading knowhow .... To us this is just a bunch of nonsense we do not pay any attention to. Please just provide us the Chartbook.
Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing |
| [2019-04-02 13:03:55] |
| User12089 - Posts: 350 |
|
guys there is nothing special in the chartbooks - 2 chartbooks loaded at the same time and each with about 16 charts as each chart has only two studies - historical market depth and current price .... in the meantime i just had another crash and the same Deadlock between TWO separate SC Threads was identified by using the standard Win10 menu option in the Task Manager for "wait chains". It is identical to the screenshot already provided previously - two Threads with different thread IDs each belonging to the same SC process with the same PID Regarding the multiple "threads" observed by me in the Win10 Task Manager under the main SC process - i guess each of the entries there corresponds to a separate SC window ie a detached chart but whether that's a separate Thread i don't know - i am a Linux/Unix guy - but i will deploy commandline process analysis tool to profile in a more technical way what is a process and what is a thread in SC Also this time there were NO detached charts at all but the crash did occur as well .... I also tried to attach MS VC 2017 debugger to the frozen SC but it required the SC PDB files - do you provide them or they are "secrete" ? one other thing i will investigate in the meantime is whether TeamViwer on Win10 can be messing up somehow - i run it on my other laptops without any problems but may be there are special side effects on Win10 i will also reinstall that Win 10 from scatch - it is vendir installed - Alienware Date Time Of Last Edit: 2019-04-02 13:06:12
|
| [2019-04-02 16:52:44] |
| User12089 - Posts: 350 |
|
f... shi.... my suspicion of TV may have had some foundation after all (although again I have two other laptops with Win 8 and older versions of SC and TV and have not observed sc thread lock issue there) https://stackoverflow.com/questions/26561462/vb-net-application-freezes-with-teamviewer |
| [2019-04-02 17:10:17] |
| |
|
Provide us the Chartbook instead of all of these details. None of these details are helpful to us. We need our procedures followed. The only thing that is helpful to us is when you notice this condition if the Sierra Chart process CPU usage is at 0%. It does not help us at all for you to try to debug this. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2019-04-02 17:11:01
|
| [2019-04-02 17:16:24] |
| User12089 - Posts: 350 |
|
when this happens the CPU usage is never at 0% - I had already provided this info earlier (I will solve it don't worry - and most likely as you were saying earlier it is system background related and the trouble maker from that background is whatever version of TV on Win10 ....) |
| [2019-04-02 18:07:59] |
| |
|
We do not see how you can solve this. We do think it is related to the Market Depth Historical Graph.
Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing |
| [2019-04-23 11:43:46] |
| User12089 - Posts: 350 |
|
I am glad to report that for 3 weeks after removing TeamViewer there has not been a single SC freeze incident see above for info why TV may have been the reason also note that when I was attaching a debugger to the frozen SC process, the TV process was in the call chain too ... I have TV on Win8 and it runs with SC without problems but the platform where I was experiencing the freezes is Win10 and whatever TV versioj and the latest SC - so probably the root cause is in the specific combination of these including their versions |
| [2019-04-23 18:07:20] |
| |
|
That is good but we also discovered the problem which we think has been the cause and resolved it in the current version of Sierra Chart. And it does relate to using the Market Depth Historical Graph study. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2019-04-23 18:07:31
|
| [2019-04-23 18:41:58] |
| User12089 - Posts: 350 |
|
excellent, even better is the new version with the fix out already? |
| [2019-04-23 21:26:09] |
| |
|
Yes it is. This has been resolved in version 1906/1908 and higher.
Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2019-04-23 21:26:15
|
To post a message in this thread, you need to log in with your Sierra Chart account: