Login Page - Create Account

Support Board


Date/Time: Sun, 15 Dec 2019 16:10:34 +0000



Custom Studies has issue on windows 10

[2019-10-09 12:16:42]
ertrader - Posts: 226
Hi Support, here is a chart book that works on v1991 and crashes on v1997 Windows 10. It's a single chart, uses SC data, is in 64 bit mode and does not use OpenGL. Hangs after loading then crashes without a log.

It's a 6 range chart with numbers bars, unfinished business, Valto's support and resistance, EdgeZones and volatility trend indicator. The dll's have not changed for a couple months, just the SC version.
Date Time Of Last Edit: 2019-10-09 12:17:17
attachmentSupportResistance.Cht - Attached On 2019-10-09 12:15:17 UTC - Size: 67.83 KB - 29 views
[2019-10-09 14:26:59]
Sierra Chart Engineering - Posts: 79194
But we are not going to be able to reproduce any problem. We are not sure why there is an issue in newer versions with those custom studies. The developer of those custom studies needs to look at this. We just cannot get involved in this.

We are not aware of any changes which would affect custom studies and being that those custom studies easily could have a bug in them, we just cannot be spending the time debugging these things. And we do not have any ability to do so either.

The first thing they should do is recompile the custom studies for the current version and see if that makes any difference.

For complete information about all of this, refer to help topic 17:
Https://www.sierrachart.com/index.php?page=doc/helpdetails17.html
Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. If possible please keep your questions brief and to the point. Please be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

If your question/request has been answered and you do not have anything further, then it is easiest for us if you do not reply again to say thank you.
Date Time Of Last Edit: 2019-10-09 14:28:58
[2019-10-09 16:58:34]
binaryduke - Posts: 199
We (emojitrading) have been examining the chartbooks of this user and user tommartin321 in relation to this thread and the issue logged here:
https://www.sierrachart.com/SupportBoard.php?ThreadID=46015

We have recompiled the relevant studies serving the chartbooks against v1997.

The issue in this thread seems to have disappeared as a result.

The chartbook in the other thread still causes problems. Debugging with Visual Studio attached to Sierra Chart implies that there is heap corruption occurring with Sierra Chart. We find this occurs consistently when the chartbook is closed. The VS debugger does not point to the custom study dll, but to the windows ntdll.dll (not that we are closed to the possibility of a bug within the custom study dll, however it has been working fine up until v1997).

The custom study uses persistent pointers to allocate and release dynamic memory as per the various ACSIL examples. At the last call to the function, the study checks for a non-null pointer, deletes the memory and clears the pointer as per the ACSIL examples.

Prior to recompiling against v1997 and using the VS debugger, Sierra Chart and Windows dlls (not the custom study dll) were causing errors on closing that seemed to relate to accessing an out of bounds index in an array of characters. There is nothing in the custom study that deals with arrays of characters and we are wondering whether this could relate in some way to the study ID numbering bug identified in v1996 perhaps when the study defaults (or chartbook parameters for studies) are being read/written? The pre-v1997 compiled version was compiled against v1928.

We've attached the output of the Visual Studio debugger. We would also be happy to supply Sierra Chart support with the custom study DLL if this helps investigate a potential bug within v1997. As I say, we are not closed to the idea that there is a bug within our code, however, the Visual Studio debugging process is not breaking at areas within the custom study's code.
imageScreen Shot 2019-10-09 at 17.36.58.png / V - Attached On 2019-10-09 16:58:18 UTC - Size: 790.15 KB - 97 views
[2019-10-09 17:17:57]
binaryduke - Posts: 199
Further to this, we believe this behaviour may relate to the issue outlined in v1996 here:
https://www.sierrachart.com/SupportBoard.php?ThreadID=45957

We provide various study collections of standard Sierra Chart studies (Numbers Bars, Daily OHLC, Numbers Bars Calculated Values) as part of our distribution. These are all set up to prompt whether existing studies should be deleted.

On v1997, we experienced the following:

1. Open a new intraday chart.
2. Apply a study collection
3. Apply another study collection to add to these studies (not clearing the existing studies on the chart).
4. After checking the list of studies applied to the chart in the Studies window, the name for one of the studies was corrupted displaying oriental characters. Closing the study window resulted in Sierra Chart closing.
[2019-10-09 17:54:21]
Sierra Chart Engineering - Posts: 79194
We would also be happy to supply Sierra Chart support with the custom study DLL if this helps investigate a potential bug within v1997.
Yes provide us the files and we will test.

Regarding what is described in post #4 we cannot reproduce that. And also, the issue that you are linking to was definitively resolved:
https://www.sierrachart.com/SupportBoard.php?ThreadID=45957

It simply is not an issue in 1997. It could not be.
Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. If possible please keep your questions brief and to the point. Please be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

If your question/request has been answered and you do not have anything further, then it is easiest for us if you do not reply again to say thank you.
Date Time Of Last Edit: 2019-10-09 17:55:00
[2019-10-09 18:30:15]
binaryduke - Posts: 199
Please find relevant files attached.
Private File
Private File
[2019-10-10 02:31:29]
Sierra Chart Engineering - Posts: 79194
One thing we will say if users are still having a problem, even after recompiling the study for the current version, therefore it is 99% certain the problem is in your own code.

We will get to testing this as soon as we can.
Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. If possible please keep your questions brief and to the point. Please be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

If your question/request has been answered and you do not have anything further, then it is easiest for us if you do not reply again to say thank you.
[2019-10-10 08:42:53]
binaryduke - Posts: 199
Thank you. We will continue hunting from our side too with the relevant chartbooks. If our investigations find anything not in our own code that may help we will share the details here. If we resolve this from our side we'll advise here to to save your time. Likewise if you have any queries of ourselves or would like the dlls compiled with debugging symbols please get in touch directly.
[2019-10-12 11:19:10]
binaryduke - Posts: 199
We have undertaken further testing on this issue and attach another chartbook from an affected user. Our studies make use of dynamic memory allocation for vectors of structs and make use of sc.SetPersistentPointer. We have verified all of our code for memory allocation and deallocation which occurs in line with Sierra Chart's examples. We have also verified that no out of bounds vector indexes are being referenced.

What has been consistent in all cases has been heap corruption and two of the three affected chartbooks make use of a large number of charts including several with short timeframes (1 minute), a large number of days loaded relative to the chart timeframe (60 days) and multiple custom indicators with relatively loose parameters. Several charts include Numbers Bars. Charts also include thin instruments with missing price levels.

During the latest testing we have noticed this error from the standard Numbers Bars study on one of the affected chartbooks (screen dump attached):
Chart: NQZ9-CME [C] 1 Min #10 | Study: Numbers Bars | Exception occurred while calling study function. | 2019-10-12 11:06:30.809 *

We also notice from sierrachart.h that there are revised functions for SetPersistentSCString and SetPersistentSCStringForChartStudy as from v1988. Given we make use of sc.SetPersistentPointer we are wondering whether any changes to dynamic memory allocation as from v1988 could be the cause of the heap corruption.
Date Time Of Last Edit: 2019-10-12 11:26:47
imageScreen Shot 2019-10-12 at 12.09.47.png / V - Attached On 2019-10-12 11:17:55 UTC - Size: 378.7 KB - 41 views
Private File
[2019-10-12 11:39:50]
Sierra Chart Engineering - Posts: 79194
We also notice from sierrachart.h that there are revised functions for SetPersistentSCString and SetPersistentSCStringForChartStudy as from v1988.
There were new versions of these added but the existing ones were still maintained.

Given we make use of sc.SetPersistentPointer we are wondering whether any changes to dynamic memory allocation as from v1988 could be the cause of the heap corruption.
No.

This occurs because there has been a memory corruption which affected that study:
Chart: NQZ9-CME [C] 1 Min #10 | Study: Numbers Bars | Exception occurred while calling study function. | 2019-10-12 11:06:30.809 *
It is not the study itself which has a problem. It is something else related to your custom study.
Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. If possible please keep your questions brief and to the point. Please be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

If your question/request has been answered and you do not have anything further, then it is easiest for us if you do not reply again to say thank you.
Date Time Of Last Edit: 2019-10-16 23:11:58
[2019-10-16 21:57:41]
Sierra Chart Engineering - Posts: 79194
We are starting to look this over now.
Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. If possible please keep your questions brief and to the point. Please be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

If your question/request has been answered and you do not have anything further, then it is easiest for us if you do not reply again to say thank you.
[2019-10-16 22:10:15]
Sierra Chart Engineering - Posts: 79194
It is 100% clear, there is a problem in your code. It is occurring in this function:
ETOFSv3_64.scsf_Emoji_License_Manager

The debugger is reporting this:
Exception thrown at 0x000007FEFD65B87D in SierraChart_64.exe: Microsoft C++ exception: std::ios_base::failure at memory location 0x000000000009A920.

But we are suspicious of whether that is really the accurate exception.

This is the call stack:
>  KernelBase.dll!RaiseException() + 61 bytes  Unknown
  ETOFSv3_64.dll!000007fedb54b14d()  Unknown
  ETOFSv3_64.dll!000007fedb4da556()  Unknown
  ETOFSv3_64.dll!000007fedb4e3c05()  Unknown
  ETOFSv3_64.dll!000007fedb4ec49a()  Unknown
  ETOFSv3_64.dll!000007fedb4ec859()  Unknown
  ETOFSv3_64.dll!000007fedb4d8fa3()  Unknown
  ETOFSv3_64.dll!scsf_Emoji_License_Manager() + 1140 bytes  Unknown
Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. If possible please keep your questions brief and to the point. Please be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

If your question/request has been answered and you do not have anything further, then it is easiest for us if you do not reply again to say thank you.
[2019-10-18 08:59:55]
binaryduke - Posts: 199
Thank you for reviewing this. The other handful of affected user chartbooks we are reviewing do not have this study applied. We have also tested all chartbooks with modified code that bypasses any license management routines and we still find heap corruption that in the majority of cases is not listing any of our code in the call stack.

We will continue to work on this and if we find that we can isolate it so some specific code that is non-erroneous we will share our findings for further investigation by the Sierra Chart team.
[2019-10-18 12:17:43]
Sierra Chart Engineering - Posts: 79194
In regards to post #12 above, that occurred before any of your custom studies were added to a chart and then we selected Analysis >> Studies >> Add Custom Study. At that time when Sierra Chart is browsing through the available studies in the DLL , the exception occurred.

Are there any ACSIL (sc) functions getting called within the sc.SetDefaults code block for the studies contained within the DLL?
Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. If possible please keep your questions brief and to the point. Please be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

If your question/request has been answered and you do not have anything further, then it is easiest for us if you do not reply again to say thank you.
[2019-10-18 14:29:58]
binaryduke - Posts: 199
We don't call any ACSIL (sc) functions during the sc.SetDefaults code blocks for the DLL's studies.

The only possible contentious code in this block for the DLL's studies is:

- some dynamic memory allocation in the ETOFSv3_64.scsf_Emoji_License_Manager. We have checked this and it is allocated and freed correctly and the associated pointers are set to nullptr as a further failsafe. Further, we have tested by deleting this study from the DLL and user chartbooks still crash with heap corruption

- routines to set inputs. We have a bank of around 70 inputs that studies access. Not every input uses every study, however we have taken heed of the guidance within this post: https://www.sierrachart.com/SupportBoard.php?ThreadID=39216 and our code:
- ensures that inputs are used in a continuous set in the sc.Input array starting from the zero index
- does not present unused inputs by NOT setting the name member for the input

I recall reading about study behaviour when Sierra Chart browses available studies within custom DLLs but this is not documented here: https://www.sierrachart.com/index.php?page=doc/ACS_ArraysAndLooping.html#WhenTheStudyFunctionIsCalled - perhaps it was in a support thread. Please could you provide some guidance as to what checks in the DLL so we can investigate around these areas. We do use some global variables.

We have not experienced the situation that the program crashes when Analysis >> Studies >> Add Custom Study is selected which is odd. We continue to compile excluding potentially 'dangerous' code involving dynamic memory allocation but we are still finding heap corruption and the VS debugger is 90% of the time not referring back to our code but to the call stack that we have previously provided.
[2019-10-18 18:24:25]
Sierra Chart Engineering - Posts: 79194
Provide us a debug build of the DLL built under visual C++ and we will test it again and see if we get any additional information which is useful. But ultimately we probably need the source code as well.

There really is little doubt the problem is within the DLL itself. It must be something you are just overlooking.

We will update the documentation about the custom study functions being called when using Add Custom Study.
Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. If possible please keep your questions brief and to the point. Please be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

If your question/request has been answered and you do not have anything further, then it is easiest for us if you do not reply again to say thank you.
Date Time Of Last Edit: 2019-10-18 18:27:21
[2019-10-19 15:50:38]
binaryduke - Posts: 199
Thank you. We identified a small issue in how we construct the sc.Input array and in improving this code it appears that v1997 is much more sensitive to the 'quality' of the members within this array than prior releases. This seems to relate to handling of strings for the input names and this seems consistent with strange crashes when the studies are removed from the chart.

As we refactor our code that constructs the sc.Input array if we are able to provide you with an example of sc.Input array code that crashes in v1997 but is reliable in earlier versions we will share this.

As a starting point:
1. we have noticed that if the sc.Input[].DisplayOrder member is set to the same value as another input, this will cause heap corruption in v1997
2. v1997 appears strict in requiring sc.Input[].SetCustomInputIndex to be set after sc.Input[].SetCustomInputStrings() (which makes total sense if it is doing string/pointer manipulation) although prior to v1997 this did not seem to be an issue.
[2019-10-19 16:52:53]
binaryduke - Posts: 199
Attached please find:

- custom study that has been compiled as a debug build
- two specimen chartbooks - one simple, one more complex

The simple chartbook will quite consistently cause heap corruption when the chartbook is closed. We have traced this behaviour to after the following code runs, i.e. when the function returns from the test of sc.LastCallToFunction. Dynamic memory is allocated for two arrays of structs as follows:



struct s_v
  {
   int a, b, c, d, e, f, g, h, i, j;
  };

struct s_e
  {
   int z, y, x, w, v, u;
  };

  std::vector<s_v>* p_v = (std::vector<s_v>*)sc.GetPersistentPointer(1);
  std::vector<s_e>* p_e = (std::vector<s_e>*)sc.GetPersistentPointer(2);

  //Allocate dynamic memory
  if (sc.LastCallToFunction)
  {
    if (p_v != NULL)
    {
      delete p_v;
      sc.SetPersistentPointer(1, NULL);
    }
    if (p_e != NULL)
    {
      delete p_e;
      sc.SetPersistentPointer(2, NULL);
    }
    return;
//after closing the simple chartbook, this is when a hang occurs
  }

  if (p_v == NULL)
  {
    p_v = new std::vector<s_v>;

    if (p_v != NULL)
      sc.SetPersistentPointer(1, p_v);
    else
    {
      sc.AddMessageToLog("Memory allocation error 1", 1);
      return;
    }
  }

  if (p_e == NULL)
  {
    p_e = new std::vector<s_e>;

    if (p_e != NULL)
      sc.SetPersistentPointer(2, p_e);
    else
    {
      sc.AddMessageToLog("Memory allocation error 2", 1);
      return;
    }
  }

The supplied study and chartbooks all work well in v1991.

The study will need activation per the supplied notes in post #6. It would be appreciated if you would check the debug version so we can understand whether there is anything amiss from our side or if it helps identify any issue in v1997.
Date Time Of Last Edit: 2019-10-19 16:53:18
Private File
Private File
Private File
[2019-10-19 17:53:01]
Sierra Chart Engineering - Posts: 79194
1. we have noticed that if the sc.Input[].DisplayOrder member is set to the same value as another input, this will cause heap corruption in v1997
No that could not be the problem. We checked.

But there were some changes using this variable related to an optimization.

The valid range is from 1 through SC_INPUTS_AVAILABLE (128). If you are using an out of range value, this could cause instability. But we added a safety check which will be out in the next release.

2. v1997 appears strict in requiring sc.Input[].SetCustomInputIndex to be set after sc.Input[].SetCustomInputStrings() (which makes total sense if it is doing string/pointer manipulation) although prior to v1997 this did not seem to be an issue.
We do not see how this matters. We looked at this quite closely. It will not matter at all.
Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. If possible please keep your questions brief and to the point. Please be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

If your question/request has been answered and you do not have anything further, then it is easiest for us if you do not reply again to say thank you.
Date Time Of Last Edit: 2019-10-19 17:58:26
[2019-10-19 17:59:19]
Sierra Chart Engineering - Posts: 79194
We need to make a correction to the above post. We said:
It is not valid to repeat the value used for DisplayOrder within the same study.
This is harmless to do. It does not cause any problem. We saw what you wrote, and we did test this but it is not an issue. But we just wanted the documentation to explain that it is not something that should be done but it does not cause any problem either.
Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. If possible please keep your questions brief and to the point. Please be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

If your question/request has been answered and you do not have anything further, then it is easiest for us if you do not reply again to say thank you.
Date Time Of Last Edit: 2019-10-19 17:59:46
[2019-10-19 18:12:44]
Sierra Chart Engineering - Posts: 79194
Just using your license manager study, we encountered:
Exception thrown at 0x000007FED62F205E (ETOFSv3_64.dll) in SierraChart_64.exe: 0xC0000096: Privileged instruction.

This tells us, that the code area is getting corrupted. And for the record, we have never seen an exception like that ever. Ever. This tells us there is a serious corruption going on. And it is exceptionally unlikely to be coming from Sierra Chart. We have never seen anything like this before in Sierra Chart. And if the custom study is interacting with ACSIL and there is some problem going wrong in ACSIL it would not cause an exception like this. It would be something more straightforward and obvious.

This is the call stack:

  ETOFSv3_64.dll!000007fed62f205e()  Unknown
  ETOFSv3_64.dll!000007fed62e7945()  Unknown
  ETOFSv3_64.dll!000007fed62cc9ba()  Unknown
  ETOFSv3_64.dll!000007fed62da78f()  Unknown
  ETOFSv3_64.dll!000007fed62da98a()  Unknown
  ETOFSv3_64.dll!000007fed62d77e2()  Unknown
  ETOFSv3_64.dll!000007fed62a5ace()  Unknown
  ETOFSv3_64.dll!scsf_Emoji_License_Manager() + 1908 bytes  Unknown
Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. If possible please keep your questions brief and to the point. Please be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

If your question/request has been answered and you do not have anything further, then it is easiest for us if you do not reply again to say thank you.
Date Time Of Last Edit: 2019-10-19 18:18:24
[2019-10-19 23:04:56]
Sierra Chart Engineering - Posts: 79194
We skipped over that Privileged Instruction exception and we are not seeing any further problem. Maybe these two exceptions we encountered, and are handled or just ignored, during normal execution. We just have the debugger to break on all exceptions and they are not something we normally would see and we assumed they are indicating a problem but probably not.
Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. If possible please keep your questions brief and to the point. Please be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

If your question/request has been answered and you do not have anything further, then it is easiest for us if you do not reply again to say thank you.
[2019-10-19 23:14:12]
Sierra Chart Engineering - Posts: 79194
We do see what appears like an endless loop when testing your simple Chartbook in post #18, and it is the Unfinished Business study repeatedly calling GetACSDrawingByIndex.
Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. If possible please keep your questions brief and to the point. Please be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

If your question/request has been answered and you do not have anything further, then it is easiest for us if you do not reply again to say thank you.
Date Time Of Last Edit: 2019-10-19 23:14:19
[2019-10-20 07:33:13]
Sierra Chart Engineering - Posts: 79194
We tested your complextest Chartbook, with extensive checking of memory for corruptions, and no problems were found. We opened the Chartbook, left it open for hours and then closed it. Under the debugger this whole process took many hours. And no corruptions or exceptions other than the exceptions already mentioned.

Perhaps the issue was related to invalid DisplayOrder indexes. This probably makes sense especially being there was a recent change with working with that variable. And under our testing described above, the safety checks relating to the use of that variable were already implemented. Even without the safety check though, there should never have been any type of heap correction. At worst you would have had a garbled display of inputs on the chart or an access violation.
Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. If possible please keep your questions brief and to the point. Please be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

If your question/request has been answered and you do not have anything further, then it is easiest for us if you do not reply again to say thank you.
Date Time Of Last Edit: 2019-10-20 07:34:17
[2019-10-20 11:39:35]
binaryduke - Posts: 199
Thank you for looking at this. We did experience one test where an input was garbled and appeared in oriental characters. We now have 2-3 areas to focus on for optimisation so thank you again for taking the time on this.
[2019-10-20 14:08:51]
Sierra Chart Engineering - Posts: 79194
We will release a new version now to see if the issue is resolved related to the DisplayOrder for Inputs.

Update: 2001 has been released.
Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. If possible please keep your questions brief and to the point. Please be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

If your question/request has been answered and you do not have anything further, then it is easiest for us if you do not reply again to say thank you.
Date Time Of Last Edit: 2019-10-20 14:16:39
[2019-10-21 16:57:39]
binaryduke - Posts: 199
Thanks! We have tested unmodified code with v2001 and all is good. We have also identified that we were populating the sc.Input[].DisplayOrder member with an out of range value. This has been fixed and we will issue an update to our users shortly. This update works just fine with v1997 so it looks like between us we have been able to pinpoint the issue to our error on the DisplayOrder member combined with your recent change to working with that variable. Thank you for your assistance.
[2019-10-21 17:23:54]
Sierra Chart Engineering - Posts: 79194
Ok so it is a combination of things on both sides. Lack of safety on our side, and using an out of range value on your side.
Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. If possible please keep your questions brief and to the point. Please be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

If your question/request has been answered and you do not have anything further, then it is easiest for us if you do not reply again to say thank you.
[2019-10-24 02:33:34]
ertrader - Posts: 226
I appreciate the hard work getting to root causes. SC 2002 and Emoji V305 are working well now in all my test cases. I am seeing significant performance improvement and stability on both Windows 10 and Linux/Wine.
Date Time Of Last Edit: 2019-10-24 15:09:03

To post a message in this thread, you need to login with your Sierra Chart account:

Login

Login Page - Create Account