Support Board
Support Board
Search Board
Control Panel
View My Posts / Threads
Direct MessagesDate/Time: Sun, 19 May 2024 02:02:06 +0000
[Programming Help] - Inaccurate back tests results
View Count: 858
| [2018-12-17 20:58:49] |
| UserZiltoid - Posts: 57 |
|
Hello SC, It is some time now that we are strugelling with different results for the same back tests. We encounter that each test ends with different filled price results. For example, I have re-run the todays trading day with speed 80X and with speed 10X. For each test we got different results, which are actual different then the online results. Also the 10X and the 80X results are differ from one another. We are using "Calculate event tick mode". Since we can't rely on speed of 80X then we need to use maximum speed of 10X which is very hard (then 10X results are more closer the then online results). Therefore our question is, how to get accurate results which are the same for each test regardless the X speed. Any help will be much appreciated. Thanks! Date Time Of Last Edit: 2018-12-17 21:00:56
|
| [2018-12-17 21:22:40] |
| |
|
Refer to: Auto Trade System Back Testing: Consistency Between Back Tests Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing |
| [2018-12-18 06:28:42] |
| UserZiltoid - Posts: 57 |
|
Already tried that with no help. What I suspect is that sometimes when a new Index starts, the VOLUME of the previous index is not yet updated somehow, so in the first seconds our study see the volume value of the last index. How can I make sure that the study is seing the correct information of the current index? Thanks! |
| [2018-12-18 08:56:18] |
| |
|
What I suspect is that sometimes when a new Index starts, the VOLUME of the previous index is not yet updated somehow, No, this could not be the case. If you are referring to the ACSIL sc.BaseData array which contains volume.
Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing |
| [2018-12-18 09:19:17] |
| UserZiltoid - Posts: 57 |
|
I have continue to investigate and you are correct about the previous index. Yet.. what I do see is when running on 80X speed, many times in the first seconds of each BAR it shows volume that should appear only later. In other words, I see that in second 3 of new BAR the volume is 2073, but when I run on 1X speed this volume only appears at second 35. I am investigating the same BAR over and over again and it happens each time from speed 30X. Date Time Of Last Edit: 2018-12-18 09:24:24
|
| [2018-12-18 11:23:00] |
| UserZiltoid - Posts: 57 |
|
As I suspected. It seems that something is inocrrect in higher speed. I have printed each new second with it's volume and it is not correct. 504 981 Chart: Replay 80X: F.US.EPH19 1 Min #1 | Study: TEST Study | CHEKING | Index[1266], Volume[1025.000000] - 12:36:24 | 2018-12-18 05:13:07 505 982 Chart: Replay 80X: F.US.EPH19 1 Min #1 | Study: TEST Study | CHEKING | Index[1266], Volume[1114.000000] - 12:36:25 | 2018-12-18 05:13:07 506 983 Chart: Replay 80X: F.US.EPH19 1 Min #1 | Study: TEST Study | CHEKING | Index[1266], Volume[1312.000000] - 12:36:26 | 2018-12-18 05:13:07 507 971 Chart: Replay 80X: F.US.EPH19 1 Min #1 | Study: TEST Study | CHEKING | Index[1266], Volume[137.000000] - 12:36:12 | 2018-12-18 05:13:07 508 972 Chart: Replay 80X: F.US.EPH19 1 Min #1 | Study: TEST Study | CHEKING | Index[1266], Volume[138.000000] - 12:36:13 | 2018-12-18 05:13:07 509 984 Chart: Replay 80X: F.US.EPH19 1 Min #1 | Study: TEST Study | CHEKING | Index[1266], Volume[1428.000000] - 12:36:29 | 2018-12-18 05:13:07 510 985 Chart: Replay 80X: F.US.EPH19 1 Min #1 | Study: TEST Study | CHEKING | Index[1266], Volume[1432.000000] - 12:36:30 | 2018-12-18 05:13:07 ------------------------------------------- 534 1004 Chart: Replay 80X: F.US.EPH19 1 Min #1 | Study: TEST Study | CHEKING | Index[1266], Volume[5915.000000] - 12:36:58 | 2018-12-18 05:13:07 535 1005 Chart: Replay 80X: F.US.EPH19 1 Min #1 | Study: TEST Study | CHEKING | Index[1266], Volume[6210.000000] - 12:36:59 | 2018-12-18 05:13:07 536 1006 Chart: Replay 80X: F.US.EPH19 1 Min #1 | Study: TEST Study | CHEKING | Index[1266], Volume[6443.000000] - 12:37:0 | 2018-12-18 05:13:07 537 1007 Chart: Replay 80X: F.US.EPH19 1 Min #1 | Study: TEST Study | CHEKING | Index[1266], Volume[6694.000000] - 12:37:1 | 2018-12-18 05:13:07 538 977 Chart: Replay 80X: F.US.EPH19 1 Min #1 | Study: TEST Study | CHEKING | Index[1266], Volume[673.000000] - 12:36:18 | 2018-12-18 05:13:07 the order of the seconds is incorrect. Attached is the full log. The code that I use for this: --------------------------------------- auto currentTime = sc.GetCurrentDateTime(); auto currentSecond = currentTime.GetSecond(); int& lastSecond = sc.GetPersistentInt(persistentLastSecondTest); auto currentVolume = sc.Volume[sc.Index]; if (currentSecond != lastSecond) { SCString test; test.Format("CHEKING | Index[%d], Volume[%f] - %d:%d:%d", sc.Index, currentVolume, currentTime.GetHour(), currentTime.GetMinute(), currentSecond); sc.AddMessageToLog(test, 0); lastSecond = currentSecond; } |
Private File |
| [2018-12-18 11:33:03] |
| UserZiltoid - Posts: 57 |
|
This is the full source code for reproduding the bug in speed of 80X: ----------------------------- #include "sierrachart.h" int persistentLastSecondTest = 0; SCSFExport scsf_testVolumeTImeStudy(SCStudyInterfaceRef sc) { if (sc.SetDefaults) { sc.GraphName = "Test Volume time Study"; sc.AutoLoop = 1; sc.GraphRegion = 0; sc.FreeDLL = 0; } auto currentTime = sc.GetCurrentDateTime(); auto currentSecond = currentTime.GetSecond(); int& lastSecond = sc.GetPersistentInt(persistentLastSecondTest); auto currentVolume = sc.Volume[sc.Index]; if (currentSecond != lastSecond) { SCString test; test.Format("CHEKING | Index[%d], Volume[%f] - %d:%d:%d", sc.Index, currentVolume, currentTime.GetHour(), currentTime.GetMinute(), currentSecond); sc.AddMessageToLog(test, 0); lastSecond = currentSecond; } } ----------------------------------- For example - bug result: Chart: Replay 80X: F.US.EPH19 1 Min #1 | Study: Test Volume time Study | CHEKING | Index[1271], Volume[5119.000000] - 12:41:58 | 2018-12-18 05:29:02 Chart: Replay 80X: F.US.EPH19 1 Min #1 | Study: Test Volume time Study | CHEKING | Index[1271], Volume[5167.000000] - 12:42:3 | 2018-12-18 05:29:02 as you can see there is a new minute, and yet the index didn't change and the volume that appears is relevant to the previous index. later in the next second the index changes and so the volume is now correct: Chart: Replay 80X: F.US.EPH19 1 Min #1 | Study: Test Volume time Study | CHEKING | Index[1272], Volume[109.000000] - 12:42:4 | 2018-12-18 05:29:02 Date Time Of Last Edit: 2018-12-18 12:08:43
|
| [2018-12-19 17:41:02] |
| |
|
Refer to: Automated Trading Management: Troubleshooting Automated Trading System Behavior Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing |
| [2018-12-19 18:13:44] |
| UserZiltoid - Posts: 57 |
|
Your'e response is not serious. ------------------------------ Please take my simple code above which has nothing to do with trading. it is only prints to the message logs time and volume. If you'll do that you will see that the results in speed 1X and 80X are different. Again, it does nothing but print to the log. No trading. You're replay system is inaccurate regadless any activities the study does. Please read again my previous post and give a corresponding response. thanks in advanced. |
| [2018-12-19 18:16:24] |
| |
|
You're replay system is inaccurate regadless any activities the study does. This is not true. If you want us to undertake analysis this is going to be chargeable and we are not sure when we can get to it. Are you willing to pay for this at 100 USD per hour?
Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2018-12-19 18:19:06
|
| [2018-12-19 18:20:33] |
| UserZiltoid - Posts: 57 |
|
If I'll have to pay you for you to fix the bug so be it. But again this is a bug in you're side. each time the log results are different. did you try to run this simple code? It should take you no more than 5 minutes to see what I am talking about. |
| [2018-12-19 18:38:16] |
| |
|
We would never test something like this using users code. And really it surprises us that you do not even pay attention to the section of documentation we linked to. It takes time to understand your code and what you are doing. This is not part of our support offerings. And our offer to look at this at 100 USD per hour is contingent on our available time. We do not have that time. Maybe we do not get to this until a year from now. Or two years or three years. Use the below code at any replay speed when using the Replay Mode to Calculate at Every Tick and you will get a consistent result of the log messages every time. We verified that. SCSFExport scsf_TestFunction6(SCStudyInterfaceRef sc)
{ if (sc.SetDefaults) { // Set the configuration and defaults sc.GraphName = "Test Function 6"; sc.AutoLoop = 1; return; } if (sc.IsFullRecalculation) return; SCString VolumeMessage; VolumeMessage.Format("Bar Volume at Index %d is %f", sc.Index, sc.Volume[sc.Index]); sc.AddMessageToLog(VolumeMessage, 0); } Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2018-12-19 18:39:36
|
| [2018-12-19 19:03:55] |
| UserZiltoid - Posts: 57 |
|
My code below does the same as you're code with one differece. It also prints the current second; ------------ auto currentTime = sc.GetCurrentDateTime(); auto currentSecond = currentTime.GetSecond(); ---------------------- Please add the current second to the print in you're code and you will see that in speed 1X and 80X the results are not the same. After you verify what am I talking about we can talk about additional payment for you're side if required. P.S The code above that I have attached I have written in order to reproduce a bug in you're side. We are not asking for any analysys in our code. Date Time Of Last Edit: 2018-12-19 19:07:26
|
| [2018-12-19 19:18:41] |
| |
|
Okay we see where you are going wrong. Do not use sc.GetCurrentDateTime(). There is going to be some variation in that current Date-Time during a replay from back test to back test especially if you are varying the acceleration. It is not going to be 100% stable like you think. It cannot be. You really need to use: ACSIL Interface Members - Variables and Arrays: sc.BaseDateTimeIn[] ACSIL Interface Members - Variables and Arrays: sc.BaseDataEndDateTime[] You must use a solid and consistent point of reference for the Date-Time which can only be the bar timestamps themselves. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2018-12-19 19:19:51
|
| [2018-12-20 06:23:45] |
| UserZiltoid - Posts: 57 |
|
Thanks for this response. I will indeeed solve the current issue. Yet, We have noticed that there is another issue. In different speeds, the filled prices are not the same. We suspect that it is related to an issue we saw in different speeds 0.1X, 1X, 40X, 80X using the above TEST program. The log prints fewer prices when running in higher speed. * See the attached screen shot of a test result that we did in speeds 1X and 80X, in the right side (80X) there are missing prices. We assume that it results with different filled prices. Thanks! Date Time Of Last Edit: 2018-12-20 06:27:53
|
 missing_prices.jpg / V - Attached On 2018-12-20 06:22:35 UTC - Size: 250.17 KB - 298 views |
To post a message in this thread, you need to log in with your Sierra Chart account: