Support Board
Support Board
Search Board
Control Panel
View My Posts / Threads
Direct MessagesDate/Time: Thu, 25 Apr 2024 01:37:14 +0000
Replay (back-testing) from V2123 to 2200 is different
View Count: 690
| [2020-11-25 18:52:50] |
| Ernie Doo - Posts: 46 |
|
Dear SC team, I developed an ACSIL algorithm and was about to put it to run live. However, yesterday when opening Sierra Chart, I read in the opening messages that I better update the software for some modifications done to replay. So I updated from Version 2123 64-bit (31804M) to Version 2200 64-bit (34642:34644). I also read in the release notes that on V2196 a problem was resolved where replay does not continue at the transition from one futures contract month to the next, when the chart is using the Continuous Futures Contract option. So my problem is the following. I used the same code/algorithm and compiled under version 2123 and ran a test from 2018 to today and documented the results. Then I updated SC to V2200 and compiled the same code, then re-ran the same test and documented. It turns out that with the same algorithm I am getting different results. The configuration I use in the replay window is: "Charts with the same Link Number" (I am running two different windows, 1 Min Chart and 1 Second Chart) "Calculate Same as Real Time" "Skip Empty Periods" Now, in both charts I use the Continues Futures Contract option: "Continuous Futures Contract - Date Rule Rollover" PLEASE! How can I be getting different results? Which results should I trust? Thank you for your help! Ernesto |
| [2020-11-25 19:51:07] |
| User657945 - Posts: 226 |
|
HI Ernesto I did read your message and of course I have no answer for you but a question: how much time did it takes to you to test that huge period? at what speed you use the back testing? When I use the replay chart I need to set from 2to 5 times the speed otherwise I get not attendible risults. thx for replying |
| [2020-11-26 09:10:52] |
| Ernie Doo - Posts: 46 |
|
Hello User657945, I have found out after a couple of years of using SC that it is very accurate during back-testing. The speed I run my replays is at the highest speed 30720 and SC has always been accurate, meaning that I would run the same test twice and I will always get the same result; and I am talking about 2-3 years back-testing with in between 1,000 and 8,000 trades within that time, and as I said before, the results were always accurate. So, if you are not getting accuracy and repeatability in your results then you should check what might be the problem. As a note, I do not have any other studies loaded into my charts, just my algorithms. I found out that the more studies you add the more difficult and slower it is for SC to replay tests. This makes sense as SC would have to consider a lot more information than if you had no studies. It takes me about 1.5 hours to run a test from 2018 until today. It depends which market you are testing. In my case the duration of the test varies depending on the market I use. It also depends on the capacity of your computer. I tried some tests on a weaker computer and it just took forever. Having explained all of the above, this is why I need SC's team response, because this is the first time I am getting different results with the same code and I don't know why. I would like to know if the changes they made to the Replay function has something to do to it. SC team, whenever you are ready :) Thanks PS. @ SC Team: I am also reading the posts regarding the symbols new format. Could this have something to do with my problem? I ran the test I am talking about for the mini S&P 500 (symbol ES) Date Time Of Last Edit: 2020-11-26 10:09:36
|
| [2020-11-26 09:42:57] |
| Ackin - Posts: 1865 |
|
Many traders have had a problem with: Chart Settings: Always Simulate Bid and Ask Prices During Replay (Chart >> Chart Settings >> Trading menu) As a next step I would try to downgrade to version 2194 (version before the historical market depth data) and also version 2149 (changes in the functions with time). Find out if there is a difference or not after this intervention ... Otherwise, I doubt that the Sierra team will answer you other than by referring to the documentation ... 77 versions jump: there has definitely changed a lot |
| [2020-11-26 10:25:27] |
| User657945 - Posts: 226 |
|
Hi Ernesto thx for replying! well after your explanation I think I have a big issue becasue I simply can't do that I do not get any data if I use that speed.Probably one of the issue is that I use the Trading system alert of sierra that need more than one study to be in the charts and I have also 2 charts with study overlay and I'm using bar volumes and not open close simple bars.
|
| [2020-11-26 18:47:17] |
| Ernie Doo - Posts: 46 |
|
Hello Ackin, Thanks a lot for your comments. I took up on your suggestion and decided to start testing from the SC version I was using (2123) up until the current one and find out where does the whole situation goes crazy. It is the only way I will find out what is going on. However, in the future this means I should be always up to date with SC's version and immediately test my algorithms to see if anything changed. Otherwise I can run into the same problem. Is this the way you do it? Regards, Ernesto |
| [2020-11-26 19:28:54] |
| Ackin - Posts: 1865 |
|
What Is New 2196
https://www.sierrachart.com/index.php?page=doc/Whats_New.php-Resolved problem where replay does not continue at the transition from one futures contract month to the next, when the chart is using the Continuous Futures Contract option. 2164 -Added new Jump type option to the replay control panel to jump to the next specified time increment. 2124 -Support for replaying past simulated trade fills for non-simulated/live trading. However, in the future this means I should be always up to date with SC's version and immediately test my algorithms to see if anything changed.
The versions I wrote above (2194 and 2149) were borderline for major changes in Sierrachart's behavior + ver 2124,2164,2196 with the Replay changes. It is always accompanied by further updates to calm the situation. This is a tax on the fact that Sierra is constantly innovating - it's good ... users should be back several versions or they must be flexible and sometimes in case of minor issues downgrade back to the SC version which was stable for them.Is this the way you do it?
In your case (from the user's position) probably yes. I'm a developer, so I test the strategy in a different way and with my own tools.
Date Time Of Last Edit: 2020-11-26 19:31:51
|
| [2020-11-27 12:08:47] |
| Ernie Doo - Posts: 46 |
|
Ok, so I started to test all versions starting at 2123. Until Version 2157 everything was working the same. Then I installed version 2164 and my results were different. The only thing I found related to replaying in the release notes is: "Added new Jump type option to the replay control panel to jump to the next specified time increment". What does this mean? How does it work? All this time I selected "By Time (hh:mm:ss.ms)" as a Jump type with Unit = 00:00:00.000 I still don't understand why would this give me different results, but then again I don't know how this works. I really could use your help. Thanks in advance! Ernesto Editted: PS. I also found these 2 modifications: Corrected some small edge case issues involving the new integer based microsecond Date-Time object within Sierra Chart. Corrected an issue with the determination of the trading day date in some less than common cases. I think one of these might be the problem because I am finding out that in some trades that do repeat themselves in both tests the Minute or second are different whereas before they were always the same. PS2. I am still trying to figure out why version 2164 behaves differently with my same code. I noticed that some of the trades that happened in the previous version happen now at other points in time. I focused on the changes made to SCDateTime as I thought somehow this affected my execution. However, I do not think this is the case. I think somehow the way SC behaves when filling an order during replay is now different but I could not find any notes regarding this. Do you think this could be the case? Date Time Of Last Edit: 2020-11-29 20:56:25
|
| [2020-11-30 10:45:48] |
| |
|
There may have been at some point a change to the implementation of this Replay Mode: "Calculate Same as Real Time" We would have to check though but that is quite involved. Any change would be a technical improvement. That setting is relatively new. For the most consistency you would want to use this replay mode: Accurate Trading System Back Test Mode. Refer to: https://www.sierrachart.com/index.php?page=doc/ReplayChart.html#ReplayMode Whatever the reason for the difference you are seeing, you should be able to precisely identify it. Since it is your trading system which is submitting the orders. Through logging, you should be able to identify the change. You could count the number of times your study function is called and log this time at each call: sc.LatestDateTimeForLastBar That is a good place to start. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2020-11-30 10:46:04
|
| [2020-11-30 11:58:03] |
| Ernie Doo - Posts: 46 |
|
I didn't think about changing the replay mode to Accurate Trading System Back Test Mode. That is a great idea. I will give it a try and will let you know in case somebody is running into the same issues like me. I have been logging and testing continuously. I hope this brings something new to my testing. Thanks for your help! Ernesto |
To post a message in this thread, you need to log in with your Sierra Chart account: