Support Board
Date/Time: Wed, 01 Apr 2026 04:13:57 +0000
Inconsistency in replay mode backtests?
View Count: 234
| [2026-02-14 05:56:30] |
| RefillsFree - Posts: 3 |
|
I have been working on an ASCIL script, and replaying the script on Sim accounts on Sierra to then backtest and collect trace activity data. My problem is that I will run the script on a set time, then run the script again, same start time, same duration, same instance of Sierra, same computer, yet I will get slightly different results, up to around 15% PnL deviation. My script runs intrabar. The script follows the same logic I want it to, but the entries and exits are slightly different, and often the trade count varies by a small amount. What multiple runs of the script would look like on a PnL graph would be very similar lines, yet they slowly diverge over time to produce different ending balances. Is this more likely a Sierra Chart error, code error, or something else? Has anyone else had this problem and solved it? Any tips? I'm very open to working together to solve this! If I left out any details pls ask. I attached the profit graphs of 5 of my backtests, same script, same computer, same start and end time, no changed settings. |
 Screenshot 2026-02-14 005417.png / V - Attached On 2026-02-14 05:54:42 UTC - Size: 172.37 KB - 36 views |
| [2026-02-14 16:58:49] |
| ForgivingComputers.com - Posts: 1207 |
|
What Replay Mode are you using? I use Accurate Trading System Back Test Mode, and I try different speeds to see if I get different results. I suggest testing at Speed = 60 with 60 second display update. That will test a trading day in about 7 minutes. You definitely do not want to use Standard Replay mode if you are doing Intrabar trading. I find the higher the speed, the less accurate the results. Getting data from other charts can impact the results at higher speeds. That may or may not have any impact on consistency, but if you are trying to back test too fast, that could explain the different results. Date Time Of Last Edit: 2026-02-14 17:00:18
|
| [2026-02-14 21:54:12] |
| RefillsFree - Posts: 3 |
|
Thanks that worked! I was using standard replay on 1-2x speed. I just did two tests on the Accurate Trading System mode, much more consistent returns. Thanks again. Another question, the accuracy/consistency of a backtest at a higher speed, would that increase with a stronger cpu? I'm considering upgrading or borrowing someone elses computer to run tests. |
| [2026-02-14 23:56:45] |
| ForgivingComputers.com - Posts: 1207 |
|
the accuracy/consistency of a backtest at a higher speed, would that increase with a stronger cpu
Good question. I am interested to hear what you find. |
To post a message in this thread, you need to log in with your Sierra Chart account: