Support Board
Date/Time: Sun, 11 May 2025 01:46:25 +0000
[User Discussion] - ATBB Replay vs Tick Replay. Issues with ATBB assumptions
View Count: 292
| [2024-09-02 19:32:52] |
| User254520 - Posts: 16 |
|
My issue is very simple, but it creates very misleading backtest results. Here's the summary best I can describe it: When testing in ATBB mode, if a strategy/trade StopLoss is set short enough, the SIM processor forward-biases the outcome to just the ONE trade that would, in most cases, succeed. When doing tick replay, it obviously corrects the assumption, but it takes an incredibly long time, which is crazy inefficient for doing initial testing. Is there a middle ground I am not aware of, like there is in MultiChart, for instance? They have a setting for accuracy of back-testing so it can go down to lower time-frames to determine accurate order fills without compromising overall performance of bar-based replays. Thank you! |
| [2024-09-03 09:12:41] |
| User61168 - Posts: 442 |
|
the SIM processor forward-biases the outcome to just the ONE trade Please expand what you mean by "forward-biases".Is there a middle ground Try "Calculate Same as Realtime" replay mode which is much faster than tick accuracy. I prefer to keep speed under 240.
|
| [2024-09-03 16:38:10] |
| User254520 - Posts: 16 |
|
@User61168, Will try Calculate Same as Realtime option, but I believe SC recommends Accurate System Backtesting. I'll try both back-to-back regardless and see. To elaborate on forward bias: If you are testing a "fat tails" long strategy with a stoploss. ATBB mode, depending on the distance of OCHL of the candle, will determine direction of the bar, which makes a difference between a trade hitting stoploss and not entering again in tick replay mode vs hitting a big winner. It essentially makes the fast ATBB mode practically useless, unless there's no stoploss and the trade entry/exit are purely rule-based. Or let me clarify... it becomes useless if the SL can be reached within the same bar as trade entry, which is sometimes the case if you enter at session-open or some other vol expansion events. |
| [2024-09-05 07:18:53] |
| User61168 - Posts: 442 |
|
SC recommends Accurate System Backtesting As this is under "user discussion", I can only share my own personal experiences with running simple alert trading strategies via market replays. I have written numerous posts in the past so check it out if they are still visible (and not hidden or deleted). I stopped using "accurate trading system back test mode" 3-4 yrs ago. it's not at all "accurate". "Calculate same as realtime" is also not at all consistent. Both of these have proven to be a complete waste of time for me. Since last 12 months or so, I am only running all my backtests in "calculate at every tick" mode using a single chart in a single chartbook as it is the only one I have found to be reliable especially if your strategy involves flip/stop-and-reverse, scaling in/out at high volatile times such as session open. I also do not use linked charts, run multiple charts in replays, use chart overlay or add additional symbol studies. They all produce inconsistent and inaccurate backtesting results imo.distance of OCHL of the candle - respectfully, unless a strategy enters and exits trades only at candle close, almost all OHLC based "distance measuring" strategies will require tick accuracy for results that you can rely on. Please don't waste your time by taking short cuts. It will frustrate and make you go crazy as your forward testing results will not consistently match backtesting results and you will begin to doubt your strategy... just my 2 cents.Till date, after running backtests for past 5 yrs, I have found market replays to be buggy with numerous and non-reproducible sporadic issues that abruptly creep up and disappear on the next run. It's demonic imo lol I just use it to troubleshoot or regression test my strategies and that's just about all I got to say. To summarize, tick accuracy mode is the only reliable option. Date Time Of Last Edit: 2024-09-05 07:21:49
|
To post a message in this thread, you need to log in with your Sierra Chart account: