Support Board
Support Board
Search Board
Control Panel
View My Posts / Threads
Direct MessagesDate/Time: Tue, 30 Apr 2024 07:18:11 +0000
Reconciling Fills in Trade Simulation Mode from Trade Activity Log to Time and Sales
View Count: 1057
| [2018-10-02 13:32:36] |
| User60271 - Posts: 63 |
|
When using Trade Simulation mode with live data, I am trying to reconcile the reported fills in the Trade Activity Log to the Time and Sales window but the timestamps in the Trade Activity Log on the Fill reports seem to be in milliseconds but the Time and Sales window time stamps seem to be using some other precision. I have set the Time and Sales windows setting to show millisecond/microsecond precision but this does seem to make a difference. I am using the SC Exchange Datafeed with CME and Time and Sales Records stored set to 60000. The specific issue I have noticed is when a fill is reported in the Trade Activity Log it shows the Bid/Ask quote that triggered the fill, however, when tracing the quotes from the timestamp of the original order timestamp in the Trade Activity Log to the timestamp of the reported fill back to the Time and Sales window there are many instances where the quote used to fill the simulated order was not seen in the Time and Sales in the period between the time the order was first recorded in the Trade Activity Log and the time of the fill(and related quote) being reported in the Trade Activity Log. In other words the reported fill used a quote that was never seen during the order lifetime. These fill scenarios occur when the order lifetime is very small ie: milliseconds between placement and fill so there are relatively few quote updates which makes it straightforward to step through the quotes to identify the issue. In live trading the reported fill in these cases would not have been possible using the logic described in https://www.sierrachart.com/index.php?page=doc/TradeSimulation.php#HowOrdersAreFilled. Is there some way to match the timestamps in the Time and Sales window to the timestamps in the Trade Activity Log, with at least millisecond precision, to allow the validation of reported fills when using Trade Simulation Mode to the reported time and sales for the symbol ? |
| [2018-10-02 17:38:17] |
| |
|
In other words the reported fill used a quote that was never seen during the order lifetime. We do not see how this could be the case. Make sure the Time and Sales window is not using any filtering and that it is showing Bid and Ask records. You might also see a message with the order fill indicating that the bid and ask were corrected . This will be in the Order Action Source field in the Trade Activity Log.Is there some way to match the timestamps in the Time and Sales window to the timestamps in the Trade Activity Log, with at least millisecond precision, to allow the validation of reported fills when using Trade Simulation Mode to the reported time and sales for the symbol ? Not with exactness.
Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2018-10-02 17:39:20
|
| [2018-10-03 12:53:36] |
| User60271 - Posts: 63 |
|
I can confirm that there is no filtering on the Time and Sales window. The unusual fills are not flagged as corrected. What does the "correction" process do relative to the reported fill ? A few questions on timestamps : What is does the "Include Millisecond/Microseconds" checkbox do in the Time and Sales window ? The precision seems to vary on the timestamp in the window.Sometimes it seems to increment by exactly 1 millisecond for each entry which seems unusual. What is the source of the timestamp in the Time and Sales windows ? (local machine time/datafeed time/exchange timestamp) What is the source of the timestamp in the Trade Activity log and is it accurate to millisecond precision ? At what point is the timestamp taken when reporting a Fill/Order Modification Ack in the Trade Activity Log in Trade Simulation Mode ? Would it possible that there is queueing/delays (>100ms) in reporting and time-stamping fills in the Trade Activity Log in periods of high order activity relative to the ingress of the related quotes used to match the simulated orders? What is the best format to post details of the behaviour I'm observing ? |
| [2018-10-03 18:13:30] |
| User60271 - Posts: 63 |
|
I've identified an example of the issue even after allowing for the lack of precision of the timestamp on the Time and Sales window. The client sent the order modification request via DTC in response to a price change (bid at 7629.50) that was timestamped at receipt from the feed at 11:41:04.552 by the client. SC and the client are both connected to IQFeed running on the same machine. SC logs acceptance of the change at 11:41:04.552 but the price the order was filled at is 7629.25 (ask at 7629.25) was only available in the Time and Sales prior to that time the the bid changed to 7629.50. The attached screenshot shows that it was not possible to be filled at 7629.25 at the point where the bid moved to 7629.50 within the 1 second period. Let me know if you need any more details. Trade Activity Log (note: timezone seems to have changed on export) Order 2018-10-03 16:41:04.552 [Sim]@NQZ18 Order modify request from DTC client #2. 127.0.0.1. Using new provided price. Requested Price: 7679.25. Requested Quantity: 4 102739 102739 Limit 4 Buy 7679.0000000 Pending Modify Sim1 -2 Order 2018-10-03 16:41:04.553 [Sim]@NQZ18 Simulated order modify complete 102739 102739 Limit 4 Buy 7679.2500000 Open Sim1 -2 Order 2018-10-03 16:41:05.066 [Sim]@NQZ18 Trade simulation fill. Bid: 7679.00 Ask: 7679.25 Last: 7679.25 102739 102739 Limit 4 Buy 7679.2500000 Filled 7679.2500000 4 Sim1 -2 Fill 2018-10-03 16:41:05.067 [Sim]@NQZ18 Trade simulation fill. Bid: 7679.00 Ask: 7679.25 Last: 7679.25 102739 102739 Limit 4 Buy 7679.2500000 Filled 7679.2500000 4 Sim1 Close 2 102739.1 7679.2500000 7678.7500000 Date Time Of Last Edit: 2018-10-03 18:20:06
|
 sc_tas2.PNG / V - Attached On 2018-10-03 18:05:55 UTC - Size: 41.94 KB - 238 views |
| [2018-10-03 18:46:50] |
| |
|
You have a belief that there is something wrong that we need to solve. We cannot see how this is the case. At this point in time, the successfully modified order is checked against the current bid and ask prices. If it could be filled at this moment it would be against those prices. It appears it cannot be immediately filled because it fills 1/2 second later. Order 2018-10-03 16:41:04.553 [Sim]@NQZ18 Simulated order modify complete 102739 102739 Limit 4 Buy 7679.2500000 Open Sim1 -2 Order 2018-10-03 16:41:05.066 [Sim]@NQZ18 Trade simulation fill. Bid: 7679.00 Ask: 7679.25 Last: 7679.25 102739 102739 Limit 4 Buy 7679.2500000 Filled 7679.2500000 4 Sim1 -2 At this time the order fills by examining the order price to the last processed time and sales record and going forward to the most recent one. Perhaps the issue is that on a modification we should reset the last processed record to the current record. But we would have to think about that. And we are not sure that would make any difference here anyway.And also considering how sensitive you are to the slightest degree of imprecision based on all of your questions, we do not recommend using IQ Feed. You should be using the Sierra Chart Exchange Data feed which is a much more higher performance processed feed. There simply is no comparison. The milliseconds you see the Time and Sales window are only a counter. They are not true milliseconds. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2018-10-03 18:50:14
|
| [2018-10-03 19:02:48] |
| |
|
Also in regards to post #3, our time is very valuable. And all of this is thoroughly documented. as you know our documentation is very thorough. Refer to the relevant documentation pages and you will find the exact answers to your questions. At what point is the timestamp taken when reporting a Fill/Order Modification Ack in the Trade Activity Log in Trade Simulation Mode ? Would it possible that there is queueing/delays (>100ms) in reporting and time-stamping fills in the Trade Activity Log in periods of high order activity relative to the ingress of the related quotes used to match the simulated orders? Not really. But from our perspective without going into very detailed and knowledgeable observation of what you are doing we cannot say for sure.What is the best format to post details of the behaviour I'm observing ? We do not believe there is a problem to begin with so any further information is not welcome.
Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2018-10-03 19:13:26
|
| [2018-10-03 19:15:15] |
| |
|
One thing we would say is make sure you are running the current version to ensure that the time of a fill is using the local system time and not the data feed time because we have seen issues using the data feed time.
Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing |
| [2018-10-03 20:23:44] |
| User60271 - Posts: 63 |
|
The data feed really doesn't make a difference with this issue. In the case above it was used to eliminate any variability between SC and the DTC client and it was possible to timestamp the ingress of the data independently of SC to confirm the issue. The latest build is being used and the timestamps taken externally are using the system clock. I am assuming the time stamps in the Trading Activity Log are also using the system clock. The lack of consistent timestamp precision between the Trade Activity Log and the Time and Sales window make it difficult to clearly see the problem but the example above shows the issue even with a 1 second precision. To summarise the issue : In Trade Simulation Mode when filling limit orders, in some circumstances, SC uses a price that existed prior to the time that the order would have existed in it's most recent accepted modified state. At the time the last order modification was acknowledged by SC, the quote (Bid: 7679.00 Ask: 7679.25) used to simulate the fill was never seen in the Time and Sales. It did exist at a point prior to the order modification being received which would indicate the the reference quote SC is using is stale/out of sync when it fills the order. I understand that your time is valuable so is mine. Given the risk that SC is giving users confidence to use a trading method to trade real money based on a potentially incorrect results produced by using the Trading Simulation mode it may be worth your while to look the information I've provided further. |
| [2018-10-03 20:37:36] |
| |
|
In Trade Simulation Mode when filling limit orders, in some circumstances, SC uses a price that existed prior to the time that the order would have existed in it's most recent accepted modified state. The next release has a small improvement in this area because the evaluation rate was at one second and we have dropped this .5 second. This is for orders that do not originate from a chart. In a chart, the evaluation rate is always according to the chart update interval and is user controllable. But nevertheless there are couple of basic points, we cannot make this perfect for performance reasons and also in the real world, there is never going to be a synchronization between the real-time data feed and the order events. We know from experience these can be out of sync and lead to problems especially with trailing stop orders with fast movement. So in the end nothing is ever perfect. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2018-10-03 20:39:23
|
| [2018-10-03 20:54:17] |
| User60271 - Posts: 63 |
|
I understand that any simulation will never be a truly accurate measure of live performance. The Trade Simulation mode is very beneficial and performs extremely well with real time data. The logic used to provide fills for limits orders gives a conservative view of results which is excellent. The issue identified seems to occur only in specific circumstances, possible related to modification of the order. It does however materially impact the results reported for an active trading strategy. Let me know if there is any further detail I can provide.
|
To post a message in this thread, you need to log in with your Sierra Chart account: