Home >> Support Board

Support Board

Date/Time: Wed, 08 May 2024 23:38:10 +0000

How to Post a Message for Support

Missing intraday stock data using SC Exchange Data Feed with real time U.S. Equities

View Count: 2243

[2020-06-19 18:09:06] #1 [Go To First Post]
User975023 - Posts: 18
Hi, I'm evaluating Sierra Chart's real-time stock data feed and it seems to be missing data versus 4 other platforms that I am testing against. Sometimes this isn't too obvious, but when the market slowed down I was able to get a screen capture showing a one minute candle that was red instead of green and was missing an upper wick. In my screenshot are 1 minute charts drawn from real-time data from 5 different platforms. From left to right: Interactive Brokers, Lightspeed, Sierra Chart, TradingView, ThinkOrSwim. I pointed out an obvious candle, but the candle before it is also missing the bottom wick. Is this a known issue with the real-time data you get from barchart.com? Is there a more reliable real-time stock data provider I can use with Sierra Chart? Thanks.


6-19-2020 10-18-35 AM.png / V - Attached On 2020-06-19 17:55:41 UTC - Size: 229.11 KB - 290 views

[2020-06-19 20:02:15] #2 [Go To First Post]
Sierra Chart Engineering - Posts: 104368
We will look this over. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, change to the Teton service: Sierra Chart Teton Futures Order Routing

[2020-06-19 21:49:10] #3 [Go To First Post]
User975023 - Posts: 18
I also have this screen capture for the same stock when there was hardly any volume and Sierra Chart price wasn't updating for around 20 seconds. You can see in this capture that the last trade price marked on the chart was different from the other 4 platforms.


6-19-2020 10-14-22 AM.png / V - Attached On 2020-06-19 21:47:58 UTC - Size: 192.13 KB - 288 views

[2020-06-20 03:55:55] #4 [Go To First Post]
Sierra Chart Engineering - Posts: 104368
Okay we have looked into this, and we are contacting our data provider, Barchart, about this. We think we know where the problem is arising and we are making a change over this weekend to remove the filtering that we think is causing the issue. We are relying upon a flag variable with trades from Barchart Open Feed that indicates that data should not be included in Intraday charts but it may be true more often than should be. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, change to the Teton service: Sierra Chart Teton Futures Order Routing

[2020-06-20 16:16:54] #5 [Go To First Post]
User975023 - Posts: 18
Thanks for looking into this. Can you explain what you have been filtering out? Does this feed include trades that occur on all exchanges, ECNs, and dark pools?

[2020-06-20 17:25:38] #6 [Go To First Post]
Sierra Chart Engineering - Posts: 104368
Can you explain what you have been filtering out? Barchart will need to explain this, and if this is the case we are exceptionally disappointed in the situation. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, change to the Teton service: Sierra Chart Teton Futures Order Routing

[2020-06-21 12:08:16] #7 [Go To First Post]
JM-JO - Posts: 38
I have compared sierrachart data with nanex data for stocks. Clearly the sierra data is highly filtered. Ticks with small volume are filtered. Ticks from same exchange are concatenated. Data from some large exchanges are totally ignored.

[2020-06-21 16:15:30] #8 [Go To First Post]
User975023 - Posts: 18
Hi elji00, Is Nanex available data feed for SC? If so, how much is it and how do I get it? I think it might be the same data as IQFeed, no?

[2020-06-21 17:40:14] #9 [Go To First Post]
JM-JO - Posts: 38
Nanex is not available for Sierra. It is different from iqfeed. It is high end data, with impressive low latency.

[2020-06-21 21:32:47] #10 [Go To First Post]
Sierra Chart Engineering - Posts: 104368
I have compared sierrachart data with nanex data for stocks. Clearly the sierra data is highly filtered. Ticks with small volume are filtered. Ticks from same exchange are concatenated. Data from some large exchanges are totally ignored. Over what date range did you look at this? Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, change to the Teton service: Sierra Chart Teton Futures Order Routing

[2020-06-22 08:06:22] #11 [Go To First Post]
JM-JO - Posts: 38
Attached file is data comparison for AAL stock on February 12. Left columns : data are from Polygon (not Nanex). It is high res data (with exchange ID, ns timestamps) from Polygon that is locally concatenated into ms timestamp (all data from within same ms are concatenated ; price is average from the ticks with same millisecond). Right column : Sierra data "as is". As you can see, a lot of volume and ticks are missing from Sierra feed.


Capture.PNG / V - Attached On 2020-06-22 08:06:00 UTC - Size: 228.12 KB - 270 views

[2020-06-22 08:50:41] #12 [Go To First Post]
Sierra Chart Engineering - Posts: 104368
Okay we will see what our data provider says about this. However, understand, there are some trades filtered out which are not included Intraday charts in order to prevent out of range prices. That could be part of what you are seeing in regards to the difference. At least to some extent. We just had a closer look at this. Any trade under 100 shares is filtered. Those are odd lots. Those are not regarded as trades, which should be included in Intraday charts. IQ Feed is no different. Odd lots are not included when using IQ Feed. So we really do not see a whole lot of difference here considering this. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, change to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2020-06-22 09:03:33

[2020-06-22 15:54:28] #13 [Go To First Post]
User975023 - Posts: 18
Dear Sierra Chart, Are you saying that you will not support inclusion of odd lot trades in your real-time stock data feed? I'm pretty sure this is not acceptable to a daytrader/scalper like me, especially if this means that charts on your platform are not going to match what the majority of other daytraders/scalpers are seeing on their platforms. Please refer to the screenshots I provided earlier to see what others are seeing. As far as IQ Feed filtering out odd lots, I think that is highly unlikely since they advertise themselves as providing unfiltered tick data. I looked into this briefly and it seems that their historical one minute bars do not include odd lot trades, but I'm pretty sure they are included in the real-time data since it's unfiltered. I will sign up for their data feed and report my results here for others. Ideally all the data coming in would be unfiltered and your platform would provide a mechanism for the user to specify what type of filtering they want, if any. That's how I would code it up.

[2020-06-22 16:00:42] #14 [Go To First Post]
Sierra Chart Engineering - Posts: 104368
As far as IQ Feed filtering out odd lots, I think that is highly unlikely since they advertise themselves as providing unfiltered tick data. We will double check the code, but they do provide the odd lots and we also have access to them with the Barchart Open Feed, and we have to see whether they are getting filtered or not, but in the case of IQ Feed the last we remember is that we do not include them and that was what was requested and expected by our users long ago. This confirms what we are saying: I looked into this briefly and it seems that their historical one minute bars do not include odd lot trades, So Sierra Chart should not be processing them in the real-time data because it causes an inconsistency with the historical data downloaded from IQ Feed, and the historical data that we are recording from the real-time feed. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, change to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2020-06-22 16:03:15

[2020-06-22 17:56:52] #15 [Go To First Post]
User975023 - Posts: 18
Well, I prefer unfiltered real-time data. It's normal to me for historical charts to look different because of trade updates and bad tick filtering. All platforms that I've ever used have slightly different data when you reload a chart. That being said, I don't understand what this odd lot filtering nonsense is all about. I can't believe that anyone would want to filter out trades less than 100 shares. How about stocks like AMZN or TSLA trading at $2,700 and $1,000 per share right now? I bet they have lots of odd lot trades. Would you filter those out too?

[2020-06-22 18:33:39] #16 [Go To First Post]
User975023 - Posts: 18
I'm glad the discussion of odd lots came up because I wasn't aware of this issue, but after studying the time and sales along with the charts, I can now say that odd lot filtering is not the issue with your data feed. Interestingly, all my other brokers and data feeds are also filtering odd lot orders from their charts. It looks like barchart is filtering or dropping some of the whole lot orders. I'm looking into which ones, but so far I've seen some trades from FINRA or NASD Alternative Display Facility not make it into Sierra Chart. The main take away is that the Sierra Chart data feed from Barchart is leaving out some trades that all the other brokers and data feeds are not. I'll post back soon with more specific examples.

[2020-06-22 19:45:58] #17 [Go To First Post]
User975023 - Posts: 18
Here are a couple of screenshots showing 100 lot orders not drawn in Sierra Chart, but drawn in Interactive Brokers, Lightspeed, TradingView, and ThinkOrSwim. I'm not going to dig into this any further. This supposedly is your preferred data feed that you provide and that you claim is the only data feed that you officially support. It's obviously inadequate and I shouldn't have to waste my time testing your data and defending my claims.


Attachment Deleted. 6-22-2020 12-01-27 PM.png / V - Attached On 2020-06-22 19:44:32 UTC - Size: 961.23 KB - 268 views 6-22-2020 12-17-55 PM.png / V - Attached On 2020-06-22 19:44:46 UTC - Size: 1.03 MB - 284 views

[2020-06-23 16:06:38] #18 [Go To First Post]
User975023 - Posts: 18
Dear Sierra Chart, I have tested IQFeed with Sierra Chart and it works as expected. This combination produces the real time equities charts that I have been looking for. The charts look identical to those that I see on other platforms. However, it looks like Sierra Chart is still filtering out odd lot trades from the chart. I understand that this is what all the other platforms are doing, but I really would like the option of not filtering out odd lot trades. I see a volume filter in Advanced Chart Settings, but it appears to have no effect. I started looking at the time and sales in Sierra Chart versus IQFeed, and it looks like Sierra Chart is getting every odd lot tick, but it doesn't update the price. For example, on a stock like AMZN where there could be a string of 20 or more odd lot trades, these appear in the Sierra Chart Time & Sales but the prices are all the same as the last regular lot trade. Is this intentional? Any chance of fixing this? I can provide screenshots if you want. Thanks.

[2020-06-24 06:18:10] #19 [Go To First Post]
Sierra Chart Engineering - Posts: 104368
We will consult with Barchart in regards to all of your postings. We will be working on the capability to include odd lot trades in the Sierra Chart Exchange Data Feed for US equities. That being said, I don't understand what this odd lot filtering nonsense is all about. I can't believe that anyone would want to filter out trades less than 100 shares. How about stocks like AMZN or TSLA trading at $2,700 and $1,000 per share right now? I bet they have lots of odd lot trades. Would you filter those out too? In regards to this, there are official industry specifications, which clearly document that trades which are considered odd lots, are not meant to update the last trade price. Now whether that odd lot is always less than 100 shares we do not know. There is a series of sale/indicator codes provided with each trade, and one of those indicates whether it is an odd lot. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, change to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2020-06-24 10:42:47

[2020-06-24 07:08:55] #20 [Go To First Post]
Sierra Chart Engineering - Posts: 104368
We would not give up on the Sierra Chart Exchange Data feed US equities data. We are working on the matter with Barchart. You need to understand the root cause of these kinds of problems. There are many trades which are reported from the exchanges, which cause out of range values in Intraday charts or they are specified not to update the last. We follow the specifications related to this, but if there is some problem with the filtering due to incorrect specifications or other reasons outside of any mistake made, then there can be a problem with this. You also have to see the other side of it, when users start to show us charts that show out of range values that they think are "errors". This is what we are trying to prevent. And you also need to realize that the Sierra Chart Exchange Data Feed is superior to IQ Feed in so many ways. Incorrect filtering of equities trades, is very specific to US stock trading. And is an isolated issue relative to all the other functionality and performance of the data feed. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, change to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2020-06-24 07:13:02

[2020-06-24 10:44:39] #21 [Go To First Post]
Sierra Chart Engineering - Posts: 104368
I'm not going to dig into this any further. This supposedly is your preferred data feed that you provide and that you claim is the only data feed that you officially support. It's obviously inadequate and I shouldn't have to waste my time testing your data and defending my claims. Yes we understand. But there is a reason why this is the feed that we can only support and we eventually have to phaseout others due to financial cost reasons with what we have to pay for the feed which we get through get Barchart. We will get the data feed working properly for you. We know that we can because based upon some information that Barchart gave us, the data that you think is missing, is available on their data feed. So we just need to figure out why it is not getting sent through and then once we get that resolved it will be fine. We will reimburse you for this month (which we already did), and we can pay for the feed costs next month if you become a Sierra Chart user. We have released US equities odd lot trade support on one of our servers for the Sierra Chart Exchange Data Feed. You will start to see them. If you do not see them, just simply reconnect to the data feed: File Menu: Procedure to Reconnect to the Data and Trade Servers Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, change to the Teton service: Sierra Chart Teton Futures Order Routing Date Time Of Last Edit: 2020-06-24 10:54:18

[2020-06-24 15:07:14] #22 [Go To First Post]
User975023 - Posts: 18
Hi Sierra Chart, Thank you for looking into this issue and not giving up. I see that you have a great product and I plan on using it for my trading. Accuracy and speed of data are critical for my style of trading and I am willing to pay more for that. Hopefully you can get your equities data feed to work as well as IQFeed. Based on your responses, the issue might be packet loss as I understand IQFeed uses TCP so it doesn't lose any packets, and I wasn't able to see any pattern in which trades were missing from the Barchart data feed. I am getting the odd lot orders from your data feed now and it seems to be working as expected. I am recording a live session comparing the two feeds and I'll get back to you if I see any more missing data. Regarding including odd lot orders on the charts, it seems like it's working really well, but you might want to add an option to toggle filtering of odd lot trades in the data feed and/or chart settings. Even though odd lot volume currently accounts for 30-40% of high priced stocks, I don't know if everyone wants them included in their data feeds/charts if they are currently not considered official. Thanks!

[2020-06-24 17:59:15] #23 [Go To First Post]
User975023 - Posts: 18
I haven't seen any missing data so far. The only anomaly I've seen today seems to be caused by slight difference in timestamps causing a few candles to open/close at slightly different places. I want to say that this looks like a possible rounding error. Two cases in AMZN I examined today involved timestamps in IQFeed at 11:41:59.969590 and 11:38:59.973654 getting bumped up to the next candle at 11:42:00.000 and 11:39:00.000 in your feed. Your time and sales appear to have artificial millisecond values, so I can't really tell what the incoming timestamps are in your feed. Thanks for the odd lot change. I'm really liking what I see now!

[2020-06-25 15:23:24] #24 [Go To First Post]
User975023 - Posts: 18
Hi Sierra Chart, Just wanted to report a few things I saw today during the US market open in some stocks. One major issue is that your data feed with odd lots is including trades that are Sold Out of Sequence. In IQFeed these trades can be combined with Odd Lots or Not (i.e. Trade Conditions equals "SOLDOSEQ" or "SOLDOSEQ/ODDLOT".) Hopefully you can filter these out while leaving the Odd Lots in. Not sure if anyone would find these out of sequence trades useful. I definitely do not want to see them in my charts because they make the price jump all over the place. On the plus side, your data feed was performing better than IQFeed for me today. Your feed was slightly faster and there were a few times when IQFeed completely stalled for seconds at a time. Thanks

[2020-06-26 05:52:48] #25 [Go To First Post]
Sierra Chart Engineering - Posts: 104368
Just so you know there is a lot here in this thread, and there are several items for us to work on/review and we are still communicating with Barchart about some related details. We are going to be working on everything as soon as we we can. But just allow a couple of weeks. The plan is for us to record all data and then add filtering options to let the user control what they want to include in charts or not. This will take some time to implement. Sierra Chart Support - Engineering Level Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy: https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation For the most reliable, advanced, and zero cost futures order routing, change to the Teton service: Sierra Chart Teton Futures Order Routing

To post a message in this thread, you need to log in with your Sierra Chart account: