Prices / Data Falling Behind
This help topic explains various reasons for prices/data to fall behind or lag. And solutions for this issue. Sierra Chart displays all data from the server immediately as it is received without delay.
- 4.1 - Different Servers
- 4.2 - Chart Update Interval Too High
- 4.3 - Data Processing Not Keeping up with Data Feed
- 4.4 - Delayed Data
- 4.5 - Problem With Data Server
- 4.6 - Determining the Source of the Lag
- 4.7 - Limited Bandwidth / Wireless Connections
- 4.8 - Data/Trading Service Specific
- 4.9 - Reducing Network Bandwidth Requirements for Sierra Chart
- 4.10 - Market Not Active
- 4.11 - Incorrect Determination of Data Delay
- 4.12 - Sierra Chart Exchange Data Feeds
- 4.13 - Delayed Data Causes Delay with Order Fill Markers
- 4.14 - Firewall Software Can Cause Delays
- 4.15 - Trade and Current Quote Symbol
- 4.16 - Monitoring Sierra Chart Data Feed Performance from Perspective of Server
4.1 - Different Servers
If you notice a difference between the data displayed in Sierra Chart compared to your Data or Trading service software or another service, then this may be because that software program is connected to a different server compared with the server Sierra Chart is connected to.
The reason for this is that the Data/Trading service may have many servers. The server you are connected to with the Data/Trading service software may be different than the server you are connected to with Sierra Chart and this could be one of the reasons for any difference.
Prices could be ahead of what you see or behind. Try disconnecting with File >> Disconnect and reconnecting to the data feed with File >> Connect to Data Feed to get a different server.
Reconnecting may not necessarily give you a different server, it all depends upon the Data/Trading service you are using. In many cases reconnecting will not help.
4.2 - Chart Update Interval Too High
The Global Settings >> General Settings >> Chart Update Interval setting may be set too high. For complete details about this setting, refer to Chart Update Interval.
If you make the Chart Update Interval too low, like under 100 ms, then you can actually introduce a lag because you may overwhelm your CPU with processing. Sierra Chart does not recommend using a Chart Update Interval less than 100 milliseconds.
Each individual Chart/Trade DOM can override the global Chart Update Interval. The chart specific setting is in Chart >> Chart Settings >> Advanced Settings. Change the Chart Update Interval in Milliseconds setting to what you require or set it to 0 to use the global setting in the General Settings window.
When you want to use a fast Chart Update Interval (a low value), it is best to set it directly on the individual Chart/Trade DOM rather than globally. This is so you do not create a high CPU condition within Sierra Chart.
Set the Chart Update Interval as you require and it can be set individually for those particular charts that you want to have update at a faster interval.
Additionally it is recommended to use sub instances of Sierra Chart to distribute processing load and to use for charts which have studies which have a significant amount of processing time. This helps to increase the update frequency when using a lower Chart Update Interval, in the main instance of Sierra Chart. For more information, refer to Sierra Chart Configuration for Most Low Response Time Trading.
4.3 - Data Processing Not Keeping up with Data Feed
If market data prices are falling behind or lagging, then this may be due to the fact that your computer is not processing the market data received fast enough. There are various reasons for this.
There could be a problem caused by antivirus/anti-malware/anti-spyware software causing slow file processing which can impact the speed at which data is processed. This may not necessarily cause high CPU usage. To quickly rule out your antivirus/anti-malware/anti-spyware software as being a problem, completely disable it and restart your computer and see if the lagging issue is gone. If you are running an up-to-date version of Sierra Chart, >= 1053, then Sierra Chart is much less affected by this kind of potential issue.
The next step is to determine the source of the lag by following the instructions in help topic 4.6.
Due to how Sierra Chart processes incoming market data and the method by which the user interface is updated, slow data processing on your computer at best typically only will cause data lagging issues that are less than a second. Although, antivirus/anti-malware/anti-spyware software could create longer lags. Generally this is uncommon though. Unless there is a large amount of data coming into Sierra Chart and the processing of that data is time-consuming.
For instructions to determine if the main thread of execution in Sierra Chart is using all of the available CPU time and this is contributing to the lag problem and how to solve this, refer to 30.37 - Determining CPU Load from Sierra Chart.
If you do determine that prices are falling behind/lagging because your computer is not processing data fast enough and you have followed all of the suggestions in help topic 30 and continue to notice a problem, then you may need a faster CPU or increase your computers memory.
4.4 - Delayed Data
If you notice prices are behind the current time by 10, 20, or 30 minutes, this means you are receiving delayed data. Contact your Data service for assistance with receiving real time data. If you are receiving delayed data for a symbol, then updates may only be received once a minute. Due to exchange rules, data providers can not provide tick by tick data for some delayed markets.
4.5 - Problem With Data Server
Prices may be lagging or falling behind because the data server is not providing you the latest prices. This is not an issue with Sierra Chart. It is an issue with the service you are using. As as stated above, Sierra Chart displays all data immediately as it is received. There is no delay whatsoever. Although, we do explain some issues in the the other subsections on this page which could create a lag other than a problem with the data server from the Data/Trading service you are using.
4.6 - Determining the Source of the Lag
If the file processing is slow on your system due to antivirus/anti-malware or related software or there are other high CPU load issues related to charts, then follow the instructions below to rule these out as problems. If you still notice a lag similar to when you had charts open, then the issue is definitively with the data server (4.5), a connectivity or limited bandwidth problem (4.7).:
- Restart Sierra Chart. Do not open any charts or other windows. If there are charts or windows open, then close them. The Sierra Chart software should not be tracking any symbols at this time. To confirm this you will see ST: 0 or Not Connected in the Status Bar window below the main menu. Make sure the Status Bar is enabled through Window >> Status Bar .
- Select File >> Disconnect.
- Connect to the data feed, by selecting File >> Connect to Data Feed.
- Select File >> New Quote Board on the menu.
- On the Quote Board menu, select Settings >> Set Symbols.
- In the edit box beside the Find button on the Symbol List window, enter one of the main symbols you follow. Or press the Find button to select one.
- Press the Close button.
- You will see various fields of data for the symbol. Watch the Last price and possibly the Last Time of the last trade, to still see if you notice the data lag.
- If you do not notice a lag compared to when you had charts open, then this clearly indicates there is some inefficiency on your computer or with your Sierra Chart set up. For solutions to this problem, refer to help topic 30 and most specifically be sure to follow 30.8 in addition to the other help topics.
- If you continue to notice a data lag, then the issue will be explained in help topic 4.5 or 4.7 on this page.
4.7 - Limited Bandwidth / Wireless Connections
A lag can also be caused by limited bandwidth between your computer and the Data/Trading service data server.
Reduce the number of symbols being used in Sierra Chart to reduce the bandwidth. Refer to Reducing Number of Symbols Being Tracked by Sierra Chart.
Wireless connections also can reduce the amount of available bandwidth and also increase transmission latency from the server. It is recommended to always use a wired connection when possible.
4.8 - Data/Trading Service Specific
In the case of when using the Interactive Brokers Trading service, if Global Settings >> Data/Trade Service Settings >> Data and Other Settings >> Record True Real-Time Data in Intraday Charts is set to True, then chart bars themselves will update at most every 5 seconds.
However, you will see the last trade price box on the right side Values Scale move more frequently. For more information, refer to True Real-Time Data on the Interactive Brokers page.
4.9 - Reducing Network Bandwidth Requirements for Sierra Chart
To reduce the network bandwidth requirements for Sierra Chart do the following two things:
- Set the Global Settings >> Data/Trade Service Settings >> Intraday Data Storage Time Unit to 1 Minute.
- Enable Data feed compression. Set Global Settings >> Sierra Chart Server Settings >> Use Real-time Data Compression to Standard Compression.
This setting only applies to Sierra Chart provided data feeds (For example: Crypto currency data, Forex data, CFD data, Delayed Exchange Data Feed, Denali Exchange Data Feed, Sierra Chart Exchange Data Feed). It does not apply to external data feeds from other services.
4.10 - Market Not Active
One reason for an apparent delay with data for a particular symbol is that when you expect trading activity to occur, it is not actually occurring.
It must be understood, that symbols do not continuously update. They will only update when data is received from the data feed for the symbol. Updates will only occur when a trade occurs or an update with bid/ask data.
In the case of a micro futures contract, it will often have less trading activity than the larger sized contract.
It cannot be expected that there is trading activity occurring every single second or even multiple times a second. Even in an active market, there can be several seconds between trades at times.
For example, at the start of a new minute, you will not necessarily see a new 1 minute bar if the chart bars are set to 1 Minute per bar if there is not a trade at the very first second of the minute.
If the first trade of the minute occurs 3 seconds into the minute, then a new bar will appear 3 seconds after the start of the new minute. The bar in this case will be considered complete 57 seconds later. The Start Time set for this bar will be at the zero seconds time assuming the Chart >> Chart Settings >>Session Times >> Start Time for the chart has the seconds set to :00 like 09:00:00.
One way to determine that the market was not active for a period of time when you are not seeing updating for a particular symbol, is to open a Time and Sales Window for the chart of that symbol. If you see there is no updating of the chart for the symbol for about 10 seconds and when you look at the trades at that time, you do not see any trades with a timestamp during that time, then there was no trading.
The image below of a Time and Sales window shows a pause of nearly 10 seconds where there were no trades or best Bid/ Ask updates. This is simply not unusual. Markets do not continuously trade.
If you are a new user, and you have been referred to this section, then understand, there is not any problem and you are seeing all of the market activity for the symbol.
Another section related to what is described here, is 4.15 - Trade and Current Quote Symbol.
4.11 - Incorrect Determination of Data Delay
If you are determining that data is delayed for a particular symbol by comparing the Time at the top line of the chart to the local system derived time on the title bar of Sierra Chart, then you may notice an apparent delay because your computer's clock is ahead of the actual time.
This is also true if you have the Countdown Timer study on the chart and the Display Continuous Time Countdown Based on Real-Time Clock Input is set to Yes, and you are comparing the chart bars to this countdown timer.
For example, if you notice that the countdown timer starts a new Minute and 1 Minute chart bars consistently start later, this is an indication that your computer's clock is probably not set correctly. Otherwise, the 4.10 - Market Not Active subsection applies.
To correct the computer's clock, refer to help topic 38.
4.12 - Sierra Chart Exchange Data Feeds
When using the Sierra Chart Exchange Data Feed or the Denali Exchange Data Feed, there are things that you can do to reduce the amount of bandwidth the data feed uses and also increase/decrease the transmission speed.
The Sierra Chart provided data feeds are very advanced and provide configuration capabilities not available with any other data feed. And there is also detailed performance monitoring information available.
Select Global Settings >> Sierra Chart Server Settings.
Set the Use Real-time Data Compression setting to Standard Compression. Or for higher compression use High Compression. When using High Compression make sure you are running an up-to-date version of Sierra Chart. Using compression, reduces transmission delay. Since there is less data transmitted over the network. The compression and decompression times are very tiny, usually about 2-3 microseconds. It is not recommended to use No Compression.
Set Remote Buffer Delay Send Time in Milliseconds somewhere between 100 to 200. The higher the number the more efficient the data transfer. It results in a higher compression ratio and fewer packets and therefore fewer return acknowledgment packets and less chance for packet retransmission which results in delays if your Internet connection is not good.
This setting can be as low as 10-20 ms for the fastest updates but that can also potentially cause a problem with packet retransmission and therefore delays. It depends upon the quality of your Internet connection.
It is perfectly fine to set Remote Buffer Delay Send Time in Milliseconds to a value of 500 milliseconds or 1000 milliseconds or even higher. If you are fine with updates every second (which includes all of the trading and market depth activity within that timeframe), then use 1000 milliseconds.
Reduce the amount of Market Depth levels subscribed to through the Maximum Depth Levels setting. For complete details, refer to Max Depth Levels.
Press OK to close this settings window.
Reconnect to the data feed after making changes.
Even if you do not want to make these changes, do them temporarily to see if you notice an improvement. The purpose of these changes is in part to help diagnose the source of the issue you are having related to data delays or stopping data.
A tool to better help understand connectivity problems is the detailed heartbeat logging which is available. Refer to 4.16 - Monitoring Sierra Chart Data Feed Performance from Perspective of Server this page.
4.13 - Delayed Data Causes Delay with Order Fill Markers
When using the feature to Display Order Fills on the Chart, if market data is delayed, not intentionally delayed, but due to a lag for the different reasons explained on this page, then until there are chart bars in the chart which contain the time of the order fill, an order fill marker is not going to display on the chart, even if the actual fill execution report is received before the market data. When the market data is received which creates the chart bar which contains the time of the fill, then at that time the order fill will appear assuming the order fill execution report has already been received.
So therefore a data lag causing a delay with the display of the chart bars containing the time of the order fill, can cause a delay with the display of the order fill.
However, there is another potential problem. Even if there is already a chart bar displayed which contains the time of the order fill, if the fill execution report is delayed because of lagging market data, then the order fill is not going to be displayed until that execution report is received. This will be the case with services which use the same connection for market data and trading. When using CQG, this is the case.
The solution to the above problem is to use a service that does not use the same connection for market data and trading. The Sierra Chart Order Routing Service uses a separate connection for market data and trading.
4.14 - Firewall Software Can Cause Delays
When using firewall software other than the Windows Firewall which block network connections or examine network data, these programs have been known to to sometimes cause delays with received data and certainly connection problems. If you are noticing data is delayed or lagging or historical data is downloading slowly, other than a standard exchange required delay, try disabling your firewall software and restarting your computer.
4.15 - Trade and Current Quote Symbol
When you have set a Trade and Current Quote Symbol on the Chart or Trading DOM and Chart >> Chart Settings >> Use As Trade Only Symbol is not enabled, then the last trade price and the current Bid and Ask prices are only going to be updating according to this symbol and not the main symbol of the chart.
This is another reason why you can notice what seems to be a data "lag" if the Trade and Current Quote Symbol is a lesser traded symbol than the main symbol of the chart.
4.16 - Monitoring Sierra Chart Data Feed Performance from Perspective of Server
When there are problems with the data feed lagging, or if the data feed stops, it helps to see performance and communication related data from the server related to the connection. This is supported with Sierra Chart provided data feeds.
Overall the feedback of the Sierra Chart provided data feeds including the Denali Exchange Data Feed is very good and these data feeds do solve problems that users encounter with other services.
However, since the public Internet is involved, there can be lagging data and other issues. All data feeds are affected by this. The Sierra Chart data feeds are not immune from this. They simply cannot be.
Data feed lagging or stopping issues are due to the quality of connectivity between the server and Sierra Chart on the users computer, and the available bandwidth.
We have developed a method for you to monitor the performance of the data feed from the perspective of the server.
You need to be running an up-to-date version of Sierra Chart which supports this logging.
To enable this logging, you need to set Global Settings >> Sierra Chart Server Settings >> Detailed Heartbeat Logging to Yes.
Here is an example of the log lines added to the Window >> Message Log:
Description of available fields:
- ServerReceivedClientHeartbeatSecondsAgo: you do not want this number to exceed 20. If it exceeds 22, it indicates a degraded connection quality going from the client back to the server which is going to cause data lags or stopping data with a high probability.
- NumberOfOutstandingSendBuffers: This number should be low and should not exceed 5. This also indicates a degraded connection quality if it is a high number. When the market is very active, the number can go up but it should stay under 10 (Although in our experience this is even high). If this is high, that means the network connectivity between your server and the computer is not able to handle the data rates or is degraded in some way. Try reducing bandwidth if this is the case.
If the number is over 10, it generally will indicate the TCP acknowledgment packets are not getting through to our server. This can be the result of degraded Internet connectivity between your computer and the server. This would be outbound connectivity from your computer. What happens in this case is that at the operating system level on our server, the TCP protocol will stop the data flow. This is normal TCP behavior.
- TransmissionDelayInMilliseconds: This indicates the amount of time to allow data to be buffered up into a larger packet before being sent. This is necessary to maintain connection efficiency and overall performance.
- ServerSendBufferSizeInBytes: This indicates how much data is being held on the server still pending to be sent to the user. If this starts exceeding 100 K you likely are receiving lagging data.
- ActualMessageDelay: First, this number is likely inaccurate. It is imprecise and can be off by several seconds. It requires a very accurate clock on the server, which is not assured and a very accurate clock on the client side which is not assured and they both have to be in sync with each other which is next to impossible. So at best it is just an "interesting" number which may be simply wrong. However, and general you want to see this less than 1.
- DataCompRatio: The compression ratio when data compression is being used.
- UncompressedBytes: The number of bytes transmitted during the connection before compression.
- CompressionTime: The total compression time of all compressions during the connection. This is in seconds. .000001 represents one microsecond.
- NumCompressions: The number of compressions during the connection.
4.17 - Providing Heartbeat Messages to Sierra Chart Support
If Sierra Chart Support requests the heartbeat messages from you, then follow the instructions below.
- Set Global Settings >> Sierra Chart Server Settings >> Detailed Heartbeat Logging to Yes and then press OK.
- At the time there is some connectivity issue requiring sending of the heartbeat messages, then at that time open Window >> Message Log.
- Scroll all the way to the bottom of the Message Log and provide the last 50 lines or so of the Message Log by selecting the first of the 50 lines and then hold the Shift key and then scroll to the last line and select it and release the Shift key.
- On the Edit menu of the Message Log, select Copy Selection.
- In the message box area for the message to send the Sierra Chart support, right click and select Paste.
*Last modified Sunday, 16th May, 2021.