Login Page - Create Account

Support Board


Date/Time: Sun, 19 May 2024 05:06:55 +0000



Chart Data Reloading Times

View Count: 2131

[2015-10-01 02:10:55]
Trader Ra - Posts: 87
just out of curiosity..

Why does SC reload all the data on a chart when switching time frames? Is it simply because sc is coded in C++ and there's no way around it? sure its not much of an issue if we dont download tick data but then we're compromising on accuracy. Why can't SC switch time frames on a chart dynamically and quickly without having to re-load all of the chart data every time. That would save sooo much headache. This is probably the biggest downside of using SC. Reason i bring it up is because i know of other platforms that don't have this issue.

Dont get me wrong.. i love SC for all the value it provides but this one aspect bugs me and i dont understand why (im not a programmer).
Date Time Of Last Edit: 2015-10-01 02:39:51
[2015-10-01 02:51:13]
Sierra Chart Engineering - Posts: 104368
You are aware that the reloading only occurs from the local storage device, do not you?

It certainly does not make a remote request for data. And if you start asking other Sierra Chart users how long the data loading takes for Intraday charts, they will indicate that it is very fast.

It can be a problem if you are loading a large number of days, have a short timeframe per bar, and the Volume at Price data has to be calculated.

In any case, there is nothing to change here. This is as fast and efficient as can be, and it cannot be beat. We find it hard to believe if you do the exact same thing with another program, you will get it to load data faster. We do not think anyone will beat Sierra Chart in this area.

So this posting is puzzling to begin with. Possibly you are running antivirus/anti-malware software creating a problem?

Refer to help topic 30:
http://www.sierrachart.com/index.php?page=doc/helpdetails30.html

There were some recent programming changes to make the code more organized with building chart bars and that may have slightly increased the loading time. Not by much. We would think at most, that might extend the amount of time by 5% or 10% at most. And no one has ever mentioned any consequence of that and it was not noticeable to us.
Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service:
Sierra Chart Teton Futures Order Routing
Date Time Of Last Edit: 2015-10-01 02:52:59
[2015-10-01 02:55:55]
Sierra Chart Engineering - Posts: 104368
You need to remove all antivirus and anti-malware software from your system completely.

And you need to get an SSD. And you should be running a fast CPU. Data loading from the local disk file is incredibly fast in Sierra Chart.

And we are certain, that no one is going to beat Sierra Chart with an identical test in this area.
Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service:
Sierra Chart Teton Futures Order Routing
[2015-10-01 03:29:37]
Trader Ra - Posts: 87
maybe i should have elaborated a little more in my post.

yes im aware data loading occurs from local drives. i have a top notch dedicated trading system with ssd drives, processors etc and no additional software. in fact, i also removed all bloatware and unnecessary services from my windows system so hardware/software is not the issue. i read through and applied all the recommendations in your documentations as well.

to clarify, i was referring to large number of intraday days which is necessary for my methodology because i dont rely on just indicators and smaller time frames. yes, I use volume profiles that captures months of data to get a big picture view of the markets. smaller number of days (even 50-100) load fast on my system and i have no issue there... and i dont mind the initial time it takes to load everything when i start SC.

im talking about using it after all the data is initially loaded and i have to switch time frames on a chart with a year's worth of intraday tick data loaded, that's where the issue is. now you would say well duh.. i have too much data loaded.. end of puzzle.

my question is not about whether sc is fast or efficient or stable .. i know it is which is why im signed up... it's about why data re-loading needs to happen to begin with every time when switching time frames. i've seen and used other platforms (and i wont mention names) where switching from 60 mins to 5 mins on the same chart happens instantly with a years worth of data loaded. ie. if im on a 60 min chart and i select 5 mins, bars just appear without having to reload all the data behind it.

maybe no one else in your user base uses an intraday chart with 300 days of tick data and volume profiling and needs to switch time frames on a chart. hmm

anyway.. thanks for the reply.. as you said .. nothing's going to change so no point debating.
[2015-10-01 03:39:52]
Trader Ra - Posts: 87
like i mentioned.. im no programmer so i dont understand the jists of coding or how SC is engineered.. i wasn't looking for a solution as i knew you would say nothing can be done. i just wanted to know what the reason for this issue is. ie. its just a technical limitation of how SC is built and nothing can be done about it ? yes / no ?
Date Time Of Last Edit: 2015-10-01 03:42:30
[2015-10-01 03:50:38]
Sierra Chart Engineering - Posts: 104368
if im on a 60 min chart and i select 5 mins, bars just appear without having to reload all the data behind it.

Were these chart bars constructed from tick by tick data and were Volume Profiles used on the chart? If not, then this is why it is faster.

There could be some techniques that could be employed to make the data loading faster if it is known that you for example work with just one minute based time frames but it starts getting more involved and would increase memory use.

Otherwise, the bars have to be rebuilt again from the tick by tick data. There is no way around this. So this is the basic answer.
Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service:
Sierra Chart Teton Futures Order Routing
Date Time Of Last Edit: 2015-10-01 03:52:18
[2015-10-01 04:03:02]
ejtrader - Posts: 688
In case of Ninja - It uses minute data separate from tick data and would "appear" to be faster when 60 minute to 5 minute chart data is loaded.

In case of SC - if the underlying data is tick data ( 60 minute or 5 minute charts would have to be built from tick data ) it would have to build the chart from tick data.

One of those potential reasons why the other program might appear faster to load/switch minute data and might not be same when tick charts ( with same number of days to load as SC) would be faster when the switch to different chart period.

Thanks
[2015-10-01 04:12:33]
Trader Ra - Posts: 87
yes tick data was always used for most accuracy in all other examples as was volume profiling..

i will in fact remove all volume profile studies from my chart and see what difference it makes but again.. i need to use it so removing it defeats the purpose of my methodology. to get around this issue..

i have 1 chart with 300+ days of data loaded on a higher time frame and other charts with just 50 days loaded for smaller time frames.. i draw support and resistance levels and focus a lot on the bigger picture but need to look at the nuances of the smaller time frames to develop trade ideas. right now i have to use the copy chart drawings feature and eyeball multiple charts to get my prep work done. If switching time frames wasn't an issue then i would use just the one chart and just drill down on the time frames without having to wait 10-15s for all the data to load. It's not that long of a wait but no wait is better. i dont use anything less than 1 min. i only have one 1 min chart with 20 days data which loads as fast as i can blink. again its the bigger charts.

its not an issue during rth hours as i dont need to switch time frames then with all the screens i have.

how much more memory use are we talking about ? even if it uses an extra 8-10GB of memory i dont mind.. i guess not all users have good systems. wont be surprised if people are trying to trade on windows millenium still with 1gb of ram lol.

what if sc could be programmed to utilise more memory as a user choice to speed up data re-loading?

Otherwise, the bars have to be rebuilt again from the tick by tick data. There is no way around this
thanks.. that explains it.
[2015-10-01 04:18:52]
Trader Ra - Posts: 87
In case of Ninja - It uses minute data separate from tick data and would "appear" to be faster when 60 minute to 5 minute chart data is loaded.

In case of SC - if the underlying data is tick data ( 60 minute or 5 minute charts would have to be built from tick data ) it would have to build the chart from tick data.

One of those potential reasons why the other program might appear faster to load/switch minute data and might not be same when tick charts ( with same number of days to load as SC) would be faster when the switch to different chart period.

Thanks

brilliant explanation! that certainly solves the puzzle and makes sense. SC relies solely on only one type of data that is downloaded. if minute data can be stored separately to tick data, then SC wont need to reload the entire chart every time... just switch bars like ninja does as you mentioned. so it is a technical limitation in SC as i was alluding to.

hope sc can take your feedback on board positively.
[2015-10-01 04:39:27]
Sierra Chart Engineering - Posts: 104368
There is not anything we can do to help with this.

You cannot do Volume Profiles accurately with 1 minute data. It is not possible.

While the suggestion we made is possible, there is no chance of us getting involved with it at any time soon. So it is pointless to have any hope in that.
Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service:
Sierra Chart Teton Futures Order Routing
Date Time Of Last Edit: 2015-10-01 04:40:30
[2015-10-01 05:19:15]
Trader Ra - Posts: 87
You cannot do Volume Profiles accurately with 1 minute data. It is not possible.

agreed.. but what ejtrader mentioned was that ninja stores tick and minute data side by side. bars are built using minute data which load much faster whereas volume profile is loaded only once using tick data.. so every time a user switches time frame on a chart, only the chart bars are loaded from the minute data whereas volume profiles dont need to be reloaded all the time.. this is the difference.

This is as fast and efficient as can be, and it cannot be beat.
up for some debate now in light of the new information.

i hope other users can add to this user discussion now as everyone (sc support and users) i believe is after the same thing.. for SC to be the best all around, no debate.
[2015-10-01 05:55:37]
Sierra Chart Engineering - Posts: 104368
There is no chance of us doing anything in this area. We need to be honest about it, this discussion is pointless and will not influence us. It simply cannot. Sierra Chart is designed and structured a certain way and these kinds of changes proposed have no chance whatsoever of being done.

whereas volume profile is loaded only once using tick data..
With how Sierra Chart is designed, this does not make any sense.
Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service:
Sierra Chart Teton Futures Order Routing
Date Time Of Last Edit: 2015-10-01 06:15:50
[2015-10-01 05:59:45]
Sierra Chart Engineering - Posts: 104368
Prior post has been updated.
Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service:
Sierra Chart Teton Futures Order Routing
[2015-10-01 06:02:45]
ejtrader - Posts: 688
Raaa - I was not making any suggestion on what is right/wrong in this space. Was merely providing additional information on why it "might appear" faster to change the Time/minute based charts in ninja platform ( as the chart is built using minute bars ).

If you take an identical Tick chart ( or any other non-time based chart ) - SC can load the chart very efficiently.

There are few alternate ways where you can have one install using minute data and one install using tick based data and use them in combination to achieve what you want ( you might have go through the documentation on how to get this done) but certainly this is possible.

Thanks
Date Time Of Last Edit: 2015-10-01 06:03:33
[2015-10-01 06:20:41]
Sierra Chart Engineering - Posts: 104368

And we are certain, that no one is going to beat Sierra Chart with an identical test in this area.
We still mean what we say. If another program is doing things differently, then it is not an identical test. We do not know how ninja does this.

Sierra Chart Volume Profiles are not built from tick by tick data. They are built from the chart bars themselves and the chart bars are built from tick by tick data and internally contain the volume at price data.

This is why when you modify Volume Profiles, or draw them by hand, they immediately appear. They are not built from the tick data at that time.

If you have a one-year chart built from tick by tick data, with a lot of chart bars, it is not going to immediately reload when changing any of the Chart Settings which affect the bars. They do have to be fully rebuilt from the underlying data file in order to support all of the studies in the chart. Although the underlying data in the data file most likely is cached by the operating system so this can happen relatively fast.

One thing that we will do, is we will look at the recent changes we made with building chart bars to see if there are some changes we can make, involving function inlining, which might improve performance slightly.
Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service:
Sierra Chart Teton Futures Order Routing
[2015-10-01 06:31:36]
Trader Ra - Posts: 87
Sierra Chart is designed and structured a certain way
this is exactly what i wanted to know from my initial post.

Raaa - I was not making any suggestion on what is right/wrong in this space.
I never said you did either.

If you take an identical Tick chart ( or any other non-time based chart ) - SC can load the chart very efficiently.
again.. this thread was not about whether SC is efficient or not. i think we all agree that it is efficient and more stable than most other platforms. it was a question about design and how the software is engineered that may answer the problem i pointed out. just wanted a better understanding of how the software works.

There are few alternate ways where you can have one install using minute data and one install using tick based data and use them in combination to achieve what you want ( you might have go through the documentation on how to get this done) but certainly this is possible.
great suggestion! i will look into this.


again.. i wasn't after a solution to this because i already knew sc support's position on this area and i wasn't trying to influence a change. just after a better understanding and perhaps that may lead to some insights for possible changes or improvements.

but the outcome i was after for posting this thread has been achieved
- i know now that its a design limitation
- i know sc team will not be making any changes to this area
- i have a better understanding of how it all works and im sure others reading this will too as well.
- i have received a good suggestion about installing 2 copies (something i will definitely look into)

:)
[2015-10-01 06:34:48]
Trader Ra - Posts: 87
One thing that we will do, is we will look at the recent changes we made with building chart bars to see if there are some changes we can make, involving function inlining, which might improve performance slightly.

there we go.. thanks for being objective and the explanation is satisfactory too.
[2015-10-01 06:35:19]
i960 - Posts: 360
im talking about using it after all the data is initially loaded and i have to switch time frames on a chart with a year's worth of intraday tick data loaded, that's where the issue is. now you would say well duh.. i have too much data loaded.. end of puzzle.

The problem here OP, is your approach, you're side-stepping a very easy technical change for doing things in an inherently slow fashion. It's not reasonable to expect SC to load 100,000s of data points dynamically, redraw all bars, etc. for multiple days/weeks worth of data and expect there to not be a significant delay. This isn't even SC specific either - most chart platforms will run into similar time vs space basic limitations of computing and this is a typical one.

Here's what you need to do:

1. Create a chartbook for each of the instruments you care about.
2. Create a separate chart for each timeframe for that instrument you care about - and you put it within the same chartbook.
3. Assign [ and ] as hotkeys for prev chart or next chart.
4. Assign shift-[ and shift-] as hotkeys for prev chartbook or next chartbook.
5. Additionally, you can save your study collections from one chart and apply them to others so as to speed up the chart cloning process.

Then you just use it. There is zero delay switching between charts and chartbooks. The work has been done ahead of time by nature of having the charts already configured and being charted on an ongoing basis. The only delay is on reopening chartbooks which is usually a per-session or per-trading day type thing.
Date Time Of Last Edit: 2015-10-01 06:39:28
[2015-10-01 06:35:56]
Sierra Chart Engineering - Posts: 104368
You also have the ability to compress data previous to a certain date into a higher timeframe to increase the speed of data loading.
Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. Try to keep your questions brief and to the point. Be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

For the most reliable, advanced, and zero cost futures order routing, *change* to the Teton service:
Sierra Chart Teton Futures Order Routing
[2015-10-01 06:45:14]
Trader Ra - Posts: 87
The problem here OP, is your approach, you're side-stepping a very easy technical change for doing things in an inherently slow fashion. It's not reasonable to expect SC to load 100,000s of data points dynamically, redraw all bars, etc. for multiple days/weeks worth of data and expect there to not be a significant delay. This isn't even SC specific either - most chart platforms will run into similar time vs space basic limitations of computing and this is a typical one.
the problem with your response is that you are not viewing the issue from the perspective of a non-programmer. hence why i stated the fact that i am not a programmer. try putting on a different pair of lens.


Here's what you need to do:

1. Create a chartbook for each of the instruments you care about.
2. Create a separate chart for each timeframe for that instrument you care about - and you put it within the same chartbook.
3. Assign [ and ] as hotkeys for prev chart or next chart.
4. Assign alt-[ and alt-] as hotkeys for prev chartbook or next chartbook.
5. Additionally, you can save your study collections from one chart and apply them to others so as to speed up the chart cloning process.
you obviously did not read my post clearly. that's exactly how i have set things up so nothing of value there. you dont know my trading methodology so dont assume what works for you will work for me. i prefer drilling down on 1 chart for reasons i mentioned above.

i dont want to get into nitty gritty details here.

SC did a good job already explaining what i wanted to know.
[2015-10-01 06:47:45]
Trader Ra - Posts: 87
You also have the ability to compress data previous to a certain date into a higher time frame to increase the speed of data loading.


i actually did try doing that once and something went wrong afterwards. cant recall what happened but i had to re download all the data again.

will try that again and advise what happened thanks.
[2015-10-01 09:00:29]
i960 - Posts: 360

you obviously did not read my post clearly. that's exactly how i have set things up so nothing of value there. you dont know my trading methodology so dont assume what works for you will work for me. i prefer drilling down on 1 chart for reasons i mentioned above.

i dont want to get into nitty gritty details here.

SC did a good job already explaining what i wanted to know.

I use this exact same setup on a 15" laptop, watching probably 8+ futures in various chartbooks. If I want to "drill down", I use the up-arrow to zoom in. If a given chart doesn't have the level of detail I need, then I make a new chart for it such that it does. But I usually have the timeframes I'm interested in done ahead of time so this isn't typically a problem.

I read your original post. You clearly stated you wanted to "switch timeframes" and it was slow. Simply creating another chart with the alternate timeframe ahead of time solves the problem.
[2015-10-01 13:59:26]
Trader Ra - Posts: 87
I use this exact same setup on a 15" laptop, watching probably 8+ futures in various chartbooks. If I want to "drill down", I use the up-arrow to zoom in. If a given chart doesn't have the level of detail I need, then I make a new chart for it such that it does. But I usually have the timeframes I'm interested in done ahead of time so this isn't typically a problem.

I read your original post. You clearly stated you wanted to "switch timeframes" and it was slow. Simply creating another chart with the alternate timeframe ahead of time solves the problem.

hmm zoom and creating multiple charts.. genius! now that's sth i never thought of lol.. go back and read all the posts again until you have that 'aha' moment. im not going to waste time here. you are still clueless and dont understand the core issue... stick to your method if it works for you and again.. dont assume things. as far as im concerned as the OP here.. im satisfied with the outcomes and the matter is closed. SC is also looking into it.

good day.
Date Time Of Last Edit: 2015-10-01 14:25:30
[2015-10-01 21:38:38]
i960 - Posts: 360
What are you not getting here dude? You clearly said this:

im talking about using it after all the data is initially loaded and i have to switch time frames on a chart with a year's worth of intraday tick data loaded

You: "Doctor, it hurts when I switch timeframes on my year long intraday chart."
Doctor: "Then stop switching timeframes."

This isn't rocket science what you're doing - and everyone here is doing extremely similar stuff without a problem. You clearly have the ability to setup another chart with the alternate timeframe you typically go to - yet you refuse to consider that as a solution but instead blame the tool for not doing it the way *you* desire it to work.

60m "big picture" chart #1
5m "detail" chart #2

Chart #2 copies drawings from #1 automatically via http://www.sierrachart.com/index.php?page=doc/doc_ChartSettings.html#CopyChartDrawingsfromChart which you say you're already using anyway.

So how is there an actual workflow problem here other than "but I just wanna do it my way and not have to switch charts"? Adapt, it's what traders do.
[2015-10-02 00:43:56]
Trader Ra - Posts: 87
Looks like this doctor's head is in the wrong place and needs to go back to school. Like i said before, I have achieved the outcomes I was after for posting this thread and all you are doing is creating noise here.

Adaptation is only one part mate.. that attitude will always keep you playing small like a 2 lot trader.. seeking continuous improvement and learning by questioning the accepted norm is another. Clearly you think you know it all and that's blinding you. As i further noted, you are adding no value here by suggesting things we all know already.. think of it! and im done wasting time on this thread so after this post, i am unsubscribing from this one ie. don't expect me to be revisiting here.

Maybe book an appointment with the ego doctor. kapish!

and to SC Support .. apologies if my intentions were misconstrued.. possibly by the title i chose.. sounds bit harsh.. (just renamed it) but as i said before.. i love SC and its capabilities. continue with the great work! :)
Date Time Of Last Edit: 2015-10-02 02:06:57

To post a message in this thread, you need to log in with your Sierra Chart account:

Login

Login Page - Create Account