Login Page - Create Account

Support Board


Date/Time: Tue, 24 Apr 2018 22:05:30 +0000



orders disappear/not transmitted to IB

Support Request:
[2015-11-18 17:14:11]
freestyler - Posts: 32
Hi:

The following situation has occurred several times over the last month. And it's very distressing when I'm in the middle of trading.

It goes like this: I put on a trade in SC trade DOM (with accompanying OCO orders). The entry price gets hit and the trade is entered (connected to IB). So far so good. However, soon after, my position mysteriously disappears in trade DOM, or one of the OCO orders disappears. I don't usually notice this until after the fact. Soon after, when one of the OCO orders is hit, or so I thought, nothing is actually done, since SC thinks I have no position on. Nothing gets transmitted to IB. I then have to scramble and manually close the trade directly on IB platform, wasting valuable time and $$$.

This morning I entered a short, soon after entry, the order apparently disappeared and when I manually exited by pressing "Flatten" in trade DOM, nothing happened.

Seems like there's some sort of data synchronization problem with IB that occurs, not all the time, but certainly enough to become unreliable. Worst part is, there's absolutely no warning given by SC, so it's so easy to miss the error when I'm in the middle of trading until it's too late.

Obviously this is a BIG problem.

Please investigate and get back to me ASAP.
[2015-11-18 17:43:24]
Sierra Chart Engineering - Posts: 59398
These kinds of issues are not uncommon with Interactive Brokers:
https://www.sierrachart.com/index.php?page=doc/IB.php#TradingRelatedIssues

Provide us the exact symbol of the chart or Trade DOM. Also does your Interactive Brokers account have more than one account on it? Is it a Financial Advisor account?

We should be able to solve this for you.

Also, there will not be any warning from Sierra Chart. Sierra Chart is not aware there was a problem to begin with. It is simply reporting the information provided from Interactive Brokers Trader Workstation.
Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. If possible please keep your questions brief and to the point. Please be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

If your question/request has been answered and you do not have anything further, then it is easiest for us if you do not reply again to say thank you.
Date Time Of Last Edit: 2015-11-18 17:43:42
[2015-11-18 17:51:22]
freestyler - Posts: 32
I was trading crude oil futures, Jan./2016 contract. Symbol is CL-201601-NYMEX

I only have one account with IB (not counting a demo account). It is not a financial advisor account.

Let me know if you need more info.

Thanks.
[2015-11-18 17:57:35]
freestyler - Posts: 32
Something else just occurred to me that may or may not have played a factor.

I rolled over to a new contract month earlier this morning, prior to placing the order that had data synchro problems. The roll over was done by using the "Perform Futures Symbol Rollover" function in SC.

Hope this helps.
[2015-11-18 17:58:28]
Sierra Chart Engineering - Posts: 59398
However, soon after, my position mysteriously disappears in trade DOM, or one of the OCO orders disappears.
At the time this happens check "Trade >> Trade Orders and Positions >> Positions". Do you see the Position Quantity there? If so what is the symbol?

Soon after, when one of the OCO orders is hit, or so I thought, nothing is actually done, since SC thinks I have no position on. Nothing gets transmitted to IB.
As soon as the main/parent order fills, or when the main order is sent, at that time the Attached Orders are sent. Not when one of the OCO orders is touched. So what you are describing does not really make sense. The Attached Orders should already be working in TWS unless the orders were ignored for some reason.


If you want reliability we recommend avoiding Interactive Brokers. Interactive Brokers does not provide a well-designed interface for external programs to use.

One of the best trading platform services is CTS T4:
https://www.sierrachart.com/index.php?page=doc/CTS_T4.php

Although we recognize the problem could be that because you are from Canada that maybe Interactive Brokers is the only choice.
Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. If possible please keep your questions brief and to the point. Please be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

If your question/request has been answered and you do not have anything further, then it is easiest for us if you do not reply again to say thank you.
Date Time Of Last Edit: 2015-11-18 17:59:11
[2015-11-18 18:00:33]
Sierra Chart Engineering - Posts: 59398
You can also see exactly what is happening with the Orders and Trade Positions from the perspective of Sierra Chart in the Trade Activity Log. Here is the documentation for that:
https://www.sierrachart.com/index.php?page=doc/doc_TradeActivityLog.php#ViewingHistoricalTA
Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. If possible please keep your questions brief and to the point. Please be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

If your question/request has been answered and you do not have anything further, then it is easiest for us if you do not reply again to say thank you.
[2015-11-18 18:04:44]
freestyler - Posts: 32
I just uploaded the trade activity text file from this morning. Trade was entered at 11:04 am, and soon after I believe the data synchro error occurred.
Private File
[2015-11-18 18:33:33]
Sierra Chart Engineering - Posts: 59398
We will look this over.

However, there is something else we need from you. Any time after the close today or when you can, restart Sierra Chart and connect to Trader Workstation from Sierra Chart.

Do not open any charts. And then post a copy of the Message Log by following these instructions here:
https://www.sierrachart.com/index.php?page=PostingInformation.php#MessageLog

We want to make sure there are no problems with Trader Workstation responding to the request to subscribe to Trade Position data.
Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. If possible please keep your questions brief and to the point. Please be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

If your question/request has been answered and you do not have anything further, then it is easiest for us if you do not reply again to say thank you.
[2015-11-18 18:42:50]
freestyler - Posts: 32
Here it is:

Software version: 1316 | 2015-11-18 13:39:24
Uses New Spreadsheets | 2015-11-18 13:39:24
Usage end date: 2015-11-23 | 2015-11-18 13:39:24
Enabled for: Advanced Features. | 2015-11-18 13:39:24
Enabled for: Sierra Chart Historical Data Service. | 2015-11-18 13:39:24
Allow Support for Sierra Chart Data Feeds is enabled. | 2015-11-18 13:39:24
Current selected Data/Trading service: Interactive Brokers | 2015-11-18 13:39:24
Chart Update Interval: 500 | 2015-11-18 13:39:24
Time Zone: -05:00:00 (EST-05EDT+01,M3.2.0/02:00,M11.1.0/02:00) | 2015-11-18 13:39:24
2015-11-18 13:39:24 Local computer time | 2015-11-18 13:39:24
2015-11-18 18:39:24 Local computer time in UTC | 2015-11-18 13:39:24
2015-11-18 13:39:24 Local computer time in SC Time Zone | 2015-11-18 13:39:24
2015-11-18 18:39:24 Server time in UTC | 2015-11-18 13:39:24
Local computer UTC time and Server UTC time difference: 2 seconds. | 2015-11-18 13:39:24
Program path: C:\SierraChart\ | 2015-11-18 13:39:24
Data Files path: C:\SierraChart\Data\ | 2015-11-18 13:39:24
OS Version Number: 6.2 | 2015-11-18 13:39:24
Locale Information: C | 2015-11-18 13:39:24

Checking for new symbol settings for ib | 2015-11-18 13:39:24
HTTP connection to www.sierrachart.com:80 (0) | Resolved address sierrachart.com to IP 5.9.8.236 | 2015-11-18 13:39:24
HTTP connection to www.sierrachart.com:80 (3) | Connected. | 2015-11-18 13:39:25
Symbol settings are up-to-date for ib | 2015-11-18 13:39:25

Interactive Brokers: Connecting to 127.0.0.1:7496 | 2015-11-18 13:39:29
Socket (4) | Connected. | 2015-11-18 13:39:29
Socket connected to TWS. Sending client version: 62. Waiting for acceptance. | 2015-11-18 13:39:29
Interactive Brokers | TWS server version: 76 | 2015-11-18 13:39:29
Interactive Brokers | Connected. Waiting for account information. | 2015-11-18 13:39:29
IB Account information received. | 2015-11-18 13:39:29
Interactive Brokers | Received 1 Trade Accounts from TWS. | 2015-11-18 13:39:29
Interactive Brokers | Connected to Live account. | 2015-11-18 13:39:29
Interactive Brokers | Year-Month last connected to Live account: 2015-11 | 2015-11-18 13:39:29
Interactive Brokers | Requesting Trade Positions (IB account data updates) for account: U456* | 2015-11-18 13:39:29
Interactive Brokers | Connected to data and trading server. | 2015-11-18 13:39:29
Next valid order ID: 2010 | 2015-11-18 13:39:29
Message from IB: Market data farm connection is OK:usfuture. IB Error Code: 2104. Request ID: -1. | 2015-11-18 13:39:29
Message from IB: Market data farm connection is OK:usfarm. IB Error Code: 2104. Request ID: -1. | 2015-11-18 13:39:29
Message from IB: HMDS data farm connection is OK:ilhmds. IB Error Code: 2106. Request ID: -1. | 2015-11-18 13:39:29
Message from IB: HMDS data farm connection is OK:ushmds. IB Error Code: 2106. Request ID: -1. | 2015-11-18 13:39:29
Next valid order ID: 2010 | 2015-11-18 13:39:29
IB Account information received. | 2015-11-18 13:39:29
Interactive Brokers | Received 1 Trade Accounts from TWS. | 2015-11-18 13:39:29
Interactive Brokers | Connected to Live account. | 2015-11-18 13:39:29
Interactive Brokers | Year-Month last connected to Live account: 2015-11 | 2015-11-18 13:39:29
Interactive Brokers | Received Trade Positions download end message for account U456* | 2015-11-18 13:39:29
HTTP connection to www.sierrachart.com:80 (3) | Closed. No error. | 2015-11-18 13:39:30
HTTP connection to www.sierrachart.com:80 (3) | Shutdown and closed. | 2015-11-18 13:39:30
Interactive Brokers | Requesting historical order fills for account U45*. | 2015-11-18 13:39:34
Interactive Brokers | Received Open orders end message. | 2015-11-18 13:39:34
[2015-11-19 05:36:26]
Sierra Chart Engineering - Posts: 59398
Yes, there is a big problem:

Obviously this is a BIG problem.

Have a look at these:
Order  2015-11-18 11:04:29.000  CL-200601-NYMEX  Synchronized Internal Position to current Service Position. Current Position quantity: -1                Unspecified      U4569756

Order  2015-11-18 11:12:20.000  CL-200601-NYMEX  IB open order update  9970  -8515  Stop  1  Buy  41.84    Open      None        
Order  2015-11-18 11:12:21.000  CL-200601-NYMEX  IB order fill (execution)  9970  -8515  Stop  1  Buy  41.84    Filled  41.84  1  U456*      -1  
Order  2015-11-18 11:12:29.000  CL-200601-NYMEX  Removing Position with quantity of 0                Unspecified      U456*        
Order  2015-11-18 11:15:30.000  CL-200601-NYMEX  Removing Position with quantity of 0                Unspecified      U456*              
  

Notice the dates of those are 200601. This is something you need to explain or Interactive Brokers needs to explain.
Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. If possible please keep your questions brief and to the point. Please be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

If your question/request has been answered and you do not have anything further, then it is easiest for us if you do not reply again to say thank you.
Date Time Of Last Edit: 2015-11-20 00:09:54
[2015-11-19 14:12:05]
freestyler - Posts: 32
Thanks for the info. I contacted IB regarding this date change problem. They asked for the API log file which unfortunately I did not have set to "detail", so there was insufficient info for them to investigate. Looks like I'll have to wait for next occurrence of this problem before it can be traced to its root.

On the other hand, I do think it's more than a coincidence that this happened on the same day that I did a contract roll over. Perhaps next time I should close SC and IB after roll over and start them up again? Do you think this would help?
[2015-11-19 15:32:43]
Sierra Chart Engineering - Posts: 59398
This is not an issue within Sierra Chart. You do not have to restart it.

Probably would be a good idea to restart Trader Workstation though.
Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. If possible please keep your questions brief and to the point. Please be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

If your question/request has been answered and you do not have anything further, then it is easiest for us if you do not reply again to say thank you.
[2015-11-19 16:24:28]
Sierra Chart Engineering - Posts: 59398
Also when providing the Trade Activity Log always follow the instructions here:
https://www.sierrachart.com/index.php?page=doc/doc_TradeActivityLog.php#TradeActivityLogToSupport

The log you gave did not include any order fills which are essential to see. Although we were able to determine the problem anyway.
Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. If possible please keep your questions brief and to the point. Please be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

If your question/request has been answered and you do not have anything further, then it is easiest for us if you do not reply again to say thank you.
Date Time Of Last Edit: 2015-11-19 16:24:55
[2015-11-19 18:25:17]
freestyler - Posts: 32
Okay, I spoke to IB tech support for a good hour this morning, after the same "wrong contract year" problem happened twice in a roll within 10 minutes (activity log attached).

I emailed them the TWS log. Steven over at IB tech support(very nice guy actually) assured me that the 2006 number isn't coming from his end. He wasn't sure why SC was looking for position confirmation right after trade had been opened and position already confirmed. He did mention that the contract month format used by IB is sometimes different from others (they use end of the month instead of beginning of new month). But that still doesn't explain where 2006 is coming from.

I'm no techie. All I know is that computers do not come up with 2006 on its own, it's gotta be a line programmed somewhere that's causing it. Seems like it could just be a silly typo someone made, putting in 2006 instead of 2016, perhaps in the definition of the contract month somewhere. Question is, from whom?

Steven invited someone over at your side to contact him at 877-442-2757, option 5, then option 4. So log files can be compared from both sides and get to the bottom of this. Just ask for him since he's already familiar with this issue.

I would really appreciate it if you can do this for me, as a client who's been with you for quite awhile now.

At the end of the day, as a trader who's paying both companies, I just want to get this issue resolved (without resorting to switching brokers and/or charting software). C'mon, we're talking about an error of 10. That's it! There's a gotta a solution to this.

Thanks in advance.
Private File
Attachment Deleted.
[2015-11-19 18:57:06]
Sierra Chart Engineering - Posts: 59398
We now recognize the reason for the problem.

We are working on a solution to this.

The problem is that the expiration year and month that Interactive Brokers specifies for a particular commodity contract is in the prior month. So therefore to determine the true delivery Year and Month, we use the "Local Symbol" and that has a single digit year.

So in the case of 16 it is specified as 6 and that it was mistranslated within Sierra Chart to 2006 because that particular block of code which was written many years ago did not consider the current year. This was a patch we put in many years ago when Interactive Brokers changed how they transmit the expiration for a futures contract, that is now creating a problem. Normally you would expect the expiration to be 201601 for CL at this time. Instead it is 20151221 which does not directly correspond to CL-201601-NYMEX.

We can fix this, and we will today, but still there is not an ideal solid solution to this kind of issue. The real root of the just has to do with complex and inconsistent symbology from Interactive Brokers.

It really is a mistake for Interactive Brokers to be doing what they are doing. If they want to provide the true expiration date for a speculator, that should be a separate field in a security definition. It should not be transmitted as the year and month identifier for a futures contract for positions and fills especially when subscribing to market data and trading a symbol they accept 201601. This just does not make sense.

Furthermore, when you look at the contract specifications and quotations from the CME for these markets, this is specified as a January 2016 Crude oil contract. That is how it is identified, using January 2016. Not as December 2015.
Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. If possible please keep your questions brief and to the point. Please be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

If your question/request has been answered and you do not have anything further, then it is easiest for us if you do not reply again to say thank you.
Date Time Of Last Edit: 2015-11-19 19:00:58
[2015-11-19 21:14:08]
Sierra Chart Engineering - Posts: 59398
When we say Local Symbol what we mean is a symbol formatted like this: CLF6. This is what Interactive Brokers transmits for the local symbol.


We have determined the most reliable solution we can. We will use the year from the expiration date and the month from this local symbol and combine them to arrive at the correct year and month.
Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. If possible please keep your questions brief and to the point. Please be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

If your question/request has been answered and you do not have anything further, then it is easiest for us if you do not reply again to say thank you.
[2015-11-20 01:25:04]
freestyler - Posts: 32
Please let me know when this is fixed and how I can upload it into my software (I assume by downloading the latest version?).

Thanks.
[2015-11-20 02:48:24]
Sierra Chart Engineering - Posts: 59398
We are just about ready with release. We do have it solved.

And we did hear from Interactive Brokers, and their comments are noted, and we do acknowledge that the patch we put in many years ago, did have a problem crossing from 2015 to 2016.

This is what we told IB:
Thank you for the information. This is understood.

However, what we would suggest is that because there are three different ways of specifying a market when requesting market data and placing an order, according to what you have given below, for the same futures contract, that you provide a complete contract specification in the UpdatePortfolio message.

You acknowledge (see quotation below), that you are not relaying the Contract Year and Month. Only last trade date. But yet an API client is capable of subscribing using the Contract Year and Month. This is the inconsistency.

The problem is we have to do special processing to take the Last Trade Date and Local Symbol and extract out the correct Year and Month to match it up with the contract specification used when placing the order.

Furthermore, we have even seen for Local Symbol this format: "ZN MAR 11". This is based upon our notes. Although we could be mistaken about this since it was from a while ago. Perhaps it was for another trading service.

From Interactive Brokers:
-----
We do not relay year and month identifiers for positions, see below for a snippet of a log showing the updatePortfolio():

TX 16:04:36:776 JTS-EServerSocket-2117: [0:101:101:1:0:7:8:INFO] Sending portfolio entry.
TX 16:04:36:776 JTS-EServerSocket-2117: [0:101:101:1:0:7:8:DET] [7;8;CL;FUT;20151221;0;0;USD;CLF6;1;41.74500275;41745.0;41752.31;-7.31;0.0]
TX 16:04:36:776 JTS-EServerSocket-2117: [0:101:101:1:0:7:8:INFO] Portfolio entry sent.

As you can see we relay the Last Trade Date. We also have a separate parameters for contractMonth and lastTradeDate (see contract Details example above).
-----

Additional comments:

It is quite clear, that the contract specifications provided in a Portfolio/Position Update message, are incomplete. What is the point of providing the last trading date but not providing the other contract details like the Contract Year and Month.

Furthermore, there was a point many years ago, where Interactive Brokers changed the expiration date to the last trade date creating this problem. These are the kind of changes which lead to instability for existing users.

So in conclusion, it is not even possible to subscribe to market data and to place an order using CL-201601-NYMEX-USD-1000 which is a very detailed contract specification and can be auto generated, and then simply and reliably match up the Interactive Brokers Portfolio/Position update with it.

We have accomplished this, but it is not a simple solution which we can always rely on to work in all cases.
Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. If possible please keep your questions brief and to the point. Please be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

If your question/request has been answered and you do not have anything further, then it is easiest for us if you do not reply again to say thank you.
Date Time Of Last Edit: 2015-11-20 03:25:40
[2015-11-20 03:44:09]
Sierra Chart Engineering - Posts: 59398
To be clear, this is the particular problem identified and resolved:
This problem would not cause any other issue other than the display of the Trade Position data on the Trade Orders and Positions >> Positions tab and in charts or Trade Windows. Attached Orders were/are not affected in any way. If you notice any other problems, they are unrelated, and they might have been misinterpreted.

What you describe here, is unrelated to what we identified and solved:
or one of the OCO orders disappears. I don't usually notice this until after the fact. Soon after, when one of the OCO orders is hit, or so I thought, nothing is actually done, since SC thinks I have no position on. Nothing gets transmitted to IB. I then have to scramble and manually close the trade directly on IB platform, wasting valuable time and $$$.

In the Trade Activity log you provided you were using server side bracket orders. So all three orders, the parent order and the Attached Orders are sent all at once the Interactive Brokers and Interactive Brokers is responsible for them at that point in time.

Furthermore, this only affected 2016 futures contracts. If you were trading a 2015 contract and had problems, that is from some other reason. We have seen various issues reported with Interactive Brokers in the last month or two. None of those issues have been identified as being a problem with Sierra Chart but with TWS instead.
Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. If possible please keep your questions brief and to the point. Please be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

If your question/request has been answered and you do not have anything further, then it is easiest for us if you do not reply again to say thank you.
Date Time Of Last Edit: 2015-11-20 03:45:35
[2015-11-20 13:48:16]
freestyler - Posts: 32
Just to confirm, does version 1329 (dated 11/20/2015) contain the fixes for the "2006" year problem? I just installed it and wanted to be sure.

Thanks.
[2015-11-20 15:55:16]
Sierra Chart Engineering - Posts: 59398
Yes it does.
Sierra Chart Support - Engineering Level

Your definitive source for support. Other responses are from users. If possible please keep your questions brief and to the point. Please be aware of support policy:
https://www.sierrachart.com/index.php?l=PostingInformation.php#GeneralInformation

If your question/request has been answered and you do not have anything further, then it is easiest for us if you do not reply again to say thank you.

To post a message in this thread, you need to login with your Sierra Chart account:

Login


Login Page - Create Account