IBGateway Dropped Socket Issues with EU and HK

[emretezel]

Hi Everyone,

I am running IB Gateway 10.21 and ib-insync 0.9.70.

Very frequently at some point in the day IB Gateway Historical Data Farm connections to EU and HK drop off, causing failure to update Eurex and OSK traded futures prices, even if I pull historical prices once every hour. Does anyone else experience the same issue? Do you think this is a local internet/hardware issue? The only remedy then is to restart IB Gateway and run_daily_price_updates process.

Separately, I would appreciate if you guys would share whether you are running daily price updates only once a day (different regions different times) or in a loop from morning till evening.

Thank you,
Emre

[tgibson11]

I have not had this issue, nor heard of anyone else having it.

I would suspect most likely a local networking issue, either HW or SW. Could also be IB or Gateway. Less likely: ib-insyc. Almost certainly nothing to do with pysystemtrade. You might check IB or ib-insync forums to see if anyone else has had a similar problem.

I only run the price updates once a day, all instruments at the same time. I have mainly US instruments right now, but I'm slowly adding others. It's possible that I might get to a point where I want to separate them by region.

I've never heard of anyone updating prices continuously throughout the day with pysystemtrade. Seems like that could easily be done, but unless you're also running backtests throughout the day, I don't see much benefit to it.

[oldlore]

ib-insync 0.9.70
IB gateway 10.20.1f

I'm not having any connection issues. I run price updates just once a day, all in one batch. I subscribe to US, Eurex, JPX, KSE, SGX and it takes about 45 minutes to complete. These are my timings.

process_configuration_start_time:
  default: '00:01'
  run_stack_handler: '00:02'
  run_capital_update: '00:01'
  run_daily_prices_updates: '18:05'
  run_daily_fx_and_contract_updates: '07:00'
  run_daily_update_multiple_adjusted_prices: '19:30'
  run_systems: '20:30'
  run_strategy_order_generator: '20:40'
  run_backups: '21:25'
  run_cleaners: '21:20'
  run_reports: '21:00'

I'd be interested if any others in the EST/CST time zones can share what time they run things. I run prices after 6PM EST. Then I give myself a little bit of time to check price spikes, then run systems/strategy, then give myself time to check if there are any instrument orders and roll (if needed) before the stack handler starts at midnight, towards the end of the Asia market trading times.
I've thought about moving things up a little and trying to start stack handler at 8PM EST, which would catch the start of the Asia markets. But that would give me very little time after I get home from work to check anything before trades might happen.

[tgibson11]

I'm in Arizona, so MST or PDT depending on the time of year. I do things a bit differently.

My price updates start shortly after midnight, immediately followed by run_systems, order generation, and reports.

I wake up between 5 and 6am (for other reasons besides just trading), check reports, price spikes, rolls, deal with any issues.

Stack handler kicks off at 6:15am, but no trades are executed until 7am, 8am, or 8:30am local time, due to trading hours.

[oldlore]

Out of curiosity, what are you doing for the holiday tomorrow? Most, but not all, US and Europe futures markets are closed, Asia markets are open. Looks like Europe is closed Monday. Do you run normally and any instruments orders that can't be placed tomorrow just get erased at the end of the day or do you not run at all tomorrow?

[tgibson11]

I'm taking tomorrow off, not running anything.

Missing a day or even several days of trading is unlikely to have any significant effect on returns.

[emretezel]

I believe I found the problem after speaking with IB support.

As many of you know, IB servers in NY, Europe and HK have daily reset times.

https://www.interactivebrokers.com/en/software/systemStatus.php

It turns out despite being based in UK, I am being hosted on US servers.

US servers have daily reset times between 23:45 and 00:45 Eastern Time (NY time), which is right about the time I am loosing connection to HK historical market data server. Somehow the NY server restart is affecting my connection to HK server.

Updating historical prices for Japanese futures before 4:45 AM UK time (23:45 ET) so far seemed to have fixed the issue.