Colombao
Hi Gianluca, sorry to trouble you again :Smile_Ab:
After the login and the subscriptions, after about 20-40 minutes I correctly receive push updates from IG, I don't receive any push update anymore, like if the connection is lost.
This behavior is sistematic and reproducible with others PC or internet connections.
The workaround I adopted is to setup a timer that if he see e.g. 20 seconds of silence, re-login and make subscriptions again but...this happen each 20 minutes, it is too frequent...
Do you have any other solution / advice ? Could it be caused by the program that doesn't elaborate push updates enough quick?
Thanks again!
Colombao
socalwill
Hello Gianluca - this is excellent! With your help I got both the plain curl commands working as well as the python script, which I will use for my purposes down the line. Thanks so much! I will continue with the implementation with other parts, that are more broker-specific, and I will post a working code soon. Thanks again,
William
Gianluca Finocchiaro
Hi Dario,
this is a known problem but thank you for reporting it. I'm going to fix the demo and I'll let you know once done.
Thanks and Regards,
Gianluca
Gianluca Finocchiaro
Hi Dario,
I've just released on Github a new version of the
Basic StockList-Demo with support for bind operation, which allows a "fluent" streaming with no need of workarounds.
If you are interested at the protocol details,
here you can find an exhaustive explanation [Par. 4.2].
Let me know.
Regards,
Gianluca
socalwill
This is great, thanks, Gianluca. I notice that reproducible interruption as well and I just started simple workaround with a ten minute duration for the streaming and a ten min periodic restart via cronjob, but your fix will be much better. (This may be intentional, but I noticed you kept the debug mode in line 397.)
Gianluca Finocchiaro
Hi William,
I confirm that such workarounds go against the "session binding" as designed inside the Lightstreamer Server, so it is better to avoid them and to use the right features the Lightstreamer Network Protocol offers to leverage the power of the server.
Keeping the debug mode was a misprint, now I've fixed it and the source code is available on
Github.
Thanks and Regards,
Gianluca
Colombao
Gianluca, with your last update the number of interruptions decreased dramatically. Gianluca the best!
Gianluca Finocchiaro
Hi Dario,
thanks to your precious feedback I was finally able to fix last minor bugs. Now the implementation seems stable even when connected to several Lightstreamer instances deployed in a cluster.
Thank You!
Gianluca
epep
Hi ! And thanks for your work !
I want my LightStreamer client to try to reconnect whatever the situation, it seems to work for now (not testing since a long time in fact), but as I don't test against my own server I have to wait for errors to happen, and being new to LightStreamer and Python I suppose I'm doing some not clever things. Can you please give me your appreciation about my code ?
I modified your python client this way :
Added these two methods to LSClient class :Smile_Ah: SYNTAX=PYTHON] def reconnect(self):
fail = 0
while True :
try :
log.info("trying to reconnect")
self._stream_connection.close()
self._session.clear()
self.connect()
self.resubscribe()
break
except :
log.info("fail :", sys.exc_info()[0])
fail += 1
wait = min( 300, 0.125 * ( 2 ** fail ))
log.info("next try in {0} seconds", wait)
time.sleep(wait)
def resubscribe(self):
""""resubscribe old items after starting a new connection"""
for subscription_key in self._subscriptions.keys() :
# Send the control request to perform the subscription
subscription = self._subscriptions[subscription_key]
server_response = self._control({
"LS_Table": subscription_key,
"LS_op": OP_ADD,
"LS_data_adapter": subscription.adapter,
"LS_mode": subscription.mode,
"LS_schema": " ".join(subscription.field_names),
"LS_id": " ".join(subscription.item_names),
})
print(server_response)
log.info("Resubscribe ---> <{0}>".format(server_response))[/SYNTAX]In the _handle_stream method I replaced the last line "raise IOError()" with "self.reconnect()", and the end of the _receive method is now : [SYNTAX=PYTHON] self._stream_connection = None
if not rebind:
self.reconnect()
else:
self.bind()[/SYNTAX]
Thanks !
Gianluca Finocchiaro
Hi epep,
you can always test your code against a Lightstreamer version available from our
site(for example, you can install a free Moderato version or a Vivace demo). After installation, please follow the instructions on how to deploy the
StockList Demo Adapter .
That said, your modifications look very good, just a couple of observations:
- Replacing the "raise IOError()" in the _handle_stream method is not a good idea, because a server error response while connecting is a symptom that something is not working properly, as you can read from Paragraph 4.1 of the Network Protocol Tutorial
- It is necessary to remove the following line:
[SYNTAX=PYTHON]self._stream_connection = None [/SYNTAX]
before invoking reconnect(), otherwise you'll run against a runtime error: it is a bug that I'm going to fix in our example
Please let us know about your progresses.
Thanks and Regards,
Gianluca
epep
Gianluca Finocchiaro Replacing the "raise IOError()" in the _handle_stream method is not a good idea, because a server error response while connecting is a symptom that something is not working properly, as you can read from Paragraph 4.1 of the Network Protocol Tutorial
In fact the only error I encountered for now with IG's stream server happen here, about 3 times a day, while trying to bind. I receive a SYNC ERROR about half a second after a LOOP message, before the binding is done. In this situation the reconnection work at first try.
I understand that errors in this point should be used with care for a clean client, but for my project I believe that the only errors that can occur here are the temporary ones, which would be resolved with a later reconnection.
Gianluca Finocchiaro It is necessary to remove the following line :Smile_Ah: SYNTAX=PYTHON]self._stream_connection = None [/SYNTAX]before invoking reconnect(), otherwise you'll run against a runtime error: it is a bug that I'm going to fix in our example
Thanks a lot ! I see the point : None has no close() method. :Smile_Ag:
This line is also worthless before calling the bind() method ?
Gianluca Finocchiaro
Hi epep,
epep ...
This line is also worthless before calling the bind() method ?
yes because the bind() method replaces
self._stream_connectionwith a newly created object: see fixed example
here .
Let me know.
epep
Ok, it works fine without the "self._stream_connection = None".
I notice that my logging was badly formated in reconnect(), I should have written : [SYNTAX=PYTHON] def reconnect(self):
fail = 0
while True :
try :
log.info("Trying to reconnect")
self._stream_connection.close()
self._session.clear()
self.connect()
self.resubscribe()
break
except :
log.info("fail : {0}".format(sys.exc_info()[0]))
fail += 1
wait = min( 300, 0.125 * ( 2 ** fail ))
log.info("reconnect in {0} seconds".format(wait))
time.sleep(wait)[/SYNTAX]
About errors documented in 4.1 of the network protocol tutorial, do you agree that they should not happen after the connection is established ? I've done this at the end of the _handle_stream method : [SYNTAX=PYTHON] if self._bind_counter == 0 :
raise IOError()
else :
self.reconnect()[/SYNTAX]Like this, if an error occurred at the first connection, it is raised. If it encounters the bind problem from IG, it reconnects. Is it clean ?
Gianluca Finocchiaro
Hi epep,
epep
...
About errors documented in 4.1 of the network protocol tutorial, do you agree that they should not happen after the connection is established ? I've done this at the end of the _handle_stream method : [SYNTAX=PYTHON] if self._bind_counter == 0 :
raise IOError()
else :
self.reconnect()[/SYNTAX]Like this, if an error occurred at the first connection, it is raised. If it encounters the bind problem from IG, it reconnects. Is it clean ?
In case of binding an existing session (hence
_bind_counter is greater then 0), you could run against similar issues, as specified in the Paragraph 4.2 of the
Network Protocol tutorial.
Obviously you are free to implement the reconnection logic as per your requirements, but in my opinion you should trigger it only if you can be sure that a real network problem has been verified.
Let me know,
Gianluca
epep
You're right !
It seems that IG changed something about the persistence of authentication tokens, my script get stuck trying to reconnect with invalid password … I need more intelligence here. I'll let you know as soon as I have done something good.
Mamoa
epep You're right !
It seems that IG changed something about the persistence of authentication tokens, my script get stuck trying to reconnect with invalid password … I need more intelligence here. I'll let you know as soon as I have done something good.
Hello epep, I'm trying to get the IG Streaming API up and running but I'm having the same problem as you. Did you finally manage to solve the problem? Thanks!
fab
epep You're right !
It seems that IG changed something about the persistence of authentication tokens, my script get stuck trying to reconnect with invalid password … I need more intelligence here. I'll let you know as soon as I have done something good.
Hi guys, did you work this issue out? I am getting a SYNC ERROR and would like to re-bind the session but then an IOERROR() is raised on the rebind.
Thanks a lot for your help in advance.
strangecarr
Hi, can anyone recommend the best solution to connect to IG API using python? I've tried the trading_ig library by Femto, but can't get any data to stream - just keep getting the PROBE messages - e.g:
DEBUG:trading_ig.lightstreamer:Received message ---> <PROBE>
DEBUG:trading_ig.lightstreamer :Smile_Ae: ROBE message
DEBUG:trading_ig.lightstreamer:Waiting for a new message
Gianluca Finocchiaro
Hi strangecarr,
by using our
Python client demo, you should be able to connect to IG API for streaming, as other users of this forum have already done.
Please have a look at previous discussions on this same
thread to get more information and please get back to us for any technical details regarding the client.
Best Regards,
Gianluca
strangecarr
Hi Gianluca,
Thanks for quick reply, I tried the code of Colomboa too, which modifies the stock_list_demo script to connect to the IG API, but I only received "cannot connect to Lightstreamer server". I updated the CST and XST details. Any advice on the best way to edit the stock_list_demo to connect to IG?
Gianluca Finocchiaro
Sorry, we encountered an error while displaying this content. If you're a user, please try again later. If you're an administrator, take a look in your Flarum log files for more information.
