socalwill
Hi and thanks so much - I just saw your message after updating the post. Do you happen to have access to a Mac or Linux box or maybe cygwin under Windows? I am thinking curl could be great to efficiently get to the bottom of things.
Gianluca Finocchiaro
Hi William,
I can't manage to issue the commands of your post, because I would need your real credentials...
Anyway, in the control request for submitting the subscription (your second command) the "LS_Session" parameter must be set with the SessionId got as result of the first command, when you requested to open the session.
Let me know.
Gianluca
socalwill
Hello Gianluca - yes, that's what I did. I just sent you a pm with the commands and the server messages. I believe that should be helpful for you. Thanks!
Gianluca Finocchiaro
Ok William,
I've misinterpreted MYLSSESSIONID (very similar to MYSESSIONID), sorry...
The commands seem right, but with no direct access to the system is very hard to understand what goes wrong.
If it is ok for you, send me via private mail your credentials in order to reproduce the case.
Regards,
Gianluca.
Gianluca Finocchiaro
Hi William,
issues are involved in LS_id and LS_schema parameters. It's not clear to me how many different items and related field schemas are available from Cityindex, but reading through their forum, it seems the following should work:
LS_id=PRICE.400616114
LS_schema=MarketId%20TickDate%20Bid%20Offer%20Price
Here the Python version:
[SYNTAX=PYTHON]
subscription = Subscription(
adapter="PRICES",
mode="MERGE",
items=["PRICE.400616114"],
fields=["MarketId", "TickDate", "Bid", "Offer", "Price"]
)
# A simple function acting as a Subscription listener
def on_item_update(item_update):
print("{MarketId}, {TickDate}, {Bid}, {Offer}, {Price}".format(**item_update["values"]))[/SYNTAX]
Let me know.
Gianluca
Colombao
Hi Gianluca, sorry to trouble you again :Smile_Ab:
After the login and the subscriptions, after about 20-40 minutes I correctly receive push updates from IG, I don't receive any push update anymore, like if the connection is lost.
This behavior is sistematic and reproducible with others PC or internet connections.
The workaround I adopted is to setup a timer that if he see e.g. 20 seconds of silence, re-login and make subscriptions again but...this happen each 20 minutes, it is too frequent...
Do you have any other solution / advice ? Could it be caused by the program that doesn't elaborate push updates enough quick?
Thanks again!
Colombao
socalwill
Hello Gianluca - this is excellent! With your help I got both the plain curl commands working as well as the python script, which I will use for my purposes down the line. Thanks so much! I will continue with the implementation with other parts, that are more broker-specific, and I will post a working code soon. Thanks again,
William
Gianluca Finocchiaro
Hi Dario,
this is a known problem but thank you for reporting it. I'm going to fix the demo and I'll let you know once done.
Thanks and Regards,
Gianluca
Gianluca Finocchiaro
Hi Dario,
I've just released on Github a new version of the
Basic StockList-Demo with support for bind operation, which allows a "fluent" streaming with no need of workarounds.
If you are interested at the protocol details,
here you can find an exhaustive explanation [Par. 4.2].
Let me know.
Regards,
Gianluca
socalwill
This is great, thanks, Gianluca. I notice that reproducible interruption as well and I just started simple workaround with a ten minute duration for the streaming and a ten min periodic restart via cronjob, but your fix will be much better. (This may be intentional, but I noticed you kept the debug mode in line 397.)
Gianluca Finocchiaro
Hi William,
I confirm that such workarounds go against the "session binding" as designed inside the Lightstreamer Server, so it is better to avoid them and to use the right features the Lightstreamer Network Protocol offers to leverage the power of the server.
Keeping the debug mode was a misprint, now I've fixed it and the source code is available on
Github.
Thanks and Regards,
Gianluca
Colombao
Gianluca, with your last update the number of interruptions decreased dramatically. Gianluca the best!
Gianluca Finocchiaro
Hi Dario,
thanks to your precious feedback I was finally able to fix last minor bugs. Now the implementation seems stable even when connected to several Lightstreamer instances deployed in a cluster.
Thank You!
Gianluca
epep
Hi ! And thanks for your work !
I want my LightStreamer client to try to reconnect whatever the situation, it seems to work for now (not testing since a long time in fact), but as I don't test against my own server I have to wait for errors to happen, and being new to LightStreamer and Python I suppose I'm doing some not clever things. Can you please give me your appreciation about my code ?
I modified your python client this way :
Added these two methods to LSClient class :Smile_Ah: SYNTAX=PYTHON] def reconnect(self):
fail = 0
while True :
try :
log.info("trying to reconnect")
self._stream_connection.close()
self._session.clear()
self.connect()
self.resubscribe()
break
except :
log.info("fail :", sys.exc_info()[0])
fail += 1
wait = min( 300, 0.125 * ( 2 ** fail ))
log.info("next try in {0} seconds", wait)
time.sleep(wait)
def resubscribe(self):
""""resubscribe old items after starting a new connection"""
for subscription_key in self._subscriptions.keys() :
# Send the control request to perform the subscription
subscription = self._subscriptions[subscription_key]
server_response = self._control({
"LS_Table": subscription_key,
"LS_op": OP_ADD,
"LS_data_adapter": subscription.adapter,
"LS_mode": subscription.mode,
"LS_schema": " ".join(subscription.field_names),
"LS_id": " ".join(subscription.item_names),
})
print(server_response)
log.info("Resubscribe ---> <{0}>".format(server_response))[/SYNTAX]In the _handle_stream method I replaced the last line "raise IOError()" with "self.reconnect()", and the end of the _receive method is now : [SYNTAX=PYTHON] self._stream_connection = None
if not rebind:
self.reconnect()
else:
self.bind()[/SYNTAX]
Thanks !
Gianluca Finocchiaro
Hi epep,
you can always test your code against a Lightstreamer version available from our
site(for example, you can install a free Moderato version or a Vivace demo). After installation, please follow the instructions on how to deploy the
StockList Demo Adapter .
That said, your modifications look very good, just a couple of observations:
- Replacing the "raise IOError()" in the _handle_stream method is not a good idea, because a server error response while connecting is a symptom that something is not working properly, as you can read from Paragraph 4.1 of the Network Protocol Tutorial
- It is necessary to remove the following line:
[SYNTAX=PYTHON]self._stream_connection = None [/SYNTAX]
before invoking reconnect(), otherwise you'll run against a runtime error: it is a bug that I'm going to fix in our example
Please let us know about your progresses.
Thanks and Regards,
Gianluca
epep
Gianluca Finocchiaro Replacing the "raise IOError()" in the _handle_stream method is not a good idea, because a server error response while connecting is a symptom that something is not working properly, as you can read from Paragraph 4.1 of the Network Protocol Tutorial
In fact the only error I encountered for now with IG's stream server happen here, about 3 times a day, while trying to bind. I receive a SYNC ERROR about half a second after a LOOP message, before the binding is done. In this situation the reconnection work at first try.
I understand that errors in this point should be used with care for a clean client, but for my project I believe that the only errors that can occur here are the temporary ones, which would be resolved with a later reconnection.
Gianluca Finocchiaro It is necessary to remove the following line :Smile_Ah: SYNTAX=PYTHON]self._stream_connection = None [/SYNTAX]before invoking reconnect(), otherwise you'll run against a runtime error: it is a bug that I'm going to fix in our example
Thanks a lot ! I see the point : None has no close() method. :Smile_Ag:
This line is also worthless before calling the bind() method ?
Gianluca Finocchiaro
Hi epep,
epep ...
This line is also worthless before calling the bind() method ?
yes because the bind() method replaces
self._stream_connectionwith a newly created object: see fixed example
here .
Let me know.
epep
Ok, it works fine without the "self._stream_connection = None".
I notice that my logging was badly formated in reconnect(), I should have written : [SYNTAX=PYTHON] def reconnect(self):
fail = 0
while True :
try :
log.info("Trying to reconnect")
self._stream_connection.close()
self._session.clear()
self.connect()
self.resubscribe()
break
except :
log.info("fail : {0}".format(sys.exc_info()[0]))
fail += 1
wait = min( 300, 0.125 * ( 2 ** fail ))
log.info("reconnect in {0} seconds".format(wait))
time.sleep(wait)[/SYNTAX]
About errors documented in 4.1 of the network protocol tutorial, do you agree that they should not happen after the connection is established ? I've done this at the end of the _handle_stream method : [SYNTAX=PYTHON] if self._bind_counter == 0 :
raise IOError()
else :
self.reconnect()[/SYNTAX]Like this, if an error occurred at the first connection, it is raised. If it encounters the bind problem from IG, it reconnects. Is it clean ?
Gianluca Finocchiaro
Hi epep,
epep
...
About errors documented in 4.1 of the network protocol tutorial, do you agree that they should not happen after the connection is established ? I've done this at the end of the _handle_stream method : [SYNTAX=PYTHON] if self._bind_counter == 0 :
raise IOError()
else :
self.reconnect()[/SYNTAX]Like this, if an error occurred at the first connection, it is raised. If it encounters the bind problem from IG, it reconnects. Is it clean ?
In case of binding an existing session (hence
_bind_counter is greater then 0), you could run against similar issues, as specified in the Paragraph 4.2 of the
Network Protocol tutorial.
Obviously you are free to implement the reconnection logic as per your requirements, but in my opinion you should trigger it only if you can be sure that a real network problem has been verified.
Let me know,
Gianluca
epep
You're right !
It seems that IG changed something about the persistence of authentication tokens, my script get stuck trying to reconnect with invalid password … I need more intelligence here. I'll let you know as soon as I have done something good.
Mamoa
epep You're right !
It seems that IG changed something about the persistence of authentication tokens, my script get stuck trying to reconnect with invalid password … I need more intelligence here. I'll let you know as soon as I have done something good.
Hello epep, I'm trying to get the IG Streaming API up and running but I'm having the same problem as you. Did you finally manage to solve the problem? Thanks!
fab
epep You're right !
It seems that IG changed something about the persistence of authentication tokens, my script get stuck trying to reconnect with invalid password … I need more intelligence here. I'll let you know as soon as I have done something good.
Hi guys, did you work this issue out? I am getting a SYNC ERROR and would like to re-bind the session but then an IOERROR() is raised on the rebind.
Thanks a lot for your help in advance.
