Python client for lightstreamer

Hi epep,

epep ...
This line is also worthless before calling the bind() method ?

yes because the bind() method replaces self._stream_connectionwith a newly created object: see fixed example here .

Let me know.

Ok, it works fine without the "self._stream_connection = None".

I notice that my logging was badly formated in reconnect(), I should have written : [SYNTAX=PYTHON] def reconnect(self):
fail = 0
while True :
try :
log.info("Trying to reconnect")
self._stream_connection.close()
self._session.clear()
self.connect()
self.resubscribe()
break
except :
log.info("fail : {0}".format(sys.exc_info()[0]))
fail += 1
wait = min( 300, 0.125 * ( 2 ** fail ))
log.info("reconnect in {0} seconds".format(wait))
time.sleep(wait)[/SYNTAX]

About errors documented in 4.1 of the network protocol tutorial, do you agree that they should not happen after the connection is established ? I've done this at the end of the _handle_stream method : [SYNTAX=PYTHON] if self._bind_counter == 0 :
raise IOError()
else :
self.reconnect()[/SYNTAX]Like this, if an error occurred at the first connection, it is raised. If it encounters the bind problem from IG, it reconnects. Is it clean ?

Hi epep,

epep
...

About errors documented in 4.1 of the network protocol tutorial, do you agree that they should not happen after the connection is established ? I've done this at the end of the _handle_stream method : [SYNTAX=PYTHON] if self._bind_counter == 0 :
raise IOError()
else :
self.reconnect()[/SYNTAX]Like this, if an error occurred at the first connection, it is raised. If it encounters the bind problem from IG, it reconnects. Is it clean ?

In case of binding an existing session (hence _bind_counter is greater then 0), you could run against similar issues, as specified in the Paragraph 4.2 of the Network Protocol tutorial.

Obviously you are free to implement the reconnection logic as per your requirements, but in my opinion you should trigger it only if you can be sure that a real network problem has been verified.

Let me know,
Gianluca

You're right !

It seems that IG changed something about the persistence of authentication tokens, my script get stuck trying to reconnect with invalid password … I need more intelligence here. I'll let you know as soon as I have done something good.

Hi, can anyone recommend the best solution to connect to IG API using python? I've tried the trading_ig library by Femto, but can't get any data to stream - just keep getting the PROBE messages - e.g:
DEBUG:trading_ig.lightstreamer:Received message ---> <PROBE>
DEBUG:trading_ig.lightstreamer :Smile_Ae: ROBE message
DEBUG:trading_ig.lightstreamer:Waiting for a new message

Hi strangecarr,

by using our Python client demo, you should be able to connect to IG API for streaming, as other users of this forum have already done.
Please have a look at previous discussions on this same thread to get more information and please get back to us for any technical details regarding the client.

Best Regards,
Gianluca

Hi Gianluca,
Thanks for quick reply, I tried the code of Colomboa too, which modifies the stock_list_demo script to connect to the IG API, but I only received "cannot connect to Lightstreamer server". I updated the CST and XST details. Any advice on the best way to edit the stock_list_demo to connect to IG?

epep You're right !

It seems that IG changed something about the persistence of authentication tokens, my script get stuck trying to reconnect with invalid password … I need more intelligence here. I'll let you know as soon as I have done something good.

Hello epep, I'm trying to get the IG Streaming API up and running but I'm having the same problem as you. Did you finally manage to solve the problem? Thanks!

Hi Gianluca,

First of all, thank you very much for the code, it has been really useful for me. I'm also using it to connect to the IG Streaming API and it has work just fine, the only problem that I'm having is the reconnect issue as stated by IG:

The Lightstreamer client should also have the ability to reconnect in the event of a connection failure or if instructed by the Lightstreamer server (e.g. due to a LOOP rebind message). A re-connection attempt can fail if the CST and X-SECURITY-TOKEN tokens have expired in which case it will be necessary to re-authenticate to obtain new security tokens.

I was thinking on adding a Timer to the LSClient class, connecting the timeout (after 2 seconds for example) to a new method that unsubscribes, disconnects and reconnects and resubscribe. Obviously I will have to end the session with the rest api and start a new one to get new session tokens.
What do you think of this approach?

Thanks in advance!

Gianluca Finocchiaro Hi Mamoa,
it can certainly work, but the LSClient should be left as a general purpose Lightstreamer Client, hence I believe that your proposed logic should be put somewhere else, because it catches a specific use case.

Reconnection is a feature handled by other official Lightstreamer Client, by it is supposed that credentials do not change or, if they do, you have to write some custom logic to provide updated values.

Please let me know if your reconnection proposal also involves handling of new session tokens.

Thanks, Gianluca

Hi Gianluca,

Thanks for the reply. I have been very busy lately and I'm sorry that I didn't get back to you sooner. As you said, my case is really specific to my situation so it shouldn't be added to your lightstreamer client.
To answer your question, yes, my logic requires handling new session tokens so actually I had to use another api client (Rest API from IG) to get reconnected and getting new sessions tokens.
Finally I got it running (with a few custom modifications) but I'm having problems with a special case.

IG Streaming API (https://labs.ig.com/streaming-api-reference) has a special case where you have a DISTINCT Mode subscription , with 1 Item (TRADE) and 1 Field (OPU) but within that field you have various elements. When I subscribe to this trade I see the following message printed by the LightStreamer Client:

Received update message ---> <1,1|{"stopLevel":1.74014,"trailingStopDistance":0,"limitLevel":null,"trailingStep":0,"currency":"CAD","expiry":"-",
"guaranteedStop":false,"dealIdOrigin":"DIAAAABBHRQJLBA","direction":"SELL","epic":"CS.D.GBPCAD.MINI.IP",
"dealId":"DIAAAABBHRQJLBA","dealReference":"GCMJXRBVSFD44TD","level":1.73414,"status":"OPEN","size":1,
"channel":"PublicRestOTC","timestamp":"2017-05-26T07:23:21.000","dealStatus":"ACCEPTED"}>

but in the table listener I get this message:

{'values': {'OPU': u'{"stopLevel":1.74014'}, 'pos': 1, 'name': 'TRADE:MCMVD'}

As you can see, I only received the first element of the dict in the listener. I could make an adaptation for this case but I thought it could be positive to add this feature (multiple elements within the field) to your lightstreamer client. What do you think?

Thanks again,
Mamoa

Hi Mamoa,

thank you very much for this post, as you helped me find out a bug in the demo.
Actually, the 'values' dict should have been looked like the following:

{'values': {'OPU':u'{"stopLevel":1.74014,"trailingStopDistance":0,"limitLevel":null,"trailingStep":0,"currency":"CAD","expiry":"-","guaranteedStop":false,"dealIdOrigin":"DIAAAABBHRQJLBA","direction":"SELL","epic":"CS.D.GBPCAD.MINI.IP","dealId":"DIAAAABBHRQJLBA","dealReference":"GCMJXRBVSFD44TD","level":1.73414,"status":"OPEN","size":1,"channel":"PublicRestOC","timestamp":"2017-05-26T07:23:21.000","dealStatus":"ACCEPTED"}', 'pos': 1, 'name': 'TRADE:MCMVD'}

where the value associated with the 'OPU' key is a json string to be processed by your custom logic.
I've just fixed the client on our github repository.

Please let me know wheter it helps.

Thanks and Regards.
Gianluca

Hi GianLuca,

I tested the fixed client and now is working perfectly. Thank you very much!

Regards,
Mamoa

Hello guys

I have the following issue reoccurring somewhat randomly, like after 4, 10, 24 hours etc.

All of the sudden the streamer just stops receiving messages and this is the last one:

Waiting for a new message
Received message ---> <1,1||||||||||||||1|>
Received update message ---> <1,1||||||||||||||1|>

I've downloaded the python LSClient from your Github examples, so the rebind = True function is available. Previously I was using an old version from the pip repository and that always closed the connection after around one hour. Where the rebinding didn't even occur!

Please advise on how to debug this.

Hi majestrooo,

let me first clarify that the Lightstreamer Client Python package actually available from the pip repository is not provided by our company.

Could you provide us with a server side log excerpt of the issue?

Thanks,
Gianluca

Gianluca Finocchiaro Hi majestrooo,

let me first clarify that the Lightstreamer Client Python package actually available from the pip repository is not provided by our company.

Could you provide us with a server side log excerpt of the issue?

Thanks,
Gianluca

Hello Gianluca

I am using the lightstreamer code from this URL https://github.com/Lightstreamer/Lightstreamer-example-StockList-client-python/blob/master/src/stock_list_demo.py

The only part I removed is the stuff in the end for the demo.

What server side log do you need? I only have those of the ls_streamer. Otherwise please advise if there's a way to turn on more logging.

Gianluca Finocchiaro

{'values': {'OPU':u'{"stopLevel":1.74014,"trailingStopDistance":0,"limitLevel":null,"trailingStep":0,"currency":"CAD","expiry":"-","guaranteedStop":false,"dealIdOrigin":"DIAAAABBHRQJLBA","direction":"SELL","epic":"CS.D.GBPCAD.MINI.IP","dealId":"DIAAAABBHRQJLBA","dealReference":"GCMJXRBVSFD44TD","level":1.73414,"status":"OPEN","size":1,"channel":"PublicRestOC","timestamp":"2017-05-26T07:23:21.000","dealStatus":"ACCEPTED"}', 'pos': 1, 'name': 'TRADE:MCMVD'}

where the value associated with the 'OPU' key is a json string to be processed by your custom logic.
I've just fixed the client on our github repository.

Please let me know wheter it helps.

Thanks and Regards.
Gianluca

This works like the magic for me as well. Thank you!

Hi Majestrooo,

we need the log files produced by your Lightstreamer Server installation, under <ls_home>/logs.

Thanks
Gianluca

Gianluca Finocchiaro Hi Majestrooo,

we need the log files produced by your Lightstreamer Server installation, under <ls_home>/logs.

Thanks
Gianluca

Hi Gianluca

I'm afraid I don't have those, I'm only using the client towards IG.