Disconection

rperseguini

Hello,

I'm using a "home made" C++ TLCP client using http get to connect to ligthstreamer server.

Every time it gets disconnected with a LOOP message and I have to rebind.
I was expecting this behavior only wtih POST connections, that are size-limited.

I have another vendor "http get chunk stream" in another app that does never disconnects.

Is there anyway of avoiding this disconnection with lightstreamer ?

2020-02-03 10:31:57 INFO LOOP
2020-02-03 10:33:06 INFO LOOP
2020-02-03 10:34:15 INFO LOOP
2020-02-03 10:35:29 INFO LOOP
2020-02-03 10:36:43 INFO LOOP
2020-02-03 10:38:24 INFO LOOP
2020-02-03 10:39:42 INFO LOOP
2020-02-03 10:40:55 INFO LOOP

yes, I have a lot of data being sent.

Dario Crivelli

Hello,

There are many ways to submit requests to Lightstreamer Server and some of them lead to the LOOP command.
We should see how the Server sees the request.
Do you have the opportunity to provide the DEBUG log of the Server?
For this we need the LightstreamerLogger.requests and LightstreamerLogger.connections.http loggers set to DEBUG. This does not log all the returned data.

For now, what I can clarify is that, to LS Server, the main difference in not made by GET vs POST but by HTTP 1.0 (which enforces a content-length header) vs HTTP 1.1 (which supports "chunked" responses).

rperseguini

Hi,

I'll check the logs.

I'm using http 1.1 and I'm getting a chunked response.

Response HTTP/1.1 200 OK
Header 0 - Date: Tue, 04 Feb 2020 19:10:20 GMT
Header 1 - Server: Lightstreamer-Server/7.0.3 build 1885.8 (Lightstreamer Server - www.lightstreamer.com) ENTERPRISE edition
Header 2 - Content-Type: text/enriched; charset=UTF-8
Header 3 - Cache-Control: no-store
Header 4 - Cache-Control: no-transform
Header 5 - Cache-Control: no-cache
Header 6 - Pragma: no-cache
Header 7 - Expires: Thu, 1 Jan 1970 00:00:00 GMT
Header 8 - x-accel-buffering: no
Header 9 - Keep-Alive: timeout=5, max=100
Header 10 - Connection: Keep-Alive
Header 11 - Transfer-Encoding: chunked

Thanks

Dario Crivelli

I see.
The only strange thing is
Header 9 - Keep-Alive: timeout=5, max=100
which is not produced by LS Server; and the related
Header 10 - Connection: Keep-Alive
which is never sent by LS Server on HTTP 1.1 responses.
This may hint that something acts in the middle.

Anyway, we wait for the logs, because some request parameters can also cause the Server to interrupt the response.
For instance, settings for polling or for client inactivity checks.

rperseguini

hello,

I'm waiting the logs too.

We're moving to another datacenter and the test server is not available, I think this will require some days to be normalized.

It was not said yet, but HTTP connections to the server are handled by an Apache server, configured as a reverse proxy.

About keep-alive header:
I was sending the keep-alive header, maybe LS was echoing it.
I remove that, and this header does not arrives anymore.

Header 0 - Date: Fri, 07 Feb 2020 18:32:08 GMT
Header 1 - Server: Lightstreamer-Server/7.0.3 build 1885.8 (Lightstreamer Server - www.lightstreamer.com) ENTERPRISE edition
Header 2 - Content-Type: text/enriched; charset=UTF-8
Header 3 - Cache-Control: no-store
Header 4 - Cache-Control: no-transform
Header 5 - Cache-Control: no-cache
Header 6 - Pragma: no-cache
Header 7 - Expires: Thu, 1 Jan 1970 00:00:00 GMT
Header 8 - x-accel-buffering: no
Header 9 - Transfer-Encoding: chunked

But the server still disconnecting the client after LOOP arrival.

Dario Crivelli

Hello,
I exclude that LS Server echoes the keep-alive header and I suppose that your reverse proxy does.
Anyway, the presence of a reverse proxy in the middle is an important aspect of the issue.

If you were using one of our SDKs, I would argue that your reverse proxy doesn't support neither websocket nor HTTP streaming and that
the SDK library, by applying the StreamSense algorithm, just resorts to a polling session.
Since you have a home-made client, this cannot apply. But, perhaps, you modeled your client after running a client based on LS SDK? So, the client resorted to polling requests and you adopted the same requests for your client?

This should be clarified by looking at these requests. In particular, I refer to the create_session and bind_session requests.
If you can't get Server logs, for the moment you can directly look on the client side and see how your client performs these requests.

BTW, note that if the use of websocket or http streaming is not available because of restrictions imposed by the reverse proxy, it may be just a matter of configuration on the Apache side.

rperseguini

Hello,

I'm sorry for the lack of time, but I need to proceed with this issue.

I've some logs of lightstreamer from the test server.

And I have a hundred customers binding every minute.

Bellow an image form production server about URL hitting.

Thanks in advance.

EmersonPardo

I'd like to add our Apache proxy reverse configuration:

ProxyRequests Off

# Turn off VIA header as we know where the requests are proxied
ProxyVia Off
ProxyPreserveHost On

# Set the permissions for the proxy
<Proxy *>
AddDefaultCharset off
Order deny,allow
Allow from all
</Proxy>

ProxyPass /lightstreamer http://localhost:9080/lightstreamer

Dario Crivelli

Hello,
The log helped us to clarify the scenario.
Since the rebinds occur every minute and the flow of updates is significant, this is consistent with the Server behavior.
Actually, I was wrong when hinting that chunked responses were not limited in size. Sorry for that.
In fact, the <content_length> setting in the configuration file, which is currently 4000000, is also enforced to chunked responses.
This is done in order to avoid that some user-agent or network intermediary tries to buffer the whole response and runs into problems if the response is too long.

You can modify and enlarge the <content_length> setting. We don't support unlimited lengths, but you can set a very high value (up to the size of a Java "long").
Even better, you can override the setting only for your C++ client. The client would just add the LS_content_length=xxxx parameter to the querystring, with a proper length.
As said, you should also ensure that your Apache reverse proxy does not cache the responses, although this is probably already ensured, because the Server response headers require so.

rperseguini

Hi, Dario.

It's working well on testing environment

Thanks a lot.