Cerogil
Using the Moderato version, we are testing the push cabilities of Lightstreamer.
We are using one data adapter. The data adapter is pushing out 8000+ pieces of data seperately at client startup.
However, the clients only recieve 1,000 - 2,000 pieces of data.
It looks like Lighstreamer receives the first piece of data to send and pushes it out. It then keeps receiving data to send but waits for the one second limit that is in the Moderato version of Lightstreamer. By the time it can send out data, there is a large chunk to push. Most of this data is not getting to the client.
We initially ran this with ~700 pieces of data, and they all got to the client.
Is there a limit to Lightstreamer's buffer? Any limits from the Moderato version in this regard?
Additionally, when we bring up a second client with that same subscription (8000+ pieces of data), most of the time the client only receives 999 pieces of data. Occasionally it gets over 1,000. I believe Lightstreamer uses some kind of buffering/paging to keep this subscription data handy, so why is this new client with the same subscription getting less data?
Thanks,
Tim
Alessandro Alinone
Hi Tim,
I guess you are using "MERGE" subscription mode, which filters out events instead of buffering them and delivering them with delays.
If that's the case, give a try to "RAW" subscription mode.
Cheers,
Alessandro
Cerogil
Sorry for the late response to this. We had a workaround by limiting the amount of data we were pushing at one time to less than a thousand.
We are not using MERGE mode, we are using COMMAND mode.
When Lightstreamer goes to push the thousands of data to the client, the Lightstreamer command window shows the following: "... < INFO> Lost x updates on unfiltered/RAW subscription for item ..." The x ranges anywhere from 1 to thousands. Many of these lines show up, and they appear to add up to the number of items that do not make it to the client.
We want to stay in command mode to have a snapshot available for other users to grab from and to handle the many updates throughout the day. We'd rather not limit the pushes, and do not a way to limit the pushes from a snapshot for new clients using the same subscription - they lose the data.
Any ideas why this might be occuring?
Thanks.
Dario Crivelli
In COMMAND mode, the "one event per second" limitation imposed by Lightstreamer Moderato only affects single keys, hence the snapshot, which is made up of different keys, should flow fast.
However, your snapshot seems to be very large, so the <max_buffer_size> limitation applies in this case.
This setting avoids that the event queue for a single subscription grows beyond any limit.
If you know that snapshots of many thousand events can be expected, then you should increase <max_buffer_size> accordingly.
Note that this may affect all event queues, related to items in RAW mode or with the "unfiltered" flag, for which the Data Adapter event rate fairly exceeds the "one event per second" outbound limitation.
In the forthcoming 3.6 release of the Server, things are going to change and <max_buffer_size> will no longer limit event queues when they are related with COMMAND mode snapshot.
wwatts
In this thread, it states that the max_buffer_size will not be a limitation in the 3.6. We currently are using Presto version of 3.6 build 1463.2 and we had to configure this parameter. Therefore:
1. Is that update going to be in another build within 3.6?
2. Will the buffer now be dynamic and queue as necessary to update the snapshot using command mode?
Thanks for the help.
Dario Crivelli
I confirm that, since 3.6, the snapshots related to items subscribed to in COMMAND mode are sent to the clients entirely, even if a temporary queue longer than the configured <max_buffer_size> has to be used.
The <max_buffer_size> limitation is still obeyed in the other cases.
Are you experiencing anything different?
wwatts
Yes we seem to be experiencing different behavior. We had an update to a snapshot that was over 1000 items. We received the first 1000 items and then received that we lost raw updates on the remainder of the updates. When we changed the value for max_buffer_size in the conf file, we received all updates.
Some more information that may be helpful is that this behavior was on a demo license. We have not tried a similar situation on our production instance.
Dario Crivelli
It is important that the snapshot is formally identified as a snapshot by the Data Adapter when it invokes update/ smartUpdate on the supplied events.
Obviously, the snapshot kept internally by the Server to manage subsequent subscriptions doesn't have such problem, hence I assume that the issue occurs on the first subscription of the item.
May you please ensure that?
On the other hand, updates that are not part of the snapshot can be lost if they fill the protection buffer.
The license should not impact, but different editions (e.g. Moderato, Vivace) impact on the speed with which the updates are sent.
By the way, is your item subscribed to with the "unfiltered" flag?
