First client subscript in DISTINCT mode with unfiltered and snapshot, no update.

Alexander

I am trying to subscript in DISTINCT mode with
*subsItemList1.setRequestedSnapshot("yes");
*subsItemList1.setRequestedMaxFrequency("unfiltere d");

after the LightStreamer server just restarted.

The client just not able to receive any update after snapshot.
Please help.

Debug log attached:

Dario Crivelli

Hi, thank you for the detailed report.
This seems to be really a bug related to the combination of DISTINCT items, unfiltered dispatching, and snapshot handling.
Moreover, it is subject to a race condition that depends on the way snapshot and updates are produced.
All these conditions, unfortunately, occur in your case.

In particular, if the Data Adapter sends a snapshot event but not an end-of-snapshot event, the snapshot is kept and not sent to the subscribers.
As you can see from your log, only when a second update as non-snapshot is sent, is the snapshot forwarded to the session and a synthetic end-of-snapshot event added.
The reason for this delay is that a "distinct snapshot length" is configured, which poses a maximum limit to the snapshot to be sent to clients.
Hence the Server, as long as it doesn't know the final length of the snapshot that will be sent by the Data Adapter, it also doesn't know if it will have to truncate the first part; therefore it cannot initiate the transfer of the snapshot to the client.
As a consequence, the three events (snapshot, end-of-snapshot, and update) are sent in short sequence to the session and may be received in a different order, as happens at 21:26:57,803.
Obviously, the session is supposed to restore the order, but here comes the bug.

We will fix the bug as soon as possible thanks to your notice; but, anyway, you should escape from the bug by removing any of the various conditions described above.
I suppose that some of these conditions are not really wanted, but only side-effects of using the StockListDemo Data Adapter to receive updates in DISTINCT mode rather than MERGE.
In particular, in a real scenario, we suggest always sending the end-of-snapshot event for DISTINCT items, as the observed delay in sending the snapshot is already something that you should avoid.

Alexander

Hi,

Thank you for the prompt response.

I did try to send the EOS event for the items, but still get the same result, no new updates were pushed to the client but the snapshot only.

Please help, and have a look into the log file, thank you.

==DEBUG.log==

Dario Crivelli

I see. It seems that the two first events, the snapshot and the EOS, are enough to trigger the race condition (at 03:13:19,821) and the bug.
This means that, unfortunately, the conditions for the bug are broader.
Please try removing the "distinct snapshot length" setting; it is configured in adapters.xml. It still causes the two events to be dispatched to the session together.
If you still have problems, please also try enforcing a short delay before sending the EOS.

Dario Crivelli

I'm no longer sure the last workaround can work.
I cannot test the case today; I will come back to you tomorrrow.

Alexander

Thank you for the prompt response!

Are you able to reproduce it in your environment?
I can write down how to reproduce it step by step, if needed.

For the workaround suggestions:
1) remove the "distinct_snapshot_length" or setting it to some other value like 10,20 does not help.
2) enforcing a short delay before sending the EOS does not help too.

Dario Crivelli

Yes, I managed to replicate the bug, but not to test the proposed workarounds.
I hoped that the "distinct snapshot length" setting could be set to unlimited, but now I realize that this is not possible.
As a consequence, the snapshot and the EOS are always sent to the session at once, hence the race condition that triggers the bug is always possible.

In practice, the only condition for the bug is the combination of DISTINCT item, unfiltered dispatching, and snapshot handling.
This is worse than I initially thought.
We will publish a new release with the fix as soon as possible and will let you know.

Dario Crivelli

Hi,
A new public release of Lightstreamer is not planned yet.
In the meantime, we have uploaded a patched version (7.3.1) at this url:
https://lightstreamer.com/distr/ls-server/7.3.1/Lightstreamer-7.3.1.zip
Thank you again for the report.