Unexpected sequence of events on data adapter

kpturner

I have a sequence of events that I didn't expect. It may be normal, but I wanted to checked.

My client subscribes to a feed, and when the onSubscription() event fires on my client, I am doing a sendMessage().
The method that deals with the incoming message in my data adapter is also called "sendMessage".

On the server side, I am finding that the message is arriving in my data adapter BEFORE the "subscribe" method has completed. This causes some issues because the processing on the "sendMessage" method in my data adapter relies on an object that gets created in the "subscribe" method - but I am finding that it is often not created at that time. I have included a sequence of debug messages and it confirms the sequence of events. At the moment I have worked around it by including a very short sleep() loop in my "sendMessage" that waits until the "subscribe" method has done its thing.

In case it is relevant, the subscribe method is creating a thread and starting it(a bit like the hello world example), and the sendMessage is relying on that thread having been started. But in my sendMessage method the thread object is often still null - hence the little wait.

So it works, but I don't like it.

Am I wrong in my assumption that the client is safe to sendMessage() once the onSubscription has fired? It seems not. So when is it safe? I basically do not want my client to be able to sendMessage until the subscribe method in my data adapter has finished.

Dario Crivelli

I confirm that the onSubscription() event doesn't have the meaning you expected.
I appreciate that what reported in the documentation

called by Lightstreamer to notify that a Subscription has been successfully subscribed to through the Server

may be misleading.
Actually, the outcome of the subscription as performed by the Data Adapter (including any SubscriptionException raised) is not forwarded to the client;
so, the onSubscription()event is triggered when the Server has received the subscription request, before it invokes (if needed) subscribe() on the Data Adapter.
The goal of onSubscription is mainly that of notifying the listener when a resubscription of the item has been made under the hood, due to a reconnection for recovery purpose, because at that point the update flow may have a discontinuity.

On the other hand, the updates (including the snapshot) are guaranteed to be delivered only after subscribe() has terminated.
So, you could leverage that property in some way.

kpturner

OK thanks for that - so at what point on my client am I safe to issue the sendMessage()? During the early part of my testing, if I did a sendMessage() immediately after I have done a subscribe(), the message did not arrive in my metadataadapter:

http://forums.lightstreamer.com/showthread.php?1831-My-notifyUserMessage-method-is-not-getting-called

that aside, it is very important for me to know that the "subscribe" method on the server has completed before I send any messages. Are you saying I should do some sort of dummy snapshot before I run sendMessage()?

Or, perhaps, just do an smartupdate with a special value that tell the client that the subscribe is complete perhaps?

kpturner

I have done a smartupdate with a special value to indicate to the client that the subscription is complete. It seems to work now, but I guess I have added a small overhead to my initialisation process now.

Dario Crivelli

Ok.
By the way, I think it's worth reminding that there isn't a 1:1 relationship between the client subscription and the invocation of subscribe() on the Data Adapter, though this may be the case for your application, because of your use of session-specific items.

kpturner

Dario Crivelli Ok.
By the way, I think it's worth reminding that there isn't a 1:1 relationship between the client subscription and the invocation of subscribe() on the Data Adapter, though this may be the case for your application, because of your use of session-specific items.

Yes OK - thanks.

kpturner

It does raise another question though. I am also using a broadcast technique where multiple clients can subscribe to the same feed. When one sends a message, everyone gets it apart from the sender (I filter it out using isSelected). The subscribe() only fires when the first subscription comes in, and so my smartupdate to say that the subscription is OK only happens once. Does this mean that all the other subsequent subscribers will, or will not, get the "subscribed ok" update? I am using a RAW feed.

EDIT: I should have just tested it - and of course the answer is "no, the subsequent subscribers will not get the 'subscribed OK' update".

So I still have the question about when it is safe to do a sendMessage() in this scenario. Am I worrying about nothing? My original issue (which I linked to above) seemed to indicate that doing the sendMessage too soon meant that it did not arrive in my metadataadapter. Was I barking up the wrong tree there or do I still have a small issue to resolve?

Dario Crivelli

At the moment, the only general solution seems to be to put the due synchronization on the Adapter side.
Note that when you get onSubscription()on the client side, you are guaranteed that subscribe() , if not invoked yet, is about to be invoked on the Data Adapter.
Hence, the sendMessage management could leave the message suspended and the subsequent subscribe() would resume the message.
But probably a few cases of leaked messages would still be possible.

The simplest workaround to that is leaning on the snapshot.
If the Data Adapter provides an initial snapshot for an item:
1) the snapshot will be always sent to all subscribers;
2) in no case will the snapshot be sent before the completion of the initial subscribe().
Note that the snapshot sent may not be the same for all subscribers, because the snapshot is maintained by the Server based on the updates.

As you are using an item in RAW mode, you could change it into an item in DISTINCT mode with unfiltered dispatching enabled.
This has the same characteristics of RAW mode, but adds a snapshot made of the recent update history.
Then you could limit the history to a single update, by leveraging getDistinctSnapshotLength() on the Metadata Adapter.
This single-update snapshot will be the signal for the client, then it will be just discarded.
As said, the subscribe() method should also supply a single-update snapshot, to the benefit of the first subscribers.

Obviously, the above assumes some confidence in snapshot management; we can expand, if needed.

kpturner

Thanks for the update. I am a noob so I have very little experience of snapshot maintenance at the moment (apart from what I have seen in the chat demo). I guess I need to read up a little.

The problem I had with DISTINCT was that when I created a system wide broadcast, any new clients of the broadcast got all the previous broadcasts. Quite a useful feature, but one I specifically did not require here. Does unfiltered dispatching stop that? If so, how do I implement it?

I need to read up on getDistinctSnapshotLength() also it seems.

Giuseppe Corti

In DISTINCT mode the lenght of snapshot can be configured both server side and client side.

In your MetaData Adapter by the getDistinctSnapshotLength method you can specify the maximum lenght (in number of updates) of the snapshot for each item.

Client side, the setRequestedSnapshot method allow you to specify the requested length of the snapshot that every client wants to receive (you can also opt to not request snapshot delivery at all).
Please note that listening on onEndOfSnapshot event handler allow your client to be notified that all snapshot events for an item in the Subscription have been received.

kpturner

Great - thanks. Could yo also elaborate on

DISTINCT mode with unfiltered dispatching enabled

I can see "unfiltered dispatching" mentioned in a few places in the docs but I cannot find anything that explains exactly what it is and how it is implemented.

Giuseppe Corti

Generally we refer to "unfiltered dispatching" when no limits on the frequency of updates is applied to an Item.

As well as the snapshot settings, the frequency settings can be configured both server-side and client-side.
On the server side, by the function getAllowedMaxItemFrequency, you can set an upper limit on the frequency for each Item.
On the clients side, by the function setRequestedMaxFrequency, you can choose for each of them its proper frequency limit, even dynamically during the life of the subscription.

Please refer of section "3.3 The itemEventBuffer" of "General Concepts.pdf" document for further details.

kpturner

Seems to work very well thanks. I have switched the broadcast to DISTINCT and setRequestedSnapshot(1). The client is unable to broadcast until onEndOfSnapshot() is fired. The single snapshot update (if received) is ignored.

Thanks for your help.

FWIW this is just a client broadcasting the position of a slider, and other subscribers to the broadcast will see their slider move in unison (and vice-versa).

Giuseppe Corti

Fine!

Have you set "unfiltered" in your DISTINCT subscription in order to be more closely to the RAW mode used before?

However, if the data transmitted is the position of the slider then I do not see a strong need for move to the "unfiltered dispatching".

kpturner

Yes I did this:

blah.setRequestedMaxFrequency("unfiltered");