Order of the events at the client: onItemUpdate, onEndOfSnapshot

SlavkoParezanin

We use Vivace edition of LightStreamer.
We use our Grid (html table) as the listener.

We expect the following order of the events at the client:

1) onSubscription
2) onItemUpdate
if (command == "ADD") {
$tbody.find("tr:last").after($tr);
}
3) onEndOfSnapshot
- we set scroller
- we do another job after initial data has loaded

We have that order when we subscibe on item for the first time.

But when we subscribe to the same item again, LightStreamer returns data from Snapshot, without using Adapter, and we get the following order of the events:

1) onSubscription
2) onEndOfSnapshot
3) onItemUpdate

That way we can't do the initial job in onEndOfSnapshot event.

Another question:

When we make subsciption like this:
new Subscription("COMMAND", "Group1", cellList);
and return items:
Group1|ItemA
Group1|ItemB
Group1|ItemC

aftre that we make another subsciption:
new Subscription("COMMAND", "Group2", cellList);
and return items:
Group2|ItemX
Group2|ItemB
Group2|ItemY

When we want to push change in ItemB
What do you suggest:

a) for each Group where ItemB is involved
_listener.OnEvent(Group1, update[ItemB], false);
_listener.OnEvent(Group2, update[ItemB], false);

or

b) to force subscription at Item level, rather than Group level;
new Subscription("COMMAND", "ItemA", cellList);
new Subscription("COMMAND", "ItemB", cellList);
new Subscription("COMMAND", "ItemC", cellList);
new Subscription("COMMAND", "ItemX", cellList);
new Subscription("COMMAND", "ItemY", cellList);
?

Best regards,
Slavko Parezanin

Giuseppe Corti

Hi Slavko,

I confirm you that the expected behavior is always:

1) onSubscription
2) onItemUpdate
3) onEndOfSnapshot

in every case, both if it is the first subscription that force the Lightstreamer server to ask snapshot to the Adapter or that the server provides the snapshot from its cache.
So, the behavior that you experiment is quite strange and need to be deepened. Could you send us, if you prefer through a mail to support@lightstreamer.com, two extracts of the log of the two separate cases after setting the server log with the following levels?

<logger name="LightstreamerLogger.subscriptions" level="DEBUG"/>
<logger name="LightstreamerLogger.pump" level="DEBUG"/>
<logger name="LightstreamerLogger.preprocessor" level="DEBUG"/>

About your second question, I'm not sure I have understood correctly. However, regardless of the groups to which it belongs, on the adapter side, if you need to update the fields of an Item you just need a single update call.

Regards,
Giuseppe

SlavkoParezanin

Thanks for response Giuseppe,
We configured DEBUG as you recommended.
We are going to monitor.
It seems we had some problem with java script module pattern and subscription to LS events.

Regarding another question.
For example we want to know city population of Italy.
So we subscribe:
new Subscription("COMMAND", "Italy", cellList);
and return items:
Italy|Milano
Italy|Roma
Italy|Lanciano

We want population per region, so we subscribe:
new Subscription("COMMAND", "Abbruzzo", cellList);
and return items:
Abbruzzo|Aquila
Abbruzzo|Lanciano

When Lanciano has changes in population do we need to do:
_listener.OnEvent("Italy", update[Lanciano], false)
_listener.OnEvent("Abbruzzo", update[Lanciano], false)

or
it is better to subscribe at city level?
new Subscription("COMMAND", "Milano", cellList);
new Subscription("COMMAND", "Aquila", cellList);
new Subscription("COMMAND", "Lanciano", cellList);
new Subscription("COMMAND", "Roma", cellList);

Best regards !

Giuseppe Corti

Hi Slavko,

In your example, "Lanciano" is the same key of two different Items, "Italy" and "Abruzzo", subscribed in COMMAND mode. In this case if the value of any field of Lanciano updates you have to push the update for every Item that lists "Lanciano" in the set of own keys.

So, in your example, the two commands
[SYNTAX=CSHARP]_listener.OnEvent("Italy", update[Lanciano], false)
_listener.OnEvent("Abbruzzo", update[Lanciano], false)[/SYNTAX]
are needed.

But, in a case like this, you should consider the option to switch to the COMMAND mode with "two-level push". Please refer to the Portfolio Demo for a full example of implementation of "two-level push".
In these cases, COMMAND mode is applied to stock portfolios, rather than city population of Italy, but the principle is the same (stock name <--> city name and price <--> population count). Please note that when a stock updates its price will need to send one single update to the server Lightstreamer,
[SYNTAX=CSHARP]_listener.OnEvent("Stock_1", update[price], false)[/SYNTAX]
regardless of how many portfolios contain that stock.

SlavkoParezanin

Hi,

Regarding order of events we get:
1) onSubscription
2) onEndOfSnapshot
3) onItemUpdate (ADD)

After we return the initial set items in snapshot, we have a new Item that we want to listen.
We added it by:
_listener.Add(itemName, GetCurrentValues(true), false);

As soon as we changed it to:
_listener.Add(itemName, GetCurrentValues(true), true);

everything is OK.

Best regards,
Slavko

Giuseppe Corti

Hi Slavko,

Please can you confirm that the third parameter of this method

[SYNTAX=CSHARP]_listener.Add(itemName, GetCurrentValues(true), true);[/SYNTAX]

is the isSnapshot flag?

If this is the case, please note that it must be set to "true" only for the updates needed to realign the current state of the Item, following a Subscribe request by the Lightstreamer server, but never after the call of the EndOfSnapshot method for that Item.

Regards,
Giuseppe

SlavkoParezanin

Yes, third parameter is "isSnapshot".
Let us talk about cities in the region.

We want population of cities per region, so we subscribe:
new Subscription("COMMAND", "Abbruzzo", cellList);

At server we return the Snapshot

update.Add("command", "ADD");
_listener.OnEvent(Abbruzzo, Aquila, true);
_listener.OnEvent(Abbruzzo, Lanciano, true);
_listener.OnEndSnapshot(Abbruzzo);

and we modify the items, without any problem.
update.Add("command", "UPDATE");
_listener.OnEvent(Abbruzzo, Aquila, false);

But, when you build a new city in Abbruzzo , fer example Filetto, we did:

update.Add("command", "ADD");
_listener.OnEvent(Abbruzzo, Filetto, false);

we get the problem with order of the events at the client.

As soon as we changed it to:
update.Add("command", "ADD");
_listener.OnEvent(Abbruzzo, Filetto, true);

everything is ok.

Regards !

Giuseppe Corti

Hi Slavko,

I'm sorry but I'm a bit confused about the behavior that you expect.

Please, let me consider the sequence from the point of view of the client:
1. Subscribes to the "Abbruzzo" Item in COMMAND mode;
2. receives onSubscription event;
3. receives onItemUpdate (ADD) events for the city of Aquila and Lanciano (isSnapshot return true);
4. receives the onEndOfSnapshot event;
5. receives several onItemUpdate (UPDATE) events for the changes in population count for the city of Aquila and Lanciano (isSnapshot return false);
6. receives an onItemUpdate (ADD) event for the new city of Filetto (isSnapshot return false);
7. receives several onItemUpdate (UPDATE) events for the changes in population count for the city of Aquila, Lanciano and Filetto (isSnapshot return false);

Do you expect something different in the sequence above?

SlavkoParezanin

Yes, it is expected and it is ok.
I just wanted to show you what caused the problem of order of events at client.

- new Subscription("COMMAND", "Abbruzzo", cellList);
- we return Snapshot.
- _listener.OnEndSnapshot(Abbruzzo);
- we updates ...

we added update.Add("command", "ADD");
_listener.OnEvent(Abbruzzo, Filetto, false); // FALSE is WRONG !!!

- we refresh the browser.
- we get wrong order of events !

But after we set:
_listener.OnEvent(Abbruzzo, Filetto, true);

everything is ok !

regards!

Giuseppe Corti

Hi Slavko,

If this solution works in your application then it's fine.

Maybe I did not understand how this instruction

[SYNTAX=CSHARP]_listener.OnEvent(Abbruzzo, Filetto, true);[/SYNTAX]

exactly apply in your code.

I understood that with that statement you're pushing, from the adapter to the server, an event of command type ADD with the snapshot flag set to true; but may be that I'm wrong.
By the way, in that case you should have a warning message like this

"Unexpected snapshot event for COMMAND item ..."

in the log server.

SlavkoParezanin

I did some investigation of log, and i have no warning messages like these
"Unexpected snapshot event for COMMAND item ..."

Before i do more investigation, i would like know your suggestion for the case.

- We start the LS
- User opens the browser after a few seconds aftre, asking for Snapshot,
new Subscription("COMMAND", "Abbruzzo", cellList);

- We return 2 cities as the Snapshot.
update.Add("command", "ADD");
_listener.OnEvent(Abbruzzo, update(Aquila), true);

update.Add("command", "ADD");
_listener.OnEvent(Abbruzzo, update(Lanciano), true);

_listener.OnEndSnapshot(Abbruzzo);

- After that we get notified that a new city is built and we send:
update.Add("command", "ADD");
_listener.OnEvent(Abbruzzo, Filetto, true);

- User does browser refresh
- User have all the cities in Snapshot and right order of events.

In case of returning the new city Lanciano like this:
update.Add("command", "ADD");
_listener.OnEvent(Abbruzzo, update(Lanciano), false);
user gets wrong order of events after browser refresh.
It seems only the first time, after browser Refresh.

regards!

SlavkoParezanin

I did some investigation of log, and i have no warning messages like these
"Unexpected snapshot event for COMMAND item ..."

Before i do more investigation, i would like know your suggestion for the case.

- We start the LS
- User opens the browser after a few seconds aftre, asking for Snapshot,
new Subscription("COMMAND", "Abbruzzo", cellList);

- We return 2 cities as the Snapshot.
update.Add("command", "ADD");
_listener.OnEvent(Abbruzzo, update(Aquila), true);

update.Add("command", "ADD");
_listener.OnEvent(Abbruzzo, update(Lanciano), true);

_listener.OnEndSnapshot(Abbruzzo);

- After that we get notified that a new city is built and we send:
update.Add("command", "ADD");
_listener.OnEvent(Abbruzzo, Filetto, true);

- User does browser refresh
- User has all the cities in Snapshot and right order of events.

In case of returning the new city Lanciano like this:
_listener.OnEvent(Abbruzzo, update(Lanciano), false);
user gets wrong order of events after browser refresh.
Only for the first time, after browser Refresh.
I was able to re-produce it.

Regards!

Giuseppe Corti

Hi Slavko,

I confirm you that the isSnaphot flag on an update, passed from the adapter to the Lightstreamer server, should be set to true only in the early stages of a new subscription, and never for updates generated in real time.

So in your case, set isSnapshot to true when a new city is built is something unexpected and maybe we should investigate further the case.

Please can you repeat and isolate the case in a test environment after setting the below configurations in the lightstreamer_log_conf.xml?

[SYNTAX=XML]<logger name="LightstreamerLogger.pump" level="DEBUG"/>
<logger name="LightstreamerLogger.preprocessor" level="DEBUG"/>
<logger name="LightstreamerLogger.subscriptions" level="DEBUG"/>[/SYNTAX]

If you manage to get the data, please send them to us by mail (support@lightstreamer.com).

SlavkoParezanin

Hi Giuseppe,
I managed to re-produce the problem.
I have :

1) Adapter
itemName = "ALL"

2) Listener

3) SubItems (empty at the begining, we don't know when City will appear)
ALL/Lanciano
ALL/Aquila

- started the LS server and launchers
- started the browser as faster as i can
- clicka and ask for snapshot before any of cities has appeared
- when i use listener.Add("ALL", GetCurrentValues(true), false);
i get wrong order of events
- and it can't be repaired even if i reopen the browser

but if i use
_trueMarketListener.Add("ALL", GetCurrentValues(true), true);

i have no that problem !

It can be LS bug or i haven't found the best way to ensure that ADD precedes Updates for each subitems, ... subitems do the job even all the user unsubscribed "ALL", resubscribe "ALL", and so on ...

regards!

Giuseppe Corti

Hi Slavko,

I do not think that you have found a bug in Lightstreamer, but maybe I understand the cause of the strange behavior that you observe.

When an Item is subscribed for the first time the adapter should send one or more update with isSnapshot flag set to true and then, optionally, the endOfSnapshot.
But if you send the first update for an Item with isSnapshot flag set to false then you will get the wrong sequence of events because the server infers that for that Item no Snapshot exists. Please refer to the documentation of endOfSnapshot method, I assume you are using Java Adapter API but the concept is the same also in C#.

So if, in your previous posts, you were referring to the first update sent for an Item then it is absolutely correct to have the isSnapshot flag set to true.

Sorry for the misunderstanding.

SlavkoParezanin

Yes that is the case.
Thank you for the support !
Regards !