Basic help!

nimbusgb

Just getting to grips with LS so am 'climbing the mountain' ( Never dreamed I'd be learning new stuff at 60! Should be on a flippin' beach somewhere with a MaiTai ! )

I've got a basic set-up running. Data is subscribed from an MQTT server by a 'feed processor' which does a bunch of stuff with the data, stores it on a database, monitors etc etc

My feed adaptor then subscribes to additional 'Ui topics' triggered by the back end. Each time an event with Ui significance occurs the LS apaptor identifies the endpojnt ( by its MAC address passed in the MQTT message )

Lets say I'm monitoring the status of a bunch of switches ( several hundred ), so I have switch MAC address ( a 16 character string ) and a status, on/off which is what I want to display. A simple table with MAC address as the identifier, row identifier, index, whatever you want to call it. The data associated with each update is of course the switch status passed from the live status and say the switch location retrieved from a database. A Simple UI!

My current Adaptor, cobbled together from the samples and Documentation feeds single updates to the front end ( JavaScript ) and updates the UI...... but only a single line ever gets displayed.

The bit I am missing and failing to understand is how I map the MAC address to a row number in the table.

AIUI the UI page has no knowledge of MAC addresses when the first connection is made, it just knows it's going to be listening to a bunch of status messages. ( I have a suspicion that the Meta adaptor has a lot to do with this but the docs are not really clear on explaining this bit beyond - here's a sample go to it! )

Some of the Adaptor examples use a single subscription per row in the table, I'm failing to bridge the gap as to how I pass the details of the row ID's to the UI so that subscriptions can be formed and, as the Adaptor instance needs a connection to a database ( to get the switch location data ) I don't want to have several thousand subscriptions all maintaining connections to both MQTT and JDBC!

What I think I need is:
A web page session is started, connection made to 'SWITCH STATUS' adaptor

SWITCH STATUS then feeds a snapshot of ALL possible switch ids to a proxy type object ( smartupdate? Dynagrid? )

UI then displays the data ( first 20? sorted? paged sets? )

The adaptor feeds all switch updates to the websocket with lightstreamer and the UI page working together to only display the current ones on screen.

Would appreciate some help for an old dog enjoying learning new tricks!

Giuseppe Corti

Hi nimbusgb,

I think that in your case you should leverage the COMMAND subscription mode; indeed it was specifically designed to manage dynamic lists or tables.
Please note that the COMMAND mode requires the management of two mandatory fields, "key" and "command", indicate how progressively modifty the table.
In your case the "key" should be valued withe the MAC address since it serves as an index for the table.
The "command" field should be “ADD”, “UPDATE” or “DELETE”. In your specific case you only need to add a custom field, let's call it "status", with the value "on/off".
Independently if you are reading from the db or receiving a real-time mqtt message, each update will be sent on the same Item, but indicating in the "key" field the MAC address, in the "status" field the actual status of the switch and in the "command" field the value "UPDATE "(or" ADD "if the MAC address is new).

Please also check out this demo with sample source code for both Javascript client and adapter-side managing of the COMMAND mode:

- https://demos.lightstreamer.com/PortfolioDemo_Basic/
- https://github.com/Weswit/Lightstreamer-example-Portfolio-client-javascript#basic-portfolio-demo---html-client

Please let us know if you need any further clarifications.

Regards,
Giuseppe

Gianluca Finocchiaro

Hi nimbusgb,

also consider that your use case looks good for our new brand product, MQTT.Cool, which brings the power of Lightstreamer's engine to the web. Basically, it is a web gateway that you put in front of any existing MQTT broker to boost its native feature in terms of scalability, architecture, and performances.

Keeping focused on your specific case, by using MQTT.Cool you are no longer required to write your own Adapter Set to manage mqtt connections and subscriptions to an external broker, it is actually the MQTT.Cool's job. Just use our Eclipse Paho like JavaScript library (available for the weband node.js runtime), and develop your UI logic by making direct connections and subscriptions to your broker.

You can even deploy the product in production for free.

Have a look at the Getting Started Guide and get back to us for your feedback and further information.

Here some references:

The MQTT.Cool site: https://mqtt.cool

The Download page: https://mqtt.cool/download/server-latest/

The Documentation page: https://mqtt.cool/docs/

The Blog: https://mqtt.cool/blog/

nimbusgb

I think I'm getting to grips with things but there are some pretty hacky solutions in there at the moment.

Currently I am publishing snapshots to the client at first subscription by tracking the key and the handle passed. Following that the updates, triggered by an incoming MQTT alert are filtered and fed back to the UI as updates rather than snapshots.

All was partially working ( one field published but others not ) until I realised I had a threading issue with my database connections. I'm implemented connection pooling but now I have a conflict on slf4j

Error reading logging configuration file conf\.\lightstreamer_log_conf.xml:
java.lang.Exception: Logback library unreachable: it might be hidden by another slf4j implementation

Which I suspect is because hikariCP implements slf4j ...... trying to resolve that.

I did give MQTT-Cool a quick look and will possibly investigate further but for the moment I want to sort out this solution.

Giuseppe Corti

Hi nimbusgb,

I don't know if you've already solved the issue wiht slf4j libs, but please let me suggest a few pointers.

Could you check the <classloader> parameter in your adapters.xml conf file. Have you changed the value for that?
Have you add the hikariCP libraries in the general classpath? Maybe added in the /lib folder of the LS_HOME?

Regards,
Giuseppe

nimbusgb

Thanks will try that but to solve it I switched to a handcrafted datasource pool using dbcp which just dropped the logging. Once I've got my adaptor running I will have another look at hikari config.

i now have initial snapshots being returned to the LS server following the inital subscribe and then ongoing itemevents but I only get the first field displayed in my table and after the snapshot LS is not feeding updates to the UI,

Oddly enough doesn't matter where that field is ( tagged as 'name' ) in the Map of fields it is always displayed while everything else is ignored ....

Giuseppe Corti

Hi nimbusgb,

Please check if there is always consistency in the field names between the client subscription and the Data Adapter updates pushed.
A possible disturbance in this sense could come from the implementation of the getSchema method of the Metadata Adapter, can you confirm whether you have customized this method?
In the client how you specify the list of fields for the subscribe request? Directly with an array of strings, or using the extractFieldList utility of our widgets?

Instead, with regards to subsequent updates not dispatched to the client, could be useful to analyze the server log after changing these logger levels (lightstreamer_log_conf.xml):
[SYNTAX]

  <logger name="LightstreamerLogger.subscriptions" level="DEBUG"/>
  <logger name="LightstreamerLogger.subscriptions.upd" level="DEBUG"/>
  <logger name="LightstreamerLogger.preprocessor" level="DEBUG"/>

[/SYNTAX]

Regards,
Giuseppe

nimbusgb

Giuseppe Corti Hi nimbusgb,

Please check if there is always consistency in the field names between the client subscription and the Data Adapter updates pushed.
A possible disturbance in this sense could come from the implementation of the getSchema method of the Metadata Adapter, can you confirm whether you have customized this method?
In the client how you specify the list of fields for the subscribe request? Directly with an array of strings, or using the extractFieldList utility of our widgets?

Instead, with regards to subsequent updates not dispatched to the client, could be useful to analyze the server log after changing these logger levels (lightstreamer_log_conf.xml):
[SYNTAX]
  <logger name="LightstreamerLogger.subscriptions" level="DEBUG"/>
  <logger name="LightstreamerLogger.subscriptions.upd" level="DEBUG"/>
  <logger name="LightstreamerLogger.preprocessor" level="DEBUG"/>
[/SYNTAX]

Regards,
Giuseppe

Yes I'm extracting fields with the widget and these are matched with the getschema in case etc. I think I'm OK for now as I have a working system.