.NET with external feed

markgoldin

What I am trying to do with this example is to modify it to the point that the code can listen to some port on the same computer and an external process will be writing to that port to provide data. AS the last, the LS will deliver data to my front-end. Can you please give me a few .net tips for such solution if it is all even possible?

Thanks a lot.

Dario Crivelli

Getting data from an external process is the usual scenario.
However, you have to obey the Data Adapter interface, based on "subscribe" and "unsubscribe" to data "items". Hence, depending on the interface supplied by your external process, your Data Adapter may have more or less work to do.
For a HelloWorld-like demo, you can read from your external process and forward each line through an "update" call for the "greetings" item.
We have no .net specific tips, other than the remarks provided in the API documentation, available at http://www.lightstreamer.com/docs/adapter_dotnet_chm/DotNetAdapterAPI.chm

markgoldin

Yes, I have gotten my modified HelloWord sample working. But what I need in reality is the following:
I am going to have let's say 10 HelloWord like programs that are listening to 10 external processes to get data. Then I need LS to push data being collected from 10 adapters to the same html page. In order to achive that what I do? Just
describing all 10 adapters in my ProxyHelloWorld\adapters.xml file and that's all?

Thanks for all your hlep.

Dario Crivelli

Defining multiple Data Adapters and sending all their data to a single client page is not possible at the moment, though it will become possible with the next release of the software.
However, you can connect to all your external processes from a single Data Adapter and assign to each flow a different item name. Your client would be able to ask for all data by specifying all the item names, in a way similar to the StockList demo example provided in Lightstreamer package.

markgoldin

Alright, I uderstand.
I want to show a code fragment from my modified Hello World:
<script>
/////////////////PushPage Configuration
var debugAlerts = true;
var remoteAlerts = false;

var page = new PushPage();
page.context.setDomain("ufandd.local");
page.onEngineCreation = function(lsEngine) {
lsEngine.connection.setLSHost("ufdcrdev0005.ufandd.local");
lsEngine.connection.setLSPort(8080);
lsEngine.context.setDebugAlertsOnClientError(debugAlerts);
lsEngine.context.setRemoteAlertsOnClientError(remoteAlerts);

lsEngine.connection.setAdapterName("PROXY_HELLOWORLD");
lsEngine.changeStatus("STREAMING");
}
page.bind();
page.createEngine("HelloWorldApp", "LS", "SHARE_SESSION");

var pushtable = new OverwriteTable(null, null, "MERGE");
page.addTable(pushtable, "hellotable");
pushtable.onItemUpdate =
function(itemPos, updateInfo, itemName)
{
// send completed scan to the front-end
alert(updateInfo.getNewValue(itemPos));
serverData(updateInfo.getNewValue(itemPos));
};
</script>
As you can see I am overriding onItemUpdate because I also want to send data to somewherre else. It works fine, sort of. For the very first time alert box will pop up only once (as expected), but next data update will trigger the alert box twice: first with an empty value, second with the real value.

Can you tell me what am I doing wrong here?

BTW, my adapter's console shows one value every time.

Dario Crivelli

The snippet does not fully describe the test, because the

new OverwriteTable(null, null, "MERGE");

instruction asks the library to find all items and fields defined in the push cells, which are in the HTML part of the page.

To get a better idea of the update flow, please show us a Server log taken after modifying the log configuration file, by setting the priority of the "LightstreamerLogger.subscriptions" and "LightstreamerLogger.pump" categories to DEBUG.

markgoldin

While I am working on that I have another question:
Is it possible to hide these DIVs so they will not take any space on a page:
<div source="lightstreamer" table="hellotable" item="greetings" field="message">loading...</div>
<div source="lightstreamer" table="hellotable" item="greetings" field="timestamp">loading...</div>

Thanks

Mone

yes of course, you can set to their css the display:none value and then in the onChangingValues callback add such style to their hot and cold statuses:

[syntax="JS"]
setAttribute("message", "none", "none", "display");
setAttribute("timestamp", "none", "none", "display");
[/syntax]

Btw note that if those are the only fields in your table (ie you don't want that LS shows any cells) you can use a NonVisualTableinstead of the OverwriteTable :

[syntax="JS"]
var pushtable = new NonVisualTable(["greetings"], ["message","timestamp"], "MERGE");
[/syntax]

HTH.

markgoldin

Here is my latest code. I understand that you are not supporting .Net but still maybe you can give me a hint:

using System.Collections;
using System.Threading;
using System;
using System.Runtime.InteropServices;
using Lightstreamer.Interfaces.Data;
using System.Windows.Forms;

public class SocketToLightStreamer : IDataProvider
{
private IItemEventListener _listener;
public void Init(IDictionary parameters, string configFile)
{
}

public bool IsSnapshotAvailable(string itemName)
{
return false;
}

public void SetListener(IItemEventListener eventListener)
{
_listener = eventListener;
}
public void Subscribe(string itemName)
{
if (itemName.Equals("floorupdate"))
{
}
}

public void Unsubscribe(string itemName)
{
if (itemName.Equals("floorupdate"))
{
}
}
public void PushData(string data)
{
IDictionary eventData = new Hashtable();
eventData["scan"] = data;
_listener.Update("floorupdate", eventData, false);
}
public class GetMessage : Form
{
const int WM_COPYDATA = 0x004a;

public GetMessage()
{
Text = "my_unique_id";
}

protected override void WndProc(ref System.Windows.Forms.Message m)
{
switch (m.Msg)
{
case WM_COPYDATA:
COPYDATASTRUCT mystr = new COPYDATASTRUCT();
Type mytype = mystr.GetType();
mystr = (COPYDATASTRUCT)m.GetLParam(mytype);
IDictionary eventData = new Hashtable();
eventData["scan"] = mystr.Data;
SocketToLightStreamer a = new SocketToLightStreamer();
a._listener.Update("floorupdate", eventData, false);
break;
}
base.WndProc(ref m);
}
}
[StructLayout(LayoutKind.Sequential)]
public struct COPYDATASTRUCT
{
public Int32 ID;
public int Length;
public string Data;
}
}
While this code is compiling fine when I run it I get:
"null reference" at a._listener.Update("floorupdate", eventData, false);
Wold you suggest to how to have a form inside of SocketToLightStreamer class in a way that I could access _listener object from it?

Thanks for the hep.

Dario Crivelli

The SocketToLightStreamer class should not be instantiated directly.
Exactly one object of this class is internally created by the Lightstreamer Remote Adapter Library.
If you get data from outside code, as your GetMessage class seems to be, you should be able to access that object.

Usually, the Data Adapter class starts the feed in "Init" and asks it for data in "Subscribe". There, it can send its own reference to the feed.
This is not your case, as you feed data regardless of the subscription requests
(note that any updates you send while no subscription is active will be lost).

What you could do, for instance, is have your SocketToLightStreamer object assign itself to a static pointer that can be eventually consulted in GetMessage.
Note that the best place to assign the pointer is after setting the listener in "SetListener".
Also note that synchronization issues may need to be considered.

Alessandro Alinone

Hi Mark,

Please let me clarify that we do support .NET. In particular, Lightstreamer Moderato (the free edition) includes the SDK for .NET Adapter, while Lightstreamer Allegro/Presto/Vivace (the commercial editions) include the SDK for .NET clients too. All the SDKs we provide are supported.

Cheers
Alessandro

markgoldin

<Also note that synchronization issues may need to be considered.
And that I think I am having a problem with. I am running a process that stupidely sends current time with an interval of one second to push to the cient. I see that sometimes (not that bad though) I am missing some data. What would you suggest?

Thanks

Dario Crivelli

I warned about synchronization issues with relation to my suggestion of putting a pointer to the Data Adapter on a static variable. I did it just in case.
If you, after revising your code, can now see that some updates do reach the client, I cant' figure how synchronization problems could cause single updates to be lost.

It is possible that some events are merged by the Server, if bandwidth or frequency restrictions are configured.
You can check the event flow through the Server in the Server log, after setting the priority for the "LightstreamerLogger.subscriptions" and "LightstreamerLogger.pump" categories to DEBUG in "lightstreamer_log_conf.xml".

markgoldin

Here is my latest code with which I am missing some data sometimes:
using System.Collections;
using System.Threading;
using System;
using System.Runtime.InteropServices;
using Lightstreamer.Interfaces.Data;
using System.Windows.Forms;

public class SocketToLightStreamer : IDataProvider
{
private IItemEventListener _listener;
public void Init(IDictionary parameters, string configFile)
{
}

public bool IsSnapshotAvailable(string itemName)
{
return false;
}

public void SetListener(IItemEventListener eventListener)
{
_listener = eventListener;
}
public void Subscribe(string itemName)
{
if (itemName.Equals("floorupdate"))
{
Form f = new GetMessage(this);
f.Show();
f.Hide();
ApplicationContext ctx = new ApplicationContext();
Application.Run(ctx);
}
}

public void Unsubscribe(string itemName)
{
if (itemName.Equals("floorupdate"))
{
}
}
public void PushData(string data)
{
System.Console.WriteLine(data);
IDictionary eventData = new Hashtable();
eventData["scan"] = data;
_listener.Update("floorupdate", eventData, false);
}
public class GetMessage : Form
{
const int WM_COPYDATA = 0x004a;
private SocketToLightStreamer clientUpdater;
public GetMessage(SocketToLightStreamer o)
{
Text = "my_unique_id";
clientUpdater = o;
}

protected override void WndProc(ref System.Windows.Forms.Message m)
{
switch (m.Msg)
{
case WM_COPYDATA:
COPYDATASTRUCT mystr = new COPYDATASTRUCT();
Type mytype = mystr.GetType();
mystr = (COPYDATASTRUCT)m.GetLParam(mytype);
new Thread(delegate() { clientUpdater.PushData(mystr.Data); }).Start();
break;
}
base.WndProc(ref m);
}
}
[StructLayout(LayoutKind.Sequential)]
public struct COPYDATASTRUCT
{
public Int32 ID;
public int Length;
public string Data;
}
}

I see missing data in the Console written in PushData method.

Here is sample of data being lost:
22-Aug-08 08:19:41,266 |TRACE|LightstreamerLogger.pump |PUMP POOLED THREAD 1 |Pumping event in session Sa4839d7557b06b92T1742762: d(1,1,1,"08:19:40^text2\u000D");
22-Aug-08 08:19:42,266 |TRACE|LightstreamerLogger.subscriptions|#1 Notify Receiver |INCOMING DATA for floorupdate --> {scan=08:19:41^text1
}
22-Aug-08 08:19:42,266 |DEBUG|LightstreamerLogger.subscriptions|#1 Notify Receiver |Manager: com.lightstreamer.e.s@8046f4
22-Aug-08 08:19:42,266 |TRACE|LightstreamerLogger.pump |PUMP POOLED THREAD 1 |Pumping event in session Sa4839d7557b06b92T1742762: d(1,1,1,"08:19:41^text1\u000D");
22-Aug-08 08:19:43,376 |TRACE|LightstreamerLogger.subscriptions|#1 Notify Receiver |INCOMING DATA for floorupdate --> {scan=08:19:43^text2
}

Dario Crivelli

If I understand well, you are complaining about the absence of an event for 08:19:42.
The log shows that such event is missing in the flow from the Data Adapter, hence it is not filtered because of bandwidth or frequency restrictions.
However, I cannot find any evidence in the Data Adapter code that an event for 08:19:42 was supposed to be generated, as it seems that part of the event production process takes place externally. Moreover, I notice that the last "INCOMING DATA" event in the log file happens significantly more than one second after the preceding one.

So, the Remote Data Adapter part is now to be logged.
If you find it useful, you can enable logging by Lightstreamer libraries on the Remote Server, by configuring a <log4net> session in a way similar to the

DOCS-SDKs\sdk_adapter_dotnet\examples\DotNetStockListDemo\Deployment\Deployment_DotNet_Server\dotnet_1.1\DotNetServer.exe.config

file. Just set DEBUG level for the Lightstreamer.DotNet.Server.RequestReply logger to have the Update calls logged.

Just in case, note that the field values of your events (like "08:19:41^text1\u000D", the trailing being a return character) seem to lack some processing.

markgoldin

I am going to follow your recomdations but I think the reason for missing data is that sometimes data push by the LS takes just a moment longer (means resources are still busy) and my data adapter does not have a chance to process, hence it does not provide avery data for to LS for data push. That was the reason to have actual code for data push into another thread. It seamed helped but not for 100%.
What do you think about that?
BTW, My data is just a test data, but missings also happen with the real data that does not have a "^" symbol.

Thanks for your help.

Dario Crivelli

I see you avoid to invoke _listener.Update from a critical thread (as the one running WndProc seems to be).
However, the Update call implementation also enqueues the values, so that they are processed by another thread, provided by Lightstreamer library.
Hence, we don't advise you against calling Update from a critical thread, as we don't expect Update to be blocking.
Moreover, with so few updates, CPU overload problems are also not possible.

So, I can't understand where your Data Adapter process may be blocked.
Can you get any evidence that Lightstreamer is involved? If you comment out the Update call and just log the updates, can you see a different behaviour?

markgoldin

Yes, I did comment out the update call. I still had lost data, I would say less then with updates but still there was lost data. Knowing my code would you recommend anything to preotect from losing data?

Thanks

Dario Crivelli

I can't say I know your code, as the relevant part of the code (that is, where the WndProc is called) is not reported
and I guess it is GUI related code, which I'm not an expert of.

From Lightstreamer point of view, I think that there is no issue
and that, perhaps, the expectation that your feed should produce exactly one event for each second is wrong.
By looking at your log of the feed events, which time sequence can you find?
Are the events equally spaced at 1 second, with holes of 2 seconds when events are lost?
Or are the events equally spaced at slightly more than 1 second, so that sometimes the rounded event time increases by 2 seconds?

markgoldin

<Are the events equally spaced at 1 second, with holes of 2 seconds when events are lost?
Yes, that what's happening. It goes for 20-30 events fine, then I see one event was lost, then again no lost events for maybe a half a minute then a lost one.