.NET with external feed

markgoldin

<Also note that synchronization issues may need to be considered.
And that I think I am having a problem with. I am running a process that stupidely sends current time with an interval of one second to push to the cient. I see that sometimes (not that bad though) I am missing some data. What would you suggest?

Thanks

Dario Crivelli

I warned about synchronization issues with relation to my suggestion of putting a pointer to the Data Adapter on a static variable. I did it just in case.
If you, after revising your code, can now see that some updates do reach the client, I cant' figure how synchronization problems could cause single updates to be lost.

It is possible that some events are merged by the Server, if bandwidth or frequency restrictions are configured.
You can check the event flow through the Server in the Server log, after setting the priority for the "LightstreamerLogger.subscriptions" and "LightstreamerLogger.pump" categories to DEBUG in "lightstreamer_log_conf.xml".

markgoldin

Here is my latest code with which I am missing some data sometimes:
using System.Collections;
using System.Threading;
using System;
using System.Runtime.InteropServices;
using Lightstreamer.Interfaces.Data;
using System.Windows.Forms;

public class SocketToLightStreamer : IDataProvider
{
private IItemEventListener _listener;
public void Init(IDictionary parameters, string configFile)
{
}

public bool IsSnapshotAvailable(string itemName)
{
return false;
}

public void SetListener(IItemEventListener eventListener)
{
_listener = eventListener;
}
public void Subscribe(string itemName)
{
if (itemName.Equals("floorupdate"))
{
Form f = new GetMessage(this);
f.Show();
f.Hide();
ApplicationContext ctx = new ApplicationContext();
Application.Run(ctx);
}
}

public void Unsubscribe(string itemName)
{
if (itemName.Equals("floorupdate"))
{
}
}
public void PushData(string data)
{
System.Console.WriteLine(data);
IDictionary eventData = new Hashtable();
eventData["scan"] = data;
_listener.Update("floorupdate", eventData, false);
}
public class GetMessage : Form
{
const int WM_COPYDATA = 0x004a;
private SocketToLightStreamer clientUpdater;
public GetMessage(SocketToLightStreamer o)
{
Text = "my_unique_id";
clientUpdater = o;
}

protected override void WndProc(ref System.Windows.Forms.Message m)
{
switch (m.Msg)
{
case WM_COPYDATA:
COPYDATASTRUCT mystr = new COPYDATASTRUCT();
Type mytype = mystr.GetType();
mystr = (COPYDATASTRUCT)m.GetLParam(mytype);
new Thread(delegate() { clientUpdater.PushData(mystr.Data); }).Start();
break;
}
base.WndProc(ref m);
}
}
[StructLayout(LayoutKind.Sequential)]
public struct COPYDATASTRUCT
{
public Int32 ID;
public int Length;
public string Data;
}
}

I see missing data in the Console written in PushData method.

Here is sample of data being lost:
22-Aug-08 08:19:41,266 |TRACE|LightstreamerLogger.pump |PUMP POOLED THREAD 1 |Pumping event in session Sa4839d7557b06b92T1742762: d(1,1,1,"08:19:40^text2\u000D");
22-Aug-08 08:19:42,266 |TRACE|LightstreamerLogger.subscriptions|#1 Notify Receiver |INCOMING DATA for floorupdate --> {scan=08:19:41^text1
}
22-Aug-08 08:19:42,266 |DEBUG|LightstreamerLogger.subscriptions|#1 Notify Receiver |Manager: com.lightstreamer.e.s@8046f4
22-Aug-08 08:19:42,266 |TRACE|LightstreamerLogger.pump |PUMP POOLED THREAD 1 |Pumping event in session Sa4839d7557b06b92T1742762: d(1,1,1,"08:19:41^text1\u000D");
22-Aug-08 08:19:43,376 |TRACE|LightstreamerLogger.subscriptions|#1 Notify Receiver |INCOMING DATA for floorupdate --> {scan=08:19:43^text2
}

Dario Crivelli

If I understand well, you are complaining about the absence of an event for 08:19:42.
The log shows that such event is missing in the flow from the Data Adapter, hence it is not filtered because of bandwidth or frequency restrictions.
However, I cannot find any evidence in the Data Adapter code that an event for 08:19:42 was supposed to be generated, as it seems that part of the event production process takes place externally. Moreover, I notice that the last "INCOMING DATA" event in the log file happens significantly more than one second after the preceding one.

So, the Remote Data Adapter part is now to be logged.
If you find it useful, you can enable logging by Lightstreamer libraries on the Remote Server, by configuring a <log4net> session in a way similar to the

DOCS-SDKs\sdk_adapter_dotnet\examples\DotNetStockListDemo\Deployment\Deployment_DotNet_Server\dotnet_1.1\DotNetServer.exe.config

file. Just set DEBUG level for the Lightstreamer.DotNet.Server.RequestReply logger to have the Update calls logged.

Just in case, note that the field values of your events (like "08:19:41^text1\u000D", the trailing being a return character) seem to lack some processing.

markgoldin

I am going to follow your recomdations but I think the reason for missing data is that sometimes data push by the LS takes just a moment longer (means resources are still busy) and my data adapter does not have a chance to process, hence it does not provide avery data for to LS for data push. That was the reason to have actual code for data push into another thread. It seamed helped but not for 100%.
What do you think about that?
BTW, My data is just a test data, but missings also happen with the real data that does not have a "^" symbol.

Thanks for your help.

Dario Crivelli

I see you avoid to invoke _listener.Update from a critical thread (as the one running WndProc seems to be).
However, the Update call implementation also enqueues the values, so that they are processed by another thread, provided by Lightstreamer library.
Hence, we don't advise you against calling Update from a critical thread, as we don't expect Update to be blocking.
Moreover, with so few updates, CPU overload problems are also not possible.

So, I can't understand where your Data Adapter process may be blocked.
Can you get any evidence that Lightstreamer is involved? If you comment out the Update call and just log the updates, can you see a different behaviour?

markgoldin

Yes, I did comment out the update call. I still had lost data, I would say less then with updates but still there was lost data. Knowing my code would you recommend anything to preotect from losing data?

Thanks

Dario Crivelli

I can't say I know your code, as the relevant part of the code (that is, where the WndProc is called) is not reported
and I guess it is GUI related code, which I'm not an expert of.

From Lightstreamer point of view, I think that there is no issue
and that, perhaps, the expectation that your feed should produce exactly one event for each second is wrong.
By looking at your log of the feed events, which time sequence can you find?
Are the events equally spaced at 1 second, with holes of 2 seconds when events are lost?
Or are the events equally spaced at slightly more than 1 second, so that sometimes the rounded event time increases by 2 seconds?

markgoldin

<Are the events equally spaced at 1 second, with holes of 2 seconds when events are lost?
Yes, that what's happening. It goes for 20-30 events fine, then I see one event was lost, then again no lost events for maybe a half a minute then a lost one.

Dario Crivelli

Sorry to insist on this point, but I'm still not sure that there is no misunderstanding (maybe because of my foreign english).
If you log your events at the entrance of your "PushData" function
and provide each log line with a timestamp with a precision up to the milliseconds,
can you see a pattern like this

08:19:42,500 <event for 8:19:42>
08:19:43,500 <event for 8:19:43>
08:19:45,500 <event for 8:19:45>

or like this

08:19:42,850 <event for 8:19:42>
08:19:43,950 <event for 8:19:43>
08:19:45,050 <event for 8:19:45>

? Note that in both cases, the event for 8:19:44 is missing, but in the latter case it is just because of an event generation rate which is slightly less than one event per second.

markgoldin

Just wanna tell you that you were right, it was my mistake: My testing data generator is not in fact sending data every second. Unless something else, my solution is ready to go. Thank you so much for all the help and support all of you have been providing.

« Previous Page