Problems with ValueItemReader and SubscriptionReader

sesa197614_brid · ‎2016-09-21

I have test some samples successfully to use ValueItemReader ,ValueItemWriter, AlarmItemReader to read/write value from and to SBO, also get the updated alarms from SBO.

However, when I test the SubscriptionReader, I come across some problem:

my function is like:

private const string UserName = "admin";

private const string Password = "Admin!23";

private const string EwsEndpoint = "http://localhost:85/EcoStruxure/DataExchange";

public void SubscriptionReader(List<String> subscribeIds)

{

using (var si = new SubscriptionReader

{

EwsEndpoint = EwsEndpoint,

AuthenticationScheme = AuthenticationSchemes.Digest,

UserName = UserName,

Password = Password,

SubscriptionEventType = EwsSubscriptionEventTypeEnum.ValueItemChanged,

Ids = subscribeIds

})

{

var result = si.ReadData().DataRead;

var subItems = si.SubscribedItems;

var Id = si.SubscriptionId;

}

When I input some subscribeIds and call the function, the count of "result" will be Zero and the count of subItems also be Zero, but I do get a SubscriptionId which means it is successful.

Is there any error for my this simple function, do I still need use any getNotification function to finish the subscription?

Please help for it.

Cheers

JeffBowman · ‎2016-09-27

Hi Austen,

Thank you for this info, now that I know that the code runs in runtime, then I can see that the issue is most likely with how your unit test fixture is configured. I have attached a sample Unit Test Fixture, please take note of the FixtureSetup. Make sure your unit test project has a reference to the SmartConnector executable, or else this won't work.

As far as how you did the subscriptions, simply calling si.ReadData() in theory will work. But you need to make sure to program for the case where the Reader outputs IsResubscribeRequired = true. This means that the reader has exhausted all avenues to resubscribe to your list of IDs.. This will probably happen if the EWS Server is offline, or something else terrible . If this happens, I recommend re-instantiating the reader with your original list again. And keep doing this until it works.. Also note that if you subscribe and some Ids fail to be subscribed to for some reason (usually this is the case for Invalid_Ids, or points yet to have been created.. or perhaps other more rare errors). Then your subscription reader will never actually subscribe to these items, ever.. However, you are able to determine this list of failed item ids by comparing SubscribedItems to your original list.. (e.g. originalList.Except(SubscribedItems))

Also, a best practice would be to not use a dead-loop as you did, but leverage SmartConnector's built in scheduling. The good news is, in order to do this, you don't need to write any code, but simply need to configure the Processor to run on a schedule from the web portal. In the SmartConnector portal, select Settings > Schedules and create a schedule that runs at the frequency that you expect.

In the Processor's configuration page, make sure the processor is configured to run on a Schedule and that the schedule you created is selected.

Of course changing this to run on a schedule will fundamentally change the way you need handle the subscription reader. Instead of calling si.ReadData() in a loop, you will need to either Instantiate a new Subscription Reader between runs, or save the Subscription Reader to the InMemoryCache as a JSON String, and deserialize when you take it out and call si.ReadData(). See the below example.

_cache = MongooseObjectFactory.Current.GetInstance<ICache>();

// Sample how to add to cache
_cache.AddOrUpdateItem(si.ToJSON(), "SomeUniqueKey");

// Sample how to retrieve from cache and convert to object

var si = JsonConvert.DeserializeObject<SubscriptionReader>(_cache.RetrieveItem("SomeUniqueKey"));

-Jeff

See Answer In Context

JeffBowman · ‎2016-09-21

Hi Austen,

Does si.ReadData().Success = true?

I would write the code like this, so you can make sure you have a successful read before going forward and trying to parse the results:

var results = si.ReadData();

if (!results.Success)

{

_prompts.AddRange(results.Prompts);

return false;

}

foreach (var result in results.DataRead)
{

// Handle the result.ValueItemChangeEvent

}

Also check si.SubscribedItems, this keeps a list of successfully subscribed item Ids when a subscription is made, otherwise it is empty.

If it turns out you are getting a successful read, but are still not getting any results, would it be possible for you to use Wireshark and capture the traffic between SmartConnector and SBO? It will be easier to debug if we have this.

-Jeff

sesa197614_brid · ‎2016-09-21

The result.success is false, and the prompts exception message is as:

Object reference not set to an instance of an object.at Mongoose.Process.Ews.SubscriptionReader.TrySubscribe() at Mongoose.Process.Ews.SubscriptionReader.ReadData()

Why the si is not an instance here?

-Austen

JeffBowman · ‎2016-09-21

Hi Austen,

Can you try and capture the network traffic of the resulting EWS call (Wireshark or Fiddler)? I am curious as to if the Subscribe request is even being sent out on the wire.

-Jeff

sesa197614_brid · ‎2016-09-22

Hi Jeff:

The running result of my sample is :

Debug,EwsConsume,<no principal>,Subscribe,cb7329e4-d9fd-49c4-98f0-714d6d61b33f,Request,{"Delivery":{"NotifyTo":{"Address":""}},"Expires":"PT30M","Filter":{"EventType":"0","Ids":{"Id":["01/Server 1/Test/Machine1/M1_PowerStatus","01/Server 1/Test/Machine1/M1_RunningStatus","01/Server 1/Test/Machine2/M2_PowerStatus","01/Server 1/Test/Machine2/M2_RunningStatus"],"eventMode":"0"},"EventFilter":{"PriorityFrom":null,"PriorityTo":null,"Types":[]},"KeepAliveCycleTime":null},"version":"1.2","Mode":null}

'VLC.vshost.exe' (CLR v4.0.30319: VLC.vshost.exe): Loaded 'Microsoft.GeneratedCode'.

Debug,EwsConsume,<no principal>,Subscribe,cb7329e4-d9fd-49c4-98f0-714d6d61b33f,Response,{"SubscriptionManager":{"Address":null},"SubscriptionId":"0ab5d80d-b6cb-4639-a6cd-1802b83a647e","Expires":"PT30M","Results":[{"Id":"01/Server 1/Test/Machine1/M1_PowerStatus","Success":true,"Message":null},{"Id":"01/Server 1/Test/Machine1/M1_RunningStatus","Success":true,"Message":null},{"Id":"01/Server 1/Test/Machine2/M2_PowerStatus","Success":true,"Message":null},{"Id":"01/Server 1/Test/Machine2/M2_RunningStatus","Success":true,"Message":null}],"version":"1.2"}

Info,EwsConsume,<no principal>,Subscribe,ET=2067 mSec

Error,Reader,<no principal>,Unexpected exception occurring trying to read subscription 0ab5d80d-b6cb-4639-a6cd-1802b83a647e on endpoint http://localhost:85/EcoStruxure/DataExchange

I also use wireshark to check the data, the http response success with detail information as above.

Then you can see when I use the SubscriptionReader, Subscribe itself is successful, the exception occurred when Read with the error information:

Object reference not set to an instance of an object.at Mongoose.Process.Ews.SubscriptionReader.TrySubscribe() at Mongoose.Process.Ews.SubscriptionReader.ReadData()

It is such a simple usage with exception which make me be stuck by it.

JeffBowman · ‎2016-09-23

Hi Austen,

I took a look at the SubscriptionReader, and it is not clear to me exactly where it is failing, there really isn't much going on there, except that I am adding stuff to the In Memory Cache.. So we might have some sort of version mismatch going on and for some reason the cache never gets instantiated.

What version of the nuget packages are you using?

Also what version of the SmartConnector service do you have installed?

Is this error happening in runtime, or is it happening during a unit test? If this is happening during running a unit test, can you post your unit test class?

Also can you tell me if si.SubscribedItems is null, or if it has the list of ids as expected?

Thanks!

-Jeff

sesa197614_brid · ‎2016-09-26

Hi Jeff:

I found this error only occurs in my unit test sample, with the exactly same code in runtime, there is no error, these codes works fine. is it possible the SubscriptionReader got some limitation that MUST rely on runtime environment?

One more question about Subscription, in order to keep listening data of SBO for a long time(for years), now I create a processor with an internal Dead-Looping function to subscribe points of ES---please find attached code for details.

Is there any other better method to keep subscription? I need all data status changed in SBO will be notified immediately and there is no polling procedure, and by using subscriptionReader, I won't worry about the clear, expired,etc. all are done by default, correct?

-Austen

JeffBowman · ‎2016-09-27

Hi Austen,

Thank you for this info, now that I know that the code runs in runtime, then I can see that the issue is most likely with how your unit test fixture is configured. I have attached a sample Unit Test Fixture, please take note of the FixtureSetup. Make sure your unit test project has a reference to the SmartConnector executable, or else this won't work.

As far as how you did the subscriptions, simply calling si.ReadData() in theory will work. But you need to make sure to program for the case where the Reader outputs IsResubscribeRequired = true. This means that the reader has exhausted all avenues to resubscribe to your list of IDs.. This will probably happen if the EWS Server is offline, or something else terrible . If this happens, I recommend re-instantiating the reader with your original list again. And keep doing this until it works.. Also note that if you subscribe and some Ids fail to be subscribed to for some reason (usually this is the case for Invalid_Ids, or points yet to have been created.. or perhaps other more rare errors). Then your subscription reader will never actually subscribe to these items, ever.. However, you are able to determine this list of failed item ids by comparing SubscribedItems to your original list.. (e.g. originalList.Except(SubscribedItems))

Also, a best practice would be to not use a dead-loop as you did, but leverage SmartConnector's built in scheduling. The good news is, in order to do this, you don't need to write any code, but simply need to configure the Processor to run on a schedule from the web portal. In the SmartConnector portal, select Settings > Schedules and create a schedule that runs at the frequency that you expect.

In the Processor's configuration page, make sure the processor is configured to run on a Schedule and that the schedule you created is selected.

Of course changing this to run on a schedule will fundamentally change the way you need handle the subscription reader. Instead of calling si.ReadData() in a loop, you will need to either Instantiate a new Subscription Reader between runs, or save the Subscription Reader to the InMemoryCache as a JSON String, and deserialize when you take it out and call si.ReadData(). See the below example.

_cache = MongooseObjectFactory.Current.GetInstance<ICache>();

// Sample how to add to cache
_cache.AddOrUpdateItem(si.ToJSON(), "SomeUniqueKey");

// Sample how to retrieve from cache and convert to object

var si = JsonConvert.DeserializeObject<SubscriptionReader>(_cache.RetrieveItem("SomeUniqueKey"));

-Jeff

sesa197614_brid · ‎2016-09-27

Thanks Jeff, for the way to whether configure the processor to run on schedule or not, I have different options as different scenario:

1. For ValueItemReader or ValueItemWriter, they are normally used to read or write data from or to SBO periodically, I will configure such kind of processor to run on schedule for sure.

2. For SubscriptionReader or AlarmReader, if any new updated, it is better for us to get the changes immediately, which is not logically for us to set a schedule to poll the received data again and it brings more delay.And for sure yes, we need consider all exceptions as your comment during the internal looping.Besides, rather than internal dead-looping, is there any better way to make this?

3. For AlarmReader, I found that we MUST keep workstation running to get the live alarms, is it any way to get alarm if the workstation closed?

Expecting for your opinion.

-Austen

JeffBowman · ‎2016-09-28

Hi Austen,

For #2. If I understand what you are trying to do correctly, you want to poll as fast as possible the ES/AS for changes. However, due to the severe performance impact that EWS requests can have in SBO, it is highly recommended not to do this, but to poll at a relatively slower rate.. If it is critical, then you can poll at once per second (and a SmartConnector schedule can be configured for once per second, so I would still recommend doing it this way instead of the dead-loop). But if you can bear it.. then 5 seconds is recommended.

For #3. I don't understand what you mean, Do you mean that in order to see the SmartConnector alarms in SBO Workstation the Workstation needs to be open? Or in order to see SBO Alarms in SmartConnector you need to have WorkStation open. For the later (SBO Alarms in SmartConnector) this should not be the case, as SBO Workstation has no bearing on EWS. It is the SBO Server that handles the EWS requests. If you aren't seeing this, can you show me an example, because something weird must be going on. For the first case, with SmartConnector alarms in SBO, yes this is true, there is currently no way to see Alarms in SBO unless you are logged in to WorkStation (or WebStation).

Regards,

-Jeff

sesa197614_brid · ‎2016-10-04

Hi Jeff:

For #2, if for subscription, I am considering NO polling.

Since with SubscriptionReader, it will be notified the points with value when changed,then it only waits and listens for notification, why should we set a schedule to poll? That's why I create a dead-loop internally(still thinking a better way) to keep the subscriptionReader running. Am I right?

For #3, in my test sample, if I use the dll files to create an independent application to get updated alarms, there will be nothing received if the SBO workstation is OFF(when on, it is normal).

However, if in smartconnector environment, it will all be fine, it doesn't matter whether workstation is on or off.

-Austen

JeffBowman · ‎2016-10-04

Hi Austen,

#2 Unfortunately that is not how it works. There is no Push-Notification mechanism in EWS only Pull (polling), when you call ReadData() what the SubscriptionReader is doing is sending a GetNotification Request to SBO. So in essence every time you call si.ReadData() you are polling, so with the dead loop.. you are polling SBO as fast as possible.. which is not something that should be done. So I still strongly recommend you go with the Scheduling with the guidelines in my previous post.

#3 Again, I still have no idea how you are seeing this behavior, Alarms should always be retrievable from SBO as long as the server (ES/AS) is running. SBO EWS doesn't rely on WorkStation at all. Do you have some example code I can see? Your test fixture?

-Jeff

sesa197614_brid · ‎2016-10-09

Hi Jeff:

I fully understand now, I have changed the mechanism of subscription to scheduling looping.

For the #3 alarms issue, the original test sample has been covered by my new code, when I try to rewrite it again to simulate, I found the problem we talked about does not occur again, then it should be influenced by some other special condition.

BTW,I also tried to test the "IsResubscribeRequired", How could I simulate the IsResubscribeRequired=true? I ever stop the ES during the processor running, but only get the connection failed firstly not the IsResubscribeRequired=true. Could you please suggest me some way?

Cheers

Austen

JeffBowman · ‎2016-10-10

Hi Austen,

As far as testing IsResubscribeRequired=true.. When you stop and start the ES, you mention you get a failed connection, after the ES is started back up what happens? Does the same subscription work as expected?

-Jeff