Slow Refresh Speed for SmartConnector RESTful Gateway

[sesa145087_brid]

Dear All,

We have three projects, which use SmartConnector Restful Gateway to share data to 3rd party cloud platform. One project polled all the points from virtual points which are bound to real points in AS, and the response time are very fast. But for other two projects, all the points polled directly from B3-AS-ES-SmartConnector is very slow. One hundred points needs to take 300s to refresh. The cloud platform has its own test machine locally, and prescribes 100 points one time. This is unacceptable for the customer. But when we checked the points in SBO workstation itself, it is OK. So my question is, how to troubleshoot this situation? Is there anywhere to check what is the root cause of this issue? Thank you very much!

Andrew Ding

[sesa180908_brid]

If you are using the SoapEwsRestProvider, you should turn on "Ews Consume" logging at the debug level. 

If you are using the SboEwsRestProvider, you should enable "Csp Client" logging at the debug.

This will show response times to see if the issue is in the SOAP/CSP layer or somewhere else.

Either way, polling is never the best approach for retrieving large numbers of points (especially from field bus devices).  You should be using subscriptions when retrieving this type of data from SBO.

[JeffBowman]

Hi Andrew,

If they are using GET /Values/{id}/Value for each and every object they want the value for, then this is the kind of response time I would expect, going down to the field bus can be time consuming.

If they want a faster response time, they should use the /Subscriptions and /Notifications endpoints to get these values. See below for a general idea of how to accomplish this.

With the /Subscriptions Endpoint do a.

REQUEST 1:

POST /Subscriptions/Create with a body like:

{

  "DurationInMinutes": 15,

  "SubscriptionType": "ValueItemChanged",

  "Ids": [

    "01/Server 1/AV1",

    "01/Server 1/AV2",

    "01/Server 1/etc..."

  ]

}

RESPONSE 1:

This will return a body with an ID. When you want to check these values

REQUEST 2:

POST /Notifications/Create

{

  "SubscriptionId": "ID From above response",

  "ChangesOnly": true

}

RESPONSE 2:

This will return a notification ID. Then you do:

REQUEST 3:

GET /Notifications/{ID FROM ABOVE RESPONSE, without the {}}/Items

RESPONSE 3:

This will return a list of all values that have changes since the last notification response (since the notification has ChangesOnly = true. The first time you do this always, you will get the current value of ALL objects asked for, and subsuquent requests will return just the changes from the last notification.

Now basically just keep doing the POST /Notifications/Create and the GET /Notifications/{id}/Items over and over again.

Before the subscription expires, you will want to do a:

PUT /Subscriptions/{id}/Renew where the ID is the id returned from RESPONSE 1 with a body of:

{

     "minutes": 15

}

In order to keep your subscription alive, you can also terminate your subscription early if you want via the user of:

PUT /Subscriptions/{id}/Terminate where ID is the id returned from response 1.

If you end up getting a bad response from the web server on your subscription, just recreate it and start over.

Regards,

-Jeff

[sesa145087_brid]

Hi Jeff,

Thank you very much. They are using subsription. Sorry for my mistake. What they told me is when creating the subscription, it is very slow for the first time. But the update after that is OK. But they have to create new subscription a lot of times since  "  "ChangesOnly": true" doesn't take effect. No matter whether it is set to true or false, it only returns change only values. And why they they want the set "  "ChangesOnly": false"  is they found what the real changed values are fewer that what they really found in the system. Do you know what should be checked out to find out why this will happen?

Andrew

[sesa145087_brid]

More detail on why they want to set "  "ChangesOnly": false". If the the first subscription returns the change successfully, and second one fails due to like network timeout. Then the subsription for the 3rd time only replies changes between 2nd and 3rd. The ones between 1st and 2nd are missing. Is that correct?

Andrew

[JessieSu]

Hi Andrew,

Maybe below topic answered some questions of yours?

How can I get a batch of values if I use SoapEwsRestProvider in SmartConnector?

BRs

[sesa145087_brid]

Hi Jeff,

Please ignore my former questions and I re-raise here. The core question has only one left. If there are points offline, the subscription will not give out any response. So the client software has to setup a timeout time, which can't be too short or too long to end this session and re-issue one. This is why the first time subscription will be so slow for the project, especially for a project over 20,000 points. So my question is: is there any internal mechanism inside SmartConnector that will give out response to the client so that the client can quickly know the points are offline and we don't need to wait?

Thank you very much!

[JeffBowman]

Hi Andrew,

I am not quite sure what you mean. Are you saying that when you try to subscribe, and 1 of the points are offline, then the subscribe response never returns or is the create notification response that doesn't return?

I am not aware of an mechanisms within SmartConnector that would be able to determine this. However, I will look into the issue and report back here with my findings in the next day or so!

Regards,

-Jeff

[JeffBowman]

Hi Andrew,

When I tested this out, it seems as though SBO is returning with an "INVALID_ID" response for offline b3 points. I will have to check, but this doesn't make sense to me, as I can actually use the same ID to get its value etc.. Just when it comes to subscribing 'INVALID_ID' is returned. Is this what you were seeing?

Regards,

-Jeff

[sesa145087_brid]

Hi Jeff,

Yes we have met such error. But at most time we got no reply until we set the timeout from the client side, or the response will be very long according to SmartConnector log.

[JeffBowman]

Hi Andrew,

I am having a hard time producing this. I don't have a system available with that many values in field controllers..

Do you know if the implementation used is sending all the values in the same request? Or are you only sending a certain amount per request (Like for example sending to 1000 IDs with 20 requests?)

Regards,

-Jeff

[sesa145087_brid]

Hi Jeff,

Currently the package size is 200 values per request. And all the requst are sent at the same time.

Andrew

[sesa197614_brid]

Hi Jeff:

Some more question related with subscription behavior:

1. Is it possible to keep the same token?

2. If token has to be expired, will it cause subscription be terminated?

3. If new token was generated, could the running subscription Id be maintained.

Cheers

Austen

[JeffBowman]

Hi Austen,

Just so we are on the same page. When you say 'Token' I understand that to be the Authorization token.

1. The Auth token can only be configured to be valid for 1-60 minutes. This is configured in the HTTP Configuration of the REST Endpoint by changing the 'Access Token Expire Time Span Minutes' property. The reason for only allowing 1-60 minutes is for cyber security concerns with regards to access of BMS data.

2. If an Auth Token expires, it will NOT cause a subscription to terminate.

3. If a new token is generate, you WILL be able to continue using the same Subscription/Notification IDs.

Regards,

-Jeff

[sesa197614_brid]

Great,Jeff, I have test it, so that I only need keep generating new token and renewing the subscription at an interval, to keep the same subscription forever.

Cheers