A few performance questions about the Soap Rest and Sbo Rest endpoints

jritchey_jaxcon · ‎2018-01-05

I have a few questions about performance for the SmartConnector endpoints, just trying to make sure our client is making the best use of the endpoints, and not bogging down the ES with needless requests.

1) frequent requests

I'm wondering if making too many frequent requests to an endpoint will affect the ES, or if the effects would be isolated to the SmartConnector / endpoint?

2) polling vs proxy

Does the endpoint poll the ES, or is it just acting as a proxy (translating REST requests to Soap) between the client and the ES?

3) caching (if proxied)

if the SmartConnector is acting as a proxy, does it try to mitigate traffic to the ES by caching frequently requested values?

4) It seems unlikely, but perhaps the Smart Connector creates COV subscriptions to frequently requested (or all) values in the ES?

5) requesting values from Bacnet / IO values

For Bacnet and other IO devices, does requesting the value end up creating more traffic on IO bus, or are values directly pulled from the ES? If yes, does creating a shortcut get around this potential pitfall?

6) shortcuts

Is there any any performance penalty to grouping a collection of shortcuts into a folder so as to obtain all values in a single REST request?

7) if I have a large number of points in the same folder, and I request them all simultaneously, but each in individual http GET requests, is that going to be significantly more overhead on the ES than requesting the Children of the container in a single http GET request?

that's everything for now, hopefully someone can help.

thanks!

-Josh

JeffBowman · ‎2018-01-05

I'll try and answer these .

1) frequent requests

I'm wondering if making too many frequent requests to an endpoint will affect the ES, or if the effects would be isolated to the SmartConnector / endpoint?

In general making too many requests to anything will affect it. I guess it depends on what you mean by by frequent . Since the SoapEwsRestProvider uses EWS behind the scenes, it would affect the ES if you are really really pounding on it.

2) polling vs proxy

Does the endpoint poll the ES, or is it just acting as a proxy (translating REST requests to Soap) between the client and the ES?

In general this is what it does, but it does have some caching involved for some methods (Trends/ AlarmEvents/ Subscriptions/ Notifications).

3) caching (if proxied)

if the SmartConnector is acting as a proxy, does it try to mitigate traffic to the ES by caching frequently requested values?

Some things are cached for sure, alarm events, trend records, last known values for subscriptions and notifications, but things like folder Children are not cached, as these can change often.

4) It seems unlikely, but perhaps the Smart Connector creates COV subscriptions to frequently requested (or all) values in the ES?

There is nothing today that is creating COV subscriptions (as this concept doesn't exist in EWS, everything must be polled). However, using the /Subscriptions and /Notifications will use EWS Subscriptions behind the scenes, which is as good as you can get from a performance and overhead perspective.

5) requesting values from Bacnet / IO values

For Bacnet and other IO devices, does requesting the value end up creating more traffic on IO bus, or are values directly pulled from the ES? If yes, does creating a shortcut get around this potential pitfall?

When polling values from SBO, the REST provider uses EWS Subscriptions behind the scenes. EWS Subscriptions are far less taxing on the system than doing EWS GetValues or GetItems as these will send a request to the field bus every time they are called while EWS Subscriptions uses internal SBO methods to get updated values from the field bus.

6) shortcuts

Is there any any performance penalty to grouping a collection of shortcuts into a folder so as to obtain all values in a single REST request?

Not that I can see.

7) if I have a large number of points in the same folder, and I request them all simultaneously, but each in individual http GET requests, is that going to be significantly more overhead on the ES than requesting the Children of the container in a single http GET request?

Yes, if you send individual http GET requests, the REST provider will send individual EWS GetItems requests to the server.. If you do it while getting the Children of a container, it will send a single EWS GetItems request that gets them all at once, so it will be significantly less overhead for the ES to do it that way over an http GET for each. That said, if you are doing this in order to get the Values of all the children, I would suggest not doing it this way, but use the /Subscriptions and /Notifications endpoints to get the values. This will be far less overhead

Regards.

-Jeff

See Answer In Context

JeffBowman · ‎2018-01-05

I'll try and answer these .

1) frequent requests

I'm wondering if making too many frequent requests to an endpoint will affect the ES, or if the effects would be isolated to the SmartConnector / endpoint?

In general making too many requests to anything will affect it. I guess it depends on what you mean by by frequent . Since the SoapEwsRestProvider uses EWS behind the scenes, it would affect the ES if you are really really pounding on it.

2) polling vs proxy

Does the endpoint poll the ES, or is it just acting as a proxy (translating REST requests to Soap) between the client and the ES?

In general this is what it does, but it does have some caching involved for some methods (Trends/ AlarmEvents/ Subscriptions/ Notifications).

3) caching (if proxied)

if the SmartConnector is acting as a proxy, does it try to mitigate traffic to the ES by caching frequently requested values?

Some things are cached for sure, alarm events, trend records, last known values for subscriptions and notifications, but things like folder Children are not cached, as these can change often.

4) It seems unlikely, but perhaps the Smart Connector creates COV subscriptions to frequently requested (or all) values in the ES?

There is nothing today that is creating COV subscriptions (as this concept doesn't exist in EWS, everything must be polled). However, using the /Subscriptions and /Notifications will use EWS Subscriptions behind the scenes, which is as good as you can get from a performance and overhead perspective.

5) requesting values from Bacnet / IO values

For Bacnet and other IO devices, does requesting the value end up creating more traffic on IO bus, or are values directly pulled from the ES? If yes, does creating a shortcut get around this potential pitfall?

When polling values from SBO, the REST provider uses EWS Subscriptions behind the scenes. EWS Subscriptions are far less taxing on the system than doing EWS GetValues or GetItems as these will send a request to the field bus every time they are called while EWS Subscriptions uses internal SBO methods to get updated values from the field bus.

6) shortcuts

Is there any any performance penalty to grouping a collection of shortcuts into a folder so as to obtain all values in a single REST request?

Not that I can see.

7) if I have a large number of points in the same folder, and I request them all simultaneously, but each in individual http GET requests, is that going to be significantly more overhead on the ES than requesting the Children of the container in a single http GET request?

Yes, if you send individual http GET requests, the REST provider will send individual EWS GetItems requests to the server.. If you do it while getting the Children of a container, it will send a single EWS GetItems request that gets them all at once, so it will be significantly less overhead for the ES to do it that way over an http GET for each. That said, if you are doing this in order to get the Values of all the children, I would suggest not doing it this way, but use the /Subscriptions and /Notifications endpoints to get the values. This will be far less overhead

Regards.

-Jeff

jritchey_jaxcon · ‎2018-01-08

I think I need a little more clarification about a couple of things:

😎 When I send a GET /Values/{id} or GET /Containers/{id}/Children request, does this result in a corresponding EWS getValues and EWS getItems, or is this handled by EWS Subscriptions?

9) Is there a document that describes how to use the /Subscriptions and /Notifications of the REST endpoint?

JeffBowman · ‎2018-01-08

I think I need a little more clarification about a couple of things:

😎 When I send a GET /Values/{id} or GET /Containers/{id}/Children request, does this result in a corresponding EWS getValues and EWS getItems, or is this handled by EWS Subscriptions?

GET /Values/{id} is backed by a GetItems request to the ValueItem requested. GET /Containers/{id}/Children is backed by a combination of GetContainerItems and GetItems as needed for any additional properties that are returned as part of the Item type models. If you do a GET /Values/{id}/Value this is backed by GetValues (note that this REST endpoint only allows for a single {id} at a time)

9) Is there a document that describes how to use the /Subscriptions and /Notifications of the REST endpoint?

This question has been answered on this forum before , please take a look at: https://community.se.com/message/108252?commentID=108252#comment-108252

Regards,

-Jeff

jritchey_jaxcon · ‎2018-01-08

So, based on the answer to (9), then my client can poll the /Notifications at some absurd high frequency, and it would not interfere with the ES (albeit it may waste bandwidth between the client and the REST endpoint, and possibly bog down the machine running SmartConnector)?

JeffBowman · ‎2018-01-08

I definitely wouldn't do that!

/Subscriptions/Create does a EWS 'Subscribe' request.

/Notifications/Create will actually do the EWS 'GetNotification' request and cache that data then the /Notifications/{id}/Items will simply retrieve these values from the cache for you.

Therefore if you are doing /Notifications/Create followed by /Notifications/{id}/Items at some absurd high frequency, it's possible that could have a negative affect on the ES. Personally, I wouldn't do this any faster than 1 second apart just to be safe.. But because the EWS subscription mechanism in general way better as far as affecting the ES, then in theory you could do much faster.. It would just involve some testing on your end to make sure you aren't killing the ES .

Another thing of note, which I just ran into today, based on another post in this forum is that the 'ChangesOnly' functionality in /Notifications/Create doesn't seem to be supported by SBO. (Honestly, its probably an SBO bug that they are unlikely to fix anytime soon..) In order to get this type of functionality, you would need to effectively create a new subscription every time followed by it's first Notification..

Regards,

-Jeff

jritchey_jaxcon · ‎2018-01-08

okay. Well I guess we don't need to poll it at a very high frequency anyways if it will catalog all changes that occurred between now and the last time the notifications were requested (if I'm understanding that correctly).

JeffBowman · ‎2018-01-08

I think you may be misunderstanding a little bit.

You do not get a History of all value changes for each value item from the last time you asked, you only get the most recent change for any value that has changed since the last time you requested the values. (This is how EWS Subscriptions work.)

Regards,

-Jeff

jritchey_jaxcon · ‎2018-01-08

Oh, that would make more sense with the bug you were mentioning. So I can subscribe to a list of values by their ids, and then when i request notifications, I get the current value for each. (if 'ChangesOnly' were supported, then the results would only include when values had actually changed)

JeffBowman · ‎2018-01-08

I believe it is actually the opposite. SBO will only ever return that values that have changed since the last time you requested a notification (EWS GetNotification request). The only way to get all the values is to create a new subscription and send the notification request for the first time.

This is how EWS Subscriptions work:

1. Subscribe to a list ValueItem IDs with 'Subscribe'.

2. Send a GetNotification with an optional NotificationId (From the REST Provider perspective If ChangesOnly = false, then no NotificationId is sent, if ChangeOnly = True, then the most recent NotificationId is sent).

If a NotificationId is not specified, then all values should be returned
If a NotificationId is specified, then only values from when the NotificationId was created should be returned.

The issue with SBO is that it completely ignores NotificationId and keeps an internal cache of value changes.. So the first time you send a GetNotification to Subscription you get everything, and then the next times you send a GetNotification to the same subscription you only ever get changes from the previous time you send a GetNotification, even when specifying an earlier NotificationId.

Regards,

-Jeff

jritchey_jaxcon · ‎2018-01-09

it seems like it should be fine if I have something like below then:

1) POST /Subscription/Create --> {ids:[...]} --> Subscription:{id:"..."}

2) POST /Notifications/Create --> {SubscriptionId:Subscription.id, ChangesOnly:true} --> Notification:{id:"..."}

3) GET /Notifications/Notification.id/Items --> [{id:"...", Value:"..."}, ...]

4) DELETE /Notification/Notification.id/Delete

5) repeat (2 - 4) until Subscription is expired

and as long as I never try to go back and reuse one of my old Notification ids, the bug won't affect my code?

JeffBowman · ‎2018-01-09

As long as you are using ChangesOnly = True, then you should be all set with what you layed out .

Regards,

-Jeff

jritchey_jaxcon · ‎2018-01-10

For polling Alarm State, is there any performance gains using the /AlarmEvents?id={id} endpoint over /Alarms/{id}?

There doesn't appear to be a way to request AlarmEvents with a specified list of ids the way there is for ValueItems.

JeffBowman · ‎2018-01-10

You are correct,

With regards to Values, the /Subscription, /Notifications endpoints using EWS Subscriptions behind the scenes. However, while the spec for EWS subscriptions does have support for AlarmEvents (by listing items etc.) SBO does not support this feature of EWS Subscriptions. So therefore the only way to get AlarmEvents with all the useful information is with the /AlarmEvents Endpoint.

The reason I am calling out /AlarmEvents over /Alarms is with the /Alarms endpoint, you simply get to look at the Alarm object, and while you can see it's 'State' you cannot do anything to this alarm (e.g. acknowledge), or see other properties such as 'Acknowledgeable', 'OccurredOn', 'LastTransitionedOn', 'Message', 'Priority', and 'Type'(Alarm Category). When you use the /AlarmEvents, you also get the AlarmEvent ID that can be used to acknowledge that alarm back into SBO along with the information listed above.

If you want to use the /AlarmEvents endpoint to retrieve AlarmEvents only from a specific Alarm, you could do /AlarmEvents?alarmId={alarmId}. All the rest provider is doing is doing stanard EWS GetAlarmEvents and GetUpdatedAlarmEvent calls, and filtering the results to include only alarm events from a specific alarmId.

If you only care about the state of the alarms and don't want to acknowledge or know any additional information about the event, then in theory you could use the /Subscription, and /Notification endpoints to get all the states of the alarms in a single request. To do this, you would need to use a different path to your alarm item, instead of using the '02/Server 1/PathToAlarmItem' path use '11/Server 1/PathToAlarmItem/AlarmState'. Notice the use of '11' instead of '01' or '02' and the appending of 'AlarmState' after the path.

Regards,

-Jeff