Login swagger SBO Rest provider

[henrik_alarmdir]

Hi

I have installed the SmartConenctorServer on my development PC and trying to connect to a PC running SBO 2.0 (192.168.10.120), however I cannot figure out which user (where it's defined) to use. In this log below I'm trying to use the admin login I'm using in SBO workstation.

I have enabled EWS webservices in SBO which I assume I need to do? I have pointed the SBO connector to the PC 192.168.10.120.

2018-06-20 17:08:18.1655,Trace,Service,<no principal>,Validating framework license,

2018-06-20 17:08:18.3139,Debug,System,<no principal>,AC:FD:CE:DF:22:4E,BSN12345678901234567,NOT USED,

2018-06-20 17:08:18.3139,Debug,System,<no principal>,AC:FD:CE:DF:22:4E,BSN12345678901234567,S1X1NYAG603105,

2018-06-20 17:08:20.8560,Debug,RestServe,<no principal>,SboEwsRestOAuthProvider authentication,admin,SE#serv1,

2018-06-20 17:08:20.8560,Debug,RestServe,<no principal>,SboEwsRestUserManager is authenticating admin with password SE#serv1,

2018-06-20 17:08:20.8560,Debug,Service,<no principal>,ClientPool.CheckOutConnection for http://192.168.10.120/EcoStruxure/DataExchange,

2018-06-20 17:08:20.8560,Debug,Service,<no principal>,ClientPool.CheckOutConnection created connection 1 for http://192.168.10.120/EcoStruxure/DataExchange,

2018-06-20 17:08:21.1531,Debug,Service,<no principal>,ClientPool.CheckOutConnection for http://192.168.10.120,

2018-06-20 17:08:21.1531,Debug,Service,<no principal>,ClientPool.CheckOutConnection created connection 1 for http://192.168.10.120,

2018-06-20 17:08:21.1531,Debug,CspClient,<no principal>,ConnectNew,2214D8F602574DAD96C6760CD1FEA991,Request,{"ServerAddress":"http://192.168.10.120","UserName":"admin","Password":"XXXXXXXXXXX"},

2018-06-20 17:08:21.2633,Debug,System,<no principal>,AC:FD:CE:DF:22:4E,BSN12345678901234567,NOT USED,

2018-06-20 17:08:21.2633,Debug,System,<no principal>,AC:FD:CE:DF:22:4E,BSN12345678901234567,S1X1NYAG603105,

2018-06-20 17:08:21.4454,Debug,CspClient,<no principal>,{"Value":null,"InhibitDisplay":false,"ResultValue":null,"Failed":true,"HasErrors":true,"HasWarnings":false,"HasMessage":false,"Errors":[{"IsReconnectable":true,"Code":131085,"Domain":2,"DisplayInfo":"","Module":0,"ModuleCode":null,"TechnicalInfo":"","IsWarning":false,"Timestamp":"0001-01-01T00:00:00"}],"Warnings":null,"InnerException":null,"Message":null},

2018-06-20 17:08:21.4459,Error,CspClient,<no principal>,Authentication failed {"IsReconnectable":true,"Code":131085,"Domain":2,"DisplayInfo":"","Module":0,"ModuleCode":null,"TechnicalInfo":"","IsWarning":false,"Timestamp":"0001-01-01T00:00:00"}

   at Mongoose.Csp.CspClient.<>c__DisplayClass7_0.<ConnectNew>b__0(Object <p0>)

,[],

2018-06-20 17:08:21.4459,Error,CspClient,<no principal>,Authentication failed {"IsReconnectable":true,"Code":131085,"Domain":2,"DisplayInfo":"","Module":0,"ModuleCode":null,"TechnicalInfo":"","IsWarning":false,"Timestamp":"0001-01-01T00:00:00"}

   at Mongoose.Csp.CspClient.<>c__DisplayClass7_0.<ConnectNew>b__0(Object <p0>)

   at Mongoose.Csp.CspClient.ExecuteAndLogCall[TRequest,TResponse](String methodName, TRequest request, Func`2 methodToLog)

,["2214D8F602574DAD96C6760CD1FEA991"],

2018-06-20 17:08:21.4459,Trace,CspClient,<no principal>,ConnectNew,ET=291 mSec,

2018-06-20 17:08:21.4459,Error,RestServe,<no principal>,Authentication failed {"IsReconnectable":true,"Code":131085,"Domain":2,"DisplayInfo":"","Module":0,"ModuleCode":null,"TechnicalInfo":"","IsWarning":false,"Timestamp":"0001-01-01T00:00:00"}

   at Mongoose.Csp.CspClient.<>c__DisplayClass7_0.<ConnectNew>b__0(Object <p0>)

   at Mongoose.Csp.CspClient.ExecuteAndLogCall[TRequest,TResponse](String methodName, TRequest request, Func`2 methodToLog)

   at Mongoose.Csp.CspClient.ConnectNew(String serverAddress, String userName, String password, String domain)

   at Mongoose.Csp.CspClient.ConnectNew(ICspEndpoint endpoint)

   at Mongoose.Service.Cache.CachedCspClient..ctor(String poolKey, ICspEndpoint endpoint)

   at Mongoose.Service.Cache.CspClientPool.CreateNewInstance(String poolKey, ICspEndpoint endpoint)

   at Mongoose.Service.Cache.ClientPool`3.<>c__DisplayClass5_0.<CheckOutConnection>b__0()

   at Mongoose.Service.Cache.ThreadSafeDisposableBase.ThreadSafeCall[T](Func`1 callback)

   at Mongoose.Service.Cache.ClientPool`3.CheckOutConnection(TEndpoint endpoint)

   at Mongoose.Service.Cache.ManagedCspClient.ManageCall[T](ICspEndpoint endpoint, Func`2 method, Boolean expireClientImmediately)

   at Mongoose.Service.Cache.ManagedCspClient.ServerVersion(ICspEndpoint endpoint)

   at Ews.RestExtensions.SboEwsRestUserManager.VerifyPassword_Subclass(SoapEwsRestUser user, String password)

,[],

2018-06-20 17:08:21.4459,Debug,RestServe,<no principal>,Login failed,"http://127.0.0.1:8083/GetToken",admin,

[JeffBowman]

Hi Henrik,

You have setup the SBO EWS Rest Provider, unfortunately, the current version of SboEwsRestProvider in Smart Connector 2.3.x is not compatible with EBO 2.0.x yet. This compatibility will be available in 2.4 version of Smart Connector.

That said, in order to get the functionality you are looking for, instead setup a SoapEwsRestProvider instead of the SboEwsRestProvider and set the server address to http://192.168.10.120/EcoStruxure/DataExchange​. If you do that you should be good to go .

Regards,

-Jeff

[henrik_alarmdir]

Hi Jeff

Thank you, yes I get through using the SOAP, however I would like to use REST, can I downgrade SBO to 1.9 and use it instead?

When do you expect 2.4 to land?

[JeffBowman]

Hi Henrik,

Using the SoapEwsRestProvider instead of the SboEwsRestProvider is still using REST. In fact, in the case of the SBO vs SOAP rest provider, sometimes it might actually be better to use the SoapEwsRestProvider depending on what you are trying to accomplish. What is it you are trying to do with it?

The difference between the two, in the case using it with SBO as a data source is that the SBO Rest Provider allows for the the creation of basic object types in SBO (Analog Value, Boolean Value etc..), and also allows the mass get of objects. Since these functions are not available in EWS, the SOAP provider does not.

Regards,

-Jeff

[henrik_alarmdir]

Hi Jeff

We are supplying an independent alarm management platform where we collect alarms from common BMS/SCADA systems e.g. Vista, SBO, Desigo, Honeywell, Vijeo, iFix etc. We typically have two applications, a browser (GUI) and a plugin (windows service).

So I'm trying to do three things:

1. Read events (especially alarms), I want to read them historically, chronology and only once (therefore I need an EventId?, I suspect that isn't available).
2. Read values specific binary points, either polled or subscribed, it could be between 1 to 50 points, this is a watchdog function to monitor if the substations are running and on-line.
3. Read the data point hierarchy (browse the SBO structure, browser is used to pin point binary points)

We are not trying to create nor change anything in SBO, we only read - although, possibility to ACK alarms may be requested from our customers, we'll see.

Now with SBO the on/off line monitoring may be obsolete, if SBO generates a "real" alarm when a substation is off-line, we just don't know SBO that well yet.

REST would be very nice, Vista was not easy to interface, SBO looks much better.

[Adam_Summers]

Hi Henrik,

Jeff Bowman will have to let you know when 2.4 is on the way…

As for using REST/Soap - you are always using Rest as the client when you use one of the REST API Gateway options. What you are selecting at the time of endpoint creation is how the API talks to its server - there is no rest option here.

Your 3 options are:

Soap - Which uses EWS SOAP to talk to EBO/PME/DCE etc (anything which has an EWS endpoint)

Mongoose - Which talks directly to the SmartConnector server and can present an Restful API on top of any extension which serves EWS

Sbo - Which uses the proprietary internal EBO language to communicate - as per Jeff's point, for full support in EBO 2 you'll need SC 2.4 but EBO 1.9 will work with the current release.

My advice, in my humble opinion, unless you need  to do anything like create or delete objects, I'd use the Soap endpoint variant for now. It's more stable as it's been around much longer and supports all the functions for alarms, trends, get set values etc. It's best used with subscriptions if you are looking at polling lots of data across a number of paths in the server.

Adam.

[henrik_alarmdir]

Thanks Adam, I appreciate yours (and Jeff's) humble opinions

I'm a very visualized oriented guy, I need sketches to understand especially when so many abbreviations are in play

Do you happen to have some that visualize such setup's.

This is what I understood - am i on the right path here?

[Adam_Summers · Accepted solution]

Hi Henrik,

That's probably not quite right if I understand what you are doing correctly. Have a look at the rather quick illustration below and let me know if it helps...

[henrik_alarmdir]

Oh that helps a lot, I "only" able to access data from the EBO 2.0.
However the requests I need is not supported, hence the other thread I started.

Then can I assume this:

• The Mongoose EWS Server is irrelevant (to me) since I only want data from EBO (alarms and datapoints/values)?
• The Sbo REST and Soap REST basically exposes the same endpoints but uses different protocols on the backside.
• The Sbo REST doesn't support EBO 2.0 yet.
• The Soap REST does support EBO 2.0.
• If I need other data or implement the non-supported endpoints then I'll have to develop an endpoint extension .dll which communicates with EBO either through the CSP or EWS protocol?

I assume that your Enterprise Server is what I have running on the server. SEEBOES (Lol )

[Adam_Summers]

Hi Henrik,

Glad it helps!

To your assumptions:

·         The Mongoose EWS Server is irrelevant (to me) since I only want data from EBO (alarms and datapoints/values)? - Correct!

·         The Sbo REST and Soap REST basically exposes the same endpoints but uses different protocols on the backside. - Yes.. they expose the same endpoint (all 3 do) but not all methods are supported by all endpoints - that varies on what it's connected to at the backend.

·         The Sbo REST doesn't support EBO 2.0 yet. - Correct!

·         The Soap REST does support EBO 2.0. - Correct!

·         If I need other data or implement the non-supported endpoints then I'll have to develop an endpoint extension .dll which communicates with EBO either through the CSP or EWS protocol? - That's true, you can extend the API and implement your own logic and behaviours to EBO and other systems. Of course, you would only be able to achieve a different way of getting the same data in a custom development as you are still limited by the CSP or EWS functions, you won't be able to do "more” than what you already can, you could just implement it in a different way to suit a different design pattern. - Of course you could add further complexity in the auth etc but fundamentally the same raw methods exist at EWS and CSP to get/set values and alarms etc.

[Adam_Summers]

And yes.. .that service is the Enterprise Server service!

[sesa180908_brid]

Adam,

Nice illustration.  Perhaps Jeff Bowman could put it in the next version of the SmartConnector RESTful EWS Gateway.pdf documentation?

[henrik_alarmdir]

Okay, so writing my own REST extension would not help me e.g. getting access to SEQNO (int) from the database and expose it?

When I look at /AlarmEvents I don't see the SEQNO, unique identifier in the database, I only see Id which smells like some sort of GUID string (which is not sortable).
What I need is a way to read AlarmEvents after as specific Event.
transitionedOnOrAfter is close, but is not a reliable way since multiple AlarmEvents can have the same value.

Perhaps I could suggest that you implement a request like GET /AlarmEvents?transitionedAfter={ID}, that would still abstract from the SEQID and yet return the AlarmEvents I need. And then I'll only need to keep track if the last ID I received.

Previously in Vista, I used EventId, in EBO database you have a SEQNO which is the same, but it's not exposed in REST response nor searchable in the request.

[Adam_Summers]

Hi Henrik,

You could write a custom extension to refer to that number if you wanted to track activity at that level. I don't know if that's exposed over EWS or CSP - Jeff Bowman​ will probably be able to help further on this one.

In terms of alarm activity, there are 2 normal models you can follow in EWS which get's relayed over REST. You can use the GetAlarmHistory and supply start and end dates using a tracker on your last poll time as your new start time will give you all alarm events, all transitions of the alarms through that time period. Alternatively and much better for performance if you don't need to trace all history and you just need to get current alarms (and updates on that stack) you can use the GetAlarmEvents and GetUpdatedAlarmEvents. I'd recommend referring to the EcoStruxure Web Services Specification document which gives examples of those patterns and some use cases for each. - That document should be available in the download section of the SmartConnector site in EcoStruxure Web Services here SmartConnector Server

Adam.

[JeffBowman]

Hi Henrik,

If you are looking for the sequence number of an Alarm Event. I think perhaps EBO is different from vista. There is a GUID based ID, and there is no Sequence Number exposed via EWS. What you might want to sort by is the "OccurredOn" property. As this will show a time based when this event originally occurred. The 'LastTransitionedOn' property is the last time this alarm transitioned from one state to another.

If you need to sort by the sequence number, then you would likely need to write a custom extension as Adam Summers​ mentioned. In this case, it is likely you will need to use CSP (although I cannot confirm, that this sequence number is even available via CSP) in order to retrieve the sequence number of the alarm event.

Regards,

-Jeff

[henrik_alarmdir]

Hi Adam and Jeff

Thanks for both of your replies.

From what you write I understand that:

1. if I need access to the SEQNO then I must write an extension.
2. GetUpdatedAlarmEvents COULD potentially be a solution, but is not exposed as SOAP/SBO endpoint.
3. EcoStruxure Web Services is not a part of smartconnector but is running directly on EBO (if enabled) and is in fact the interface the smartconnector interacts with to expose endpoints?
4. EcoStruxure Web Services does not offer swagger, right?
5. The ID (the GUID thingy), is unique, however it is not sortable in any usable way.

Just some remarks from me, timestamps are not unique and cannot be used as unique identifiers, it may work in LAB but not IRL. So using transitionedOnOrAfter is NOT a reliable strategy. A method "transitionedAfter" could be used if there were used an internal tracking of the SEQNO.

Jeff, your last comment is why I wrote:

Perhaps I could suggest that you implement a request like GET /AlarmEvents?transitionedAfterID={ID}, that implementation could still abstract from the SEQNO and yet return the AlarmEvents I need. And then I'll only need to keep track if the last ID I received.

[henrik_alarmdir]

Can I get access (where) to the CSP document? in case I need to head down that road?

[JeffBowman]

Hi Henrik,

To your points.

1. If you want the Sequence number from the EBO database, then you will need to write an extension, as this is not exposed via EWS. Correct.

2. GetUpdatedAlarmEvents is an EWS method, the way this request works is that each time you send a GetUpdatedAlarmEvents request, along with a 'LastUpdate' which is a GUID for EBO, it returns all the alarms that occurred since that last update, it also returns a new LastUpdate for you to send in your next request. So if you continue doing this, sending the GetUpdatedAlarmEvents with the LastUpdate, you will receive all alarms. There are actually 2 different SOAP EWS Methods required to get alarms GetAlarmEvents and GetUpdatedAlarmEvents. Smart Connector has a class called AlarmItemReader in the Mongoose.Process.Ews namespace that uses an EWS client to manage getting these alarms events for you.

3. EcoStruxure Web Services, plays a big part in Smart Connector. Smart Connector can consume EWS Servers, and also stand up it's own EWS servers for EBO to consume data.

4. EcoStruxure Web Services does not offer swagger since it is a SOAP based Web Service. But there is a WSDL that you can consume to generate a client for it. That said, SmartConnector providers EWS client libraries, so you don't need to do this.

5. When you say 'Sort', are you more worried about sorting a list of alarms, or making sure that you get all the events?

As far as writing your own extension. I think the issue you have today, is that the EWS Restful Gateway does not do the same thing as EWS in terms of a 'Last Update' with getting alarm events, and it would be possible to miss alarm events as they could come into SBO after as the RESTful gateway relies on a passed in time frame instead of a pointer like EWS does. If that is the case, (and I actually agree this is a problem). Then today your best bet is to customize the /AlarmEvents endpoint, and 'hijack' one of the properties such as the 'transitionedAfter' or 'type' to instead taking effectively the 'last update'. This would be possible because behind the scenes the SoapEwsRestProvider is actually using EWS behind the scenes to get the data from SBO (in your case). So in theory you could simply change how the response is filtered.

Take a look at the SmartConnectorSamples GitHub report, at a project called CustomSoapProvider for an example on how to customize a RestProvider GitHub - BuildingsLabs/SmartConnectorSamples: Sample projects for SmartConnector .

As far as CSP documentation. Short answer is, this doesn't exist . If you really want to use the CspClient in Smart Connector, please feel free to send me an email, and I would be happy to send you some samples. Keep in mind CSP generally is not very intutive, so even the wrapped CspClient in Smart Connector is not easy to use, and the use can actually differ between versions of EBO.

Regards,

-Jeff

[henrik_alarmdir]

Thanks Jeff, for a really solid answer, I'll look in to your answers and get back.
I'll do everything possible to stay off CSP though, so I'll have to take a look at the suggested hijack approach.
By the way, could the source for the soap rest extension be made public available? It would make a lot of sense to me to set off in a working extension and tweak it, I need several of the other resources too.

Henrik

[JeffBowman]

The SOAP rest extension is actually built as part of the Smart Connector framework (and included by default). I don't think I will be able to make this source public, but if there is anything specific you are trying to accomplish I could send you some sample code.

Regards,

-Jeff

[henrik_alarmdir]

Hi Jeff, sorry I was off for vacation

I need two thing:

1. I need to monitor alarms and all their states when they are changing, e.g. into alarm, to normal, acknowledge, reset. I need to have a "pointer" that I can pass when requesting for updates on alarms, this could be a EventId (int) or something like GetLastAlarms(since something pointer) that'll return whatever happen since last request.
2. I need to EBO browse for alarm points - a bit like when we were browsing vista.

Point 1 has priority.

Fetching current alarms will not be a problem, although I need to fetch alarms from "history" not only fetch current alarms, hence the need for a pointer. Datetime is IMHO not reliable for this.

So far I have come to the conclusion that I need to implement the EWS SOAP, I don't see how it would be viable to use a smartconnector solution since I'll have to implement EWS any way. Even if I decide to use EWS, it's still unclear what GetUpdatetAlarmEventsRequest>LastUpdate is, it is stated that this could be a datetime reference or something else, the server will return LastUpdate.

well I need to dig into this, vacation tends to reset memory.

regards

Henrik

[JeffBowman]

Hi Henrik,

Welcome back from vacation, I hope you are well rested!

• With regards to fetching Alarm History. There is an EWS method called GetAlarmHistory, this method can return all the alarms overall. It can also take in specific alarm IDs, and returns the history for that alarm only if desired. However, alarm 'history' today is not supported today via the EWS REST Gateway. So you would also need to write an extension that takes the current alarms and historical alarms, and combines them into a single request. See below for a screenshot of this method in use via SOAP UI:
  
• As far as the EWS last update, for EBO this isn't a date-time, but a marker (not an int, but it is a marker none the less), so I think you will be OK there.
• With regards to searching for alarm points, the most efficient way of doing this would be to use the Csp Client, and use it to search, but as noted, Smart Connector 2.3 CSP Client does not support EBO 2.0, and there is not much examples of doing this, but I could likely get your some. More commonly used, you could use EWS to search for all alarm items, this has been done a fair amount, and I could get you some sample code for this. Doing it this way is less efficient, because it must look through the entire object tree branch by branch, but it will get the job done.

Overall I think you have 3 options for getting alarm events.

1. Write a custom extension for the REST gateway /AlarmEvents method as we mentioned before doing the following
   • 'Hijack' one of the properties such as the 'transitionedAfter' or 'type' to instead taking effectively the 'last update'
   • Returning the 'last update' somehow in the /AlarmEvents response.
   • Also use GetAlarmHistory along with GetAlarmEvents/ GetUpdatedAlarmEvents when returning alarms (at least when sending an initial request with no 'Last update' to make sure you get everything.)
2. Instead of using the EWS Restful Gateway, write a Smart Connector extension that will periodically query alarms for you and send them to your application somehow. Doing this will give you the advantage of using the built in EWS Client in Smart Connector (which corrects for some 'caveats' of using EWS with EBO), along with the ability to use the Managed EWS Client, which is recommended.
3. This third option is likely a possibility as well, but not recommended. Since you are creating a windows service in your application normally, then I am assuming you write your code in some .NET language, and that means you could pull in the Ews.Client NuGet package into your project, and use the EWS Client standalone. However, this isn't really recommended, as there are no guarantees everything will work as standalone. For example, logging, you will need to do something like outlined in the last post in this thread to make sure logging is configured correctly. How to change BitFactory.Logging Default Folder Path? . You also won't be able to use the Managed EWS client as this is part of the Smart Connector service and not the Ews.Client library.

Regards,

-Jeff

[henrik_alarmdir]

Hello Jeff
We have been working on your suggestions and have found the EWS SOAP sufficient and fairly easy to work with.

Thank you so much for your help on this