Generating a List for configuring a Processor based on a datasource

[jritchey_jaxcon]

I want to make a Processor that works similarly to how SimpleXMLWebservice Interface in SBO works, except with Json.  I have some code that generates a list of pseudo 'XPath' strings by traversing the Json document structure, and it works in unit testing, but I can't get the list to show up in SmartConnector.

Additionally, I can't seem to get configuration to show even a simple list of one item:

[Required]

public IList<string> AvailablePointPaths {get=>new List<string>{"Test"}}

Instead of showing a list with an item "Test", the list is empty in the UI.

I'm also not sure this is the best way to achieve the desired results, since it will not include the checkbox next to the path, so if anyone knows a way to make that happen, I'm all ears.

[JeffBowman]

Hi Josh,

It looks like what you are trying to do is add a 'default' value to a list in a processor configuration. Unfortunately default lists in configurations are not currently supported by Smart Connector.

That said, I am not sure I am completely clear on how you are achieving what you are trying to achieve. Can you explain in more detail, how EBO would consume these services?

From my experience, your best bet would be to parse these JSON responses, and then add them to a Smart Connector EWS server so that EBO can consume the data as EWS. There are a bunch of benefits to doing this.

• EBO won't run into 'response size' limitations or 'request size' limitations
• You can in theory add a bunch of 'Web Services' into the same EWS Server, so that you don't run into architectural guideline limits for the amount of web services supported.
• All values in the response can be kept up to date in the EWS server, and the user can pick which properties they want
• Its easier to get new objects with EWS than with a web service (Simply re-host).
• Response order and length doesn't have to be exact every time. For example, if your web service returns a list of rooms with some properties, with EBO Web services, the rooms have to be in the same order and all rooms need to returned in the response every time or else your created objects in EBO will be updated incorrectly, since it only uses indices. With an EWS server you can keep track of room name as a EWS ID, and it won't matter in which order the response comes back in.
• Authentication is simply the EWS authentication, any Web Service authentication is handled in your processor code, so you have way more functionality than 'basic'. (e.g. API Key authentication, OAuth if you wanted to)
• I'm sure there are more I am missing as well.

In my free time (which is not that much these days). I have also been working on a similar Smart Connector extension that generically handles both XML and JSON web services. When it is completed (whenever that may be, not likely in the near future..). In theory it will have the ability to:

• Specify any type of authentication for a web service, with the ability of a 'custom' authentication type where a developer can effectively add a custom Authentication class to use.
• Specify an HTTP verb for a request
• Specify a request stream that effectively allows a user to send multiple requests in order to a web service and only create the EWS server based on a specified response. (for example, create a subscription --> ask for updates with subscription ID and only scaffold the EWS server for the second response).
• Specify multiple web services in a single configuration, which will allow a single EWS server to be created with multiple sets of data.
• Scaffold an EWS server representation of Web Service response with a default mechanism, but like authentication also gives developers the ability to write some code for a custom scaffolding provider.

There is definitely a need for something like this, so keep us updated on your progress .

Regards,

-Jeff

[KarimHussain]

Sorry for off-topic but that generic web service consume with JSON and multiple auth mechanism support sounds very interesting. I've also has this on my mind but like you my "free time" is not much these days (got my first kid last summer ).

Keep me updated if you make any progress or want to team up in some way to make more progress.

[jritchey_jaxcon]

Like you said, we'll be populating a SmartConnector EWS Server with points brought in from the REST webservice.  I was trying to make it so that in configuration, the user could select a subset of the values to bring in, rather than just mirroring every value in the EWS Server.

This way, we would have all of the features of SimpleXMLWebservices, but without the response size limit, and with support for both json and xml.  I'm currently a little fuzzy on how the SmartConnector EWS Server works with SBO, but my understanding was that you can bring it into an ES the same way you would bring an AS into ES.

[JeffBowman]

Hi Josh,

I see! You are definitely on the right path then.

As far as how the EWS servers work with EBO. It is not the same as bring in an ES/AS. Basically, you create an interface (new > interface). Then select EcoStruxure Web Service and enter information that forms your URL and and enter your credentials. Then you navigate to the hardware folder (System > Hardware >EcoStruxure Web Services > {Your Interface Name}, and then you can navigate the EWS server and choose which points you want to host (e.g. be visible in your interface). You can either host specific points, or folders, if you host a folder you will get all the children as well. After you host, of course you can go back and delete any points that were hosted that you don't really want. Keep in mind though, with EWS, unless you are using a specific point in something (e.g. a graphic, program, alarm, etc..). It won't have any affect on your request bandwidth, as EBO is smart enough to only access for updates on points it is using.

I would say, that if you wanna do some filtering prior to creating the EWS server, your best bet is to generically give the user a list to add values to, and have them add the names of the properties they either want or don't want. As mentioned, there is really no way to add dynamically add values to a list in a configuration (although there is an existing feature request for something similar, no idea when that might be done).

Regards,

-Jeff

[jritchey_jaxcon]

Oh okay.  From what you've said, it sounds like we may not want/need to have the user pick which values they want from the webservice, because that choice is made in the EcoStruxureWebservice Interface configuration, and there won't really be any bandwidth savings.

On the subject of being smart and only updating changing points:

A secondary feature we were hoping to add is a 'bidirectional' feature, which we believe will be valuable for set-points and other similar 'rarely changing' values.  This is of course rather specific to our set of webservices, but the basic idea is that if a point changes in SBO, our Processor will detect the change, and send a PUT request back to our webservice.

[JeffBowman]

Hi Josh,

Yup, that's what I was getting it, it may not be worth custom filtering.

What you have mentioned, is a very common pattern in Smart Connector. So you should have no problems doing something like this . Take a look at the CustomSetValuesProcessor class in the WeatherExtension sample on github for an example on how this can be done. The developer's guide describes this in more detail if you are interested as well. Doing this in a 'generic' way would likely be pretty difficult though since most web services have a different ways to write values (different requests/ formats etc..). But it would be cool if you could make this generic too!

SmartConnectorSamples/CustomSetValuesProcessor.cs at master · BuildingsLabs/SmartConnectorSamples · ...

Regards,

-Jeff

[jritchey_jaxcon]

I've got a basic implementation of this working now.  I was wondering if there is a way to configure it in SBO to automatically host new values that get added.  e.g. if a new value gets added in the other system we're bringing in, it will now show up in SBO.

The way I have it configured now, the new value will make it into the EWS Server, but doesn't show up in SBO until I manually tell it to host the value.

[JeffBowman]

Hi Josh,

Typically 'manual' re-hosting is the only way to do this. But I have been thinking about this for a while, because it would be nice to automatically re-host sometimes. My gut instinct is that there should be a way to do this using the CspClient class in Smart Connector.. That said, I have no looked into this at all, so at this point I really don't know how you would do it, even with that.

I will try and see if I can take a look into this in the next day or two if I can find the time and keep you updated.

Regards,

-Jeff

[jritchey_jaxcon]

Thanks! That would awesome

[jritchey_jaxcon]

For the authentication, is there a way to make SmartConnector show a dropdown containing [None, Basic, Digest]?

I tried using an enum, but it didn't show up in SmartConnector.

[JeffBowman]

Hi Josh,

Enum's should work, can you paste your class code here or send it to me?

Regards,

-Jeff

[jritchey_jaxcon]

public class SmartConnectorTest : Processor{

    public override bool IsLicensed => false;

    [Required, DefaultValue("http://localhost:8124")]

    public string Host { get; set; }

    [DefaultValue("/points.json")]

    public string Path { get; set; }

    [Required, DefaultValue(AuthType.None)]

    public AuthType Authentication;

    [DefaultValue("user"), EncryptedString]

    public string Username { get; set; }

    [DefaultValue("pass"), EncryptedString]

    public string Password { get; set; }

    [DefaultValue(false)]

    public bool Bidirectional { get; set; }

    [DefaultValue("EWS Server")]

    public string EwsServerName { get; set; }

    [DefaultValue("user"), EncryptedString]

    public string EwsServerUsername { get; set; }

    [DefaultValue("pass"), EncryptedString]

    public string EwsServerPassword { get; set; }

    [DefaultValue("/")]

    public string EwsServerPath { get; set; }

    ...

}

public enum AuthType { None, Basic, Digest};

[JeffBowman]

Not sure if this is the issue or not, but you don't have a getter and setter set on your AuthType property.

Can you try and set:

    [Required, DefaultValue(AuthType.None)]

    public AuthType Authentication;

to

    [Required, DefaultValue(AuthType.None)]

    public AuthType Authentication {get; set;}

Regards,

-Jeff

[jritchey_jaxcon]

Yep, that fixed it.  I had it as a public field when it needed to be a property.

Not sure if this is a bug, but I had to delete my existing Processor after I had updated the dll and restarted SmartConnector to get the new configuration settings to show up (that said, I'm also running a slightly out-of-date version of SmartConnector).

anyways, it's all working now

[JeffBowman]

Hi Josh,

Great to hear the problem was fixed!

As far as your question about needing to recreate the processor. This is expected, this is because when a processor configuration is created, the properties, are created generically in the database. We currently have no way to update the properties for an existing configuration if the configuration properties are deleted/ added/ change etc.. between releases. This kind of feature is on the roadmap, but does not have high priority at the moment.

Regards,

-Jeff

[JeffBowman]

Hi Karim,

I'd be more than happy to share the code I have so far if you wanted to poke around. I will caution you in that the code is in a bit of a state of flux, as there are a lot of moving parts, and the last thing I remember working on (about 6 months ago!) was taking some stuff apart to try and shim some of the more complex stuff in (I think likely related to a name inference class.)

Let me know!

Regards,

-Jeff

[jritchey_jaxcon]

So, I'm now focusing on getting the bidirectional write functionality working.

I've set up my EWSServer in SBO, so I can see that when I change a value in my webservices, my Processor updates the value in EWSServer, and that change makes its way into SBO.

What I'm having trouble with, is that if I change the value in SBO, the EWSServer doesn't seem to reflect this change.  I hit refresh, and SBO still shows the value I picked, but the EWSServer has the old value.  I guess I would have thought that writing the value in SBO would write the value in EWSServer as well?

EDIT

okay, actually the EWSServer has the correct value as well.  For some reason, when I use valueItem.Value (for the EwsValueItem in question), it still returns the old value.

[jritchey_jaxcon]

Disregard previous.  Turns out I had made a mistake in my configuration and was looking at the wrong valueItem.  Everything is working as expected.