Welcome to the new Schneider Electric Community

It's your place to connect with experts and peers, get continuous support, and share knowledge.

  • Explore the new navigation for even easier access to your community.
  • Bookmark and use our new, easy-to-remember address (community.se.com).
  • Get ready for more content and an improved experience.

Contact SchneiderCommunity.Support@se.com if you have any questions.

Close
Invite a Co-worker
Send a co-worker an invite to the Exchange portal.Just enter their email address and we’ll connect them to register. After joining, they will belong to the same company.
Send Invite Cancel
84525members
353766posts

Geo SCADA MQTT Topic with nested data

EcoStruxure Geo SCADA Expert Forum

Find out how SCADA systems and networks, like EcoStruxure Geo SCADA Expert, help industrial organizations maintaining efficiency, processing data for smarter decision making with IoT, RTU and PLC devices.

Solved
ash_aqx
Lieutenant JG
Lieutenant JG
0 Likes
9
2040

Geo SCADA MQTT Topic with nested data

Hi All,

 

I'm wondering whether the MQTT driver is capable of supporting data in nested JSON elements.

The data that I'm receiving is split up a bit differently than what is described in the help files, with timestamps separated from data in the following fashion:

 

{

"payload_fields":{

    "sensor1":67593

    "sensor2":83463

    },

"metadata":{

    "time":"2021-02-10T12:28:16.002839378Z"

    },

}

 

 

I've tried referencing the tag names like this:

  • payload_fields.sensor1
  • payload_fields.sensor2

But that doesn't appear to be working either. My guess is that because the "time" tag is in a different place to the data GeoSCADA is getting confused..

 

Any assistance / pointers would be greatly appreciated!


Accepted Solutions
AdamWoodland
Commander Commander
Commander
4
2010

Re: Geo SCADA MQTT Topic with nested data

The trouble(*) is MQTT is really just a transport mechanism like TCP. It is like saying we have a TCP telemetry driver when you're interested in DNP3 but can also consider Kingfisher, Modbus, etc, as its just packaged up differently but it is all the same TCP transport (ignoring choosing serial or UDP options!)

 

Most MQTT communications uses some form of JSON, but I also have to deal with CSV and raw text data and that's before I have to worry about how the data is encoded (e.g. base64) and then what the values mean (e.g. raw readings versus changes since previous reading). Until the various MQTT publishers use standards such as WITS-IoT and SparkplugB dominate, I've yet to interface with a device that uses either, then it is effectively a wild west.

 

As to how you can solve the problem... you can raise a support request for an enhancement, or...

 

You can compile the SparkplugB driver from https://github.com/GeoSCADA, Steve has made it open source. I believe there is a WITS-IoT driver around from Steve too, but not sure of the commercial arrangements of that. You could copy Steve's SparkplugB driver and make the relevant changes, its in C# using the Geo SCADA DDK.

 

Beyond that I do develop our own set of MQTT drivers to interface with the numerous other devices that my customers need to interface with and so its possible you could do the same. or chat to your Schneider rep and they can reach out to me to discuss further depending on which region you're in, or my colleagues elsewhere in the world. Some of our SIs could also help develop a driver if you already have existing agreements with them.

 

(*) I use the term trouble here, but it is also an advantage with significant flexibility which is one reason why MQTT is so popular in IoT

See Answer In Context

9 Replies 9
geoffpatton
Commander
Commander
0 Likes
7
2016

Re: Geo SCADA MQTT Topic with nested data

I was only testing MQTT from a Node-Red application, but if I did not have it formatted from Node-Red exactly as documented I got nothing in ClearSCADA. So I think you are not going to get it to work with your current format.

 

@sbeadlecan probably confirm.

ash_aqx
Lieutenant JG
Lieutenant JG
0 Likes
6
2014

Re: Geo SCADA MQTT Topic with nested data

Thanks for the reply,

I thought this might be the case.

 

Do you know who might be a good contact to get driver enhancements made?

 

It seems like this is a relatively common complaint from people using the MQTT driver.

 

It is a touch rediculous that we would need to use an intermediary processor to handle the limitations of the GeoSCADA driver. The entire point of having a SCADA system is that it does these things natively.

geoffpatton
Commander
Commander
0 Likes
0
2011

Re: Geo SCADA MQTT Topic with nested data

From What I understand this is a first go at MQTT and they are going to enhance it. the problem with MQTT is that the messages can be structured any way you want but designing a can driver that is that flexible and still works otherwise with the software is a lot of development work. I am not sure they are ever going to go to that level.

 

I think they are going to add the the SparkPlug Payload B standard.

 

Contact your support channel and you can ask them to put in a driver enhancement request.

 

Steve Beadle who I tagged in my last message might have some information he can share on this.

AdamWoodland
Commander Commander
Commander
4
2011

Re: Geo SCADA MQTT Topic with nested data

The trouble(*) is MQTT is really just a transport mechanism like TCP. It is like saying we have a TCP telemetry driver when you're interested in DNP3 but can also consider Kingfisher, Modbus, etc, as its just packaged up differently but it is all the same TCP transport (ignoring choosing serial or UDP options!)

 

Most MQTT communications uses some form of JSON, but I also have to deal with CSV and raw text data and that's before I have to worry about how the data is encoded (e.g. base64) and then what the values mean (e.g. raw readings versus changes since previous reading). Until the various MQTT publishers use standards such as WITS-IoT and SparkplugB dominate, I've yet to interface with a device that uses either, then it is effectively a wild west.

 

As to how you can solve the problem... you can raise a support request for an enhancement, or...

 

You can compile the SparkplugB driver from https://github.com/GeoSCADA, Steve has made it open source. I believe there is a WITS-IoT driver around from Steve too, but not sure of the commercial arrangements of that. You could copy Steve's SparkplugB driver and make the relevant changes, its in C# using the Geo SCADA DDK.

 

Beyond that I do develop our own set of MQTT drivers to interface with the numerous other devices that my customers need to interface with and so its possible you could do the same. or chat to your Schneider rep and they can reach out to me to discuss further depending on which region you're in, or my colleagues elsewhere in the world. Some of our SIs could also help develop a driver if you already have existing agreements with them.

 

(*) I use the term trouble here, but it is also an advantage with significant flexibility which is one reason why MQTT is so popular in IoT

ash_aqx
Lieutenant JG
Lieutenant JG
0 Likes
3
2007

Re: Geo SCADA MQTT Topic with nested data

Thanks Adam,

 

That's some great insight. Definitely appreciate the complexities. 

 

Maybe a suite of seperate modules (parsers for JSON, CSV, etc that aren't necessarily tied to the MQTT driver) that can accept and process the output from the MQTT driver might be a simpler approach. Then the user could "build" an appropriate parsrer to suite the application.

 

Is it possible to intercept the output from the driver and use GeoSCADA scripting to process the payload?

 

I'll have a look at the driver source that you've linked, might be something there that I can work with.

 

Again, thanks for the response, very helpful!

AdamWoodland
Commander Commander
Commander
0 Likes
2
1959

Re: Geo SCADA MQTT Topic with nested data

I would love there to be a generic parser but it is beyond my software capabilities (I'm not a product developer, don't worry! 🙂 )

 

I did do something similar to the interception for some events that came in for a specific device, but in that case just shunted the event details to a structured text program as string/DT and the end user to do with the details what they wanted. The problem is there is no native JSON handling in the easily available server-side languages (i.e. structured text) so you don't really have anything to process the JSON unless you write your own .NET/Python/whatever code and then you're almost back at just putting it in the driver.

BevanWeiss
Spock
Spock
0 Likes
1
1951

Re: Geo SCADA MQTT Topic with nested data

I did look into the possibility of a generic parser a few years ago, and it seemed like it would be reasonably doable actually.

 

Essentially what it would need is to use some kind of grammar descriptor language, like ENBF

https://en.wikipedia.org/wiki/Extended_Backus%E2%80%93Naur_form

 

The issue then being that it's a bit harder for end-users / integrators to 'develop' because such a descriptor language is often quite disconnected from the language that it is being used to describe (i.e. if using it to describe HTML, it wouldn't look very natural to an HTML 'coder').

 

The real benefit is that it would provide a common platform for AVEVA/Schneider to develop more user friendly interfaces.

The way I'd envisage it would be there would be a drop down of 'common' formats (like xx separated values, or SparkPlugB, etc etc) but then there would also be a 'Custom Format' which would allow the raw ENBF form to be entered / edited.  Ideally the common formats would be implemented in ENBF also, so when transitioning from a common format to 'custom' the ENBF for the common format is shown... hence if just a slight tweak is required to a common format, the end user doesn't have to rewrite it from scratch, they just make the tweak required.

 

There's example parser generators around, like:

https://github.com/cwbaker/lalr   (does a modified BNF, rather than EBNF)

 

It would still only support 'character based' protocols, and I'm sure there would be other issues in getting the character data in the first place, but it at least makes the processing of the message data consistent and flexible.


Lead Control Systems Engineer for Alliance Automation (VIC).
All opinions are my own and do not represent the opinions or policies of my employer, or of my cat..
ash_aqx
Lieutenant JG
Lieutenant JG
0 Likes
0
1947

Re: Geo SCADA MQTT Topic with nested data

Definitely,

A generic string parser of some description would open up a whole range of lesser used protocols and formats for integration.

Something like this would be super convenient and allow developers / integrators a much shorter path for IoT devices.

ash_aqx
Lieutenant JG
Lieutenant JG
0
1887

Re: Geo SCADA MQTT Topic with nested data

Hi Everybody,

 

Thanks for your guidance and feedback on this topic point.

After the suggestion from @AdamWoodland I cloned and built the Driver-BasicMQTTClient from this repo https://github.com/GeoSCADA.

 

The good news is that this works perfectly with The Things Stack, providing your region has uplink fields enabled. I used the following topic descriptor: "my-app-id/devices/my-dev-id/up/<field>"

 

You can check if your region supports it by checking out this link: https://status.thethings.network/