[Imported] Efficient Embedded Symbol Parameters

[sbeadle]

>>Message imported from previous forum - Category:ClearSCADA Software<<
User: nminchin, originally posted: 2018-12-04 02:01:24 Id:330
All of the ClearSCADA projects that I've seen in my 9 years working with it off and on, have all used the same cumbersome approach to mimic parameters. That is, all symbols have defined parameter groups, and each group contains every conceivable object property that might be needed for each object in the parameter hierarchy that will come from a database object.

E.g.
 3a/kxnfzqsci17d.png "")

This can be replaced with a far more simple approach:
 dy/j8shj3xvv8ld.png "")

Then in animation expressions, you can indirectly bind to the object properties you need using:
["Parameter:Group.Value.FullName" + '.CurrentValueFormatted']

Using this method has several advantages:
- Far simpler parameter creation
- No reliance on the initial list of properties being correct. It's not a major event if you need to reference another tag property in the future.
- Far far simpler to dynamically exchange out the tags being passed into the symbol. For example, if you want an analogue display to show one tag on condition, and another tag on another condition, then this logic is done once to swap out the FullName passed in, rather than an expression to swap out every single property. Seen this lots of time and it's so frustrating to work with and work out what's going on...

The only potential downside to this approach that I can see might be a performance hit due to using indirect bindings. I've tested this however with 50 symbols embedded on a page using this approach, and haven't seen any noticeable lag.

I'm curious, is it just the projects that I've worked on, or is this parameter-per-property approach how most other projects have been done as well? Does anyone use the FullName/Indirect binding approach, and have they had any issues with performance?

Attached file: (editor/3a/kxnfzqsci17d.png), image.png File size: 28935

Attached file: (editor/dy/j8shj3xvv8ld.png), new symbol.png File size: 12324

[sbeadle · Accepted solution]

>>Responses imported from previous forum

Reply From User: dmercer, posted: 2018-12-04 03:23:31
I've used this in the past.

Indirect references do cause a performance hit, but I've only seen it cause real problems when there was some latency between the client and the server.

I like to try to put in all of the parameters that I am likely to use, and then also include FullName and Id as backups in case I need to add more functionality later and don't want to go back over every place that it's been embedded.

---

In your example above you can go further if the folder structure is always the same, by removing the limits group, and using ["Parameter:Group.Value.FullName" + '.Limits.High.CurrentValueFormatted']

---

A similar trick is that mimic scripts can read parameters and read and write to animation values. You can use this to get properties from other tags which are referenced by the tag that passed to the embedded mimic. If you want it to continually update then you have to put a loop inside the Mimic_Load() function.

Reply From User: adamwoodland, posted: 2018-12-04 03:27:40
The issue with indirect tags is scaling. On a single mimic/head you probably won't notice anything, but multi-user systems with multiple mimics open on each head expect to see a significant load start to appear on the server the clients are connected to.

This trips up a number of customers I work with causing performance issues. Defining animation using the 'cumbersome' approach may mean a quicker development but means less efficient server processing over the many years the system is in use.

If anyone chooses the path of indirect tags make sure to fully read the help, for example "Core Reference Coding Expressions Indirect Tags", for the impact and the actions to take to minimise the problems.

Reply From User: dmercer, posted: 2018-12-04 03:28:05
Another similar thing, which does cause a performance hit, is to do queries inside an expression. eg:

['?SELECT TOP(1) Id FROM CDBObject']

This should definitely be used sparingly, because I have seen it bring a system with a very powerful server and extremely low latency to its knees by having maybe a few hundred of these in a mimic.

Reply From User: nminchin, posted: 2018-12-04 03:45:17
+1 for the very noticeable effect of a select query inside of animations - I've already experimented with those and put them on my no-fly list!

Point taken RE performance hit. I did suspect that this would be the main reason this approach wasn't overly used, and why it isn't viable in the long term.

I guess I'll continue to use the ~~cumbersome~~ most server-efficient method 🙂

Reply From User: sbeadle, posted: 2018-12-04 08:37:16
With great power comes great responsibility ...

Reply From User: sbeadle, posted: 2018-12-04 08:45:39
Also note that indirect tags have different impact levels on embedded mimics when choosing either 'shared with embedded' or 'non-shared' property.

Reply From User: JesseChamberlain, posted: 2018-12-07 00:55:09
These are some of the most common causes of poor mimic performance that I've come across, so I'll write some explanations for anyone that comes across the thread.

**Indirect animations** [square bracket syntax] causes multiple round trips each mimic refresh and is very inefficient if the 'shared with other embedded mimics' option is left ticked.

You can identify this problem from DB logs if you use [CS File Analyser](https://community.exchange.se.com/t5/Geo-SCADA-Knowledge-Base/bg-p/geo-scada-knowledge-base "CS File Analyser"). The analysis will come back with something along the lines of "multiple add/remove single items".

**SQL animations** should be used **very** carefully. They are not cached in any way and do not time out, so for one such animation that is on the header of every mimic (for example) the load on your server becomes: SQL execute time * mimic refresh rate (default 1s) * client count.

If SQL execute time mimic refresh rate, they will keep backing up which will lock the server. I've seen this result in mimics not updating at all. The calling card in DB logs is [QueryTagManager], which isn't currently picked up by CS file analyser but in default logs will be pretty easy to find manually.

The workaround is to use SQL vars in ST logic, which should be declared as VAR NOCACHE, and run at a reasonable interval (does it really need to be quicker than 10s?). This will cache the SQL result, and logic can time out. The load is now: SQL execute * Logic execute interval; ie independent of client count.

**Alarm summary tags** are the built in tags like AlarmAcceptedCount or AlarmState when used on a group or instance. These are calculated at the subscription rate, so if you use them in an animation, again they are calculated at mimic refresh rate. For an object, this is fine, but when they are summarising a lot of objects in a group, the server load: mimic refresh rate * calculation time * client count - can get problematic. I've seen $Root.AlarmAcceptedCount bring systems to their knees.

If you have them parameterised for every embedded mimic as seen in Nick's screenshot, you might even have subscriptions you're not using.

If the calculation time of the alarm summary tag mimc refresh rate, again they get backed up and don't time out. We call this OPC thrashing. In DB logs this can be identified by OPCDA refreshes that you would expect to be periodic (ie at mimic refresh rate) all being executed at once as fast as the server can do it.

Again the solution is to cache the tag in ST logic.

And please - the simpler you make your mimics, with minimal scripting and less levels of embedded mimics, not only the better they will run, but the easier they will be to troubleshoot, and the less time the poor support engineer will have to spend pulling apart your mimics looking for the above. Not everything has to be crammed into the one mimic using pretty layer visibility and six levels of embedded mimics to create tabbed interfaces.

Reply From User: sbeadle, posted: 2018-12-07 08:57:42
Thanks Jesse - plenty of wisdom there. 🙂 🙂

[Tanquen]

I'd like to try this but the second image is missing.

["Parameter:Status.Position"+'.CurrentValueFormatted']

What value would need to be in the Status.Position parameter?

Was wanting to use one parameter to complete the expression for a number of display parameters like, .AlarmState, .Background, .Blink and so on.

________________________

Ok, just ("..Tags.Position.FullName") seems to work.

[Tanquen]

I changed them all and saved. The values look good but now I can't change any of them and get an "Unexpected symbol" error popup unless I remove the square brackets. Even if I don't change anything.

[BevanWeiss]

It's a TRAP!!! don't do it unless you really understand what it does.

This is syntax known as an Indirect Tag.

What needs to be within the [] is a string which refers to the fullname of an OPC Property.

So "..Position" doesn't make much sense. 

["..FullName" + '.{PROPERTYNAMEHERE}'] would be the 'typical' way of doing it.

What Nick (Minchin, no relation to Tim) had proposed was doing away with a bunch of Embedded Mimic Properties, and just supplying the FullName of the base object, and then using Indirect Tags to retrieve all of the things you actually wanted within the Embedded Mimic.

There are reasons that it's bad:

1. The performance of this isn't optimum (I haven't seen horribly bad issues from it, but I have heard anecdotes).  Certainly it should be apparent that it cant be as well optimised as could fixed references.  and if the input value changes lots then the performance CAN be atrocious (in particular if using 'Shared with other Embedded Mimics')

2. It hides data accesses.  So perhaps you don't want to actually have the embedded mimic display CurrentValueFormatted... you want it to display 'Chicken' if the value is 1, or 'Donkey' if it's 2... well you can't animate the 'CurrentValueFormatted' Property into the embedded mimic since it no longer exists, there's only 'FullName'..

3. Similar to 2. it hides data accesses, so if you do a 'Display References' it will not show up, because it's Indirect... it's not known at 'Design Time' what it's actually referencing.  It only knows at Runtime, when it evaluates the inner string and gets ['Water.WaterSite01.Pump 07.Running.CurrentStateDesc'] then it can go... "ohhh... you're referencing Water.WaterSite01.Pump 07.Running.... cool, I'll get that for you"

It does guarantee some additional display latency also, since ViewX needs to do more 'round trips' to the database.

1. Fetch Mimic

2. Fetch Embedded Mimic

3. Fetch Data values for 'tags' (referenced OPC Properties)

4. Evaluate Expressions involving Direct Tags from 3.

(assuming no Indirect Tags on top level mimic)

5. Pass property values into Embedded Mimic

6. Evaluate Expressions involving Direct Tags / Properties

7. Fetch Indirect Tag Data values for 'tags' (referenced OPC Properties)

8. Evaluate Expressions involving Indirect Tags

Steps 7 and 8 are ONLY required if you are using Indirect Tags, hence they result in performance impacts.

Extra calculations, extra OPC subscription events (if you don't untick the 'Shared with Embedded Mimics' this can REALLY kill a system), extra network traffic, extra display latency.

They can be useful... but you need to use them for the right raisons.

[Tanquen]

Position is just the name of the full name parameter and it works I just didn't know why it was giving me the error message after testing and seeing it all work. Turns out it didn't like the defaults in the parameters that were already there when I set the new expressions up in the first place. After removing them it let's me make changes. I thought the parameters were corrupt or something.

Like the folks above just trying to figure out a way not to have unique parameters for every field and have to manually enter them all the time. Again if there was a simple search and replace. A set of defaults for the parameters that got passed on when you place the object would be nice too. It's also not fun that it deletes all your expressions on all your objects if you remove the parameter they're using. I'd rather just leave it like a bad tag with the red x or something.

[BevanWeiss]

Embedded mimic parameters are definitely not the nicest situation.

If you structure them as per the structure that your 'hierarchy' will have then it's much easier however.

But yeah... any changes to the hierarchy etc and the animation expressions get very annoying to fix up.

The fastest way is to normally drag/drop an object onto your embedded mimic and use the Auto {Something} (I forget what it's called.. 'Configure', 'Assign'.. something like that) option.  If you have the properties defined right then it ties up all the animations based on that object dragged in.

[Hardin4019]

Is .Position a digital point? If you right click on the point the Database and locate it in the OPC database, then right click on the point in the OPC Data explorer, do you see .CurrentValueFormatted?

I have run into issues like this before that have forced me to create either a new mimic that is specific to a point type (digital or analog), or DNP3 vs Modbus vs Internal points, etc.

[Tanquen]

It was an analog point and is working.

It looks like you can't have spaces in you tags names as they need to match the parameter and it has a WARNING! that if you use a space it has to be accessed using the GetParameter function in a script. 😞

All the other tags in the system have spaces. So I can have the drag and drop the tags group but the tag names will look funny, Any that show up in the event/alarm list will look off. 😞

Ok, so it works with spaces but if you need to add some script you'll need to use the GetParameter function?

[BevanWeiss]

@Tanquen You can't have spaces in your Parameter Name and have them 'just work', there are a bunch of other restrictions too... essentially there isn't a 1:1 correspondence between what a Parameter Name is allowed to contain and what an Object Name is allowed to contain (from memory it was an issue with ' ', "(', ")', '\', '/', '~', '!'.. which are allowed in Object Names, but not in Parameter Names)

I think I have reported it as an annoyance (i.e. a Feature Request), since it can definitely be problematic for automatically assigning Parameters.  It would be helpful if you reported to Tech Support and asked to upvote my PDEV (I can't remember the number, but I'm sure they'll be able to find it)

[Tanquen]

The part that I need so far works with the spaces in both the parameter name and object name. When I drag a group with tags in it onto an object with the matching parameter names and it works. Will it break something else latter on?

[BevanWeiss]

Perhaps I'm remembering the space issue wrong.. When you created the Parameter Name with the space it warned you that you'd need to use scripting to access the parameter right?  I think that was the issue... I didn't want to go to scripting for such things, I had lots of animations relying on the parameter value, and avoiding scripting is a bit of a 'rule' I try to have...

[AdamWoodland]

You might be thinking about the ViewX system name and spaces. Spaces are allowed but having a space in them gives you a warning as it can break scripting.

It doesn't block you creating it though, as not allowing it on systems that had a space in the system name for years breaks even more.