Logic Execution Methods: 'Input Processed' vs 'On Interval with Trigger'

[rlao]

I'd like to know more about the performance difference between the two execution methods mentioned in the thread title.

To my understanding, 'Input Processed' causes the script to execute when it detects an change in one of its VAR inputs in the program, while setting a Trigger causes the execution when it detects a change in any database object that you specify in that property field.

However, since Triggers are used as part of 'On Interval' execution, setting a Trigger also requires setting a timed interval for the program to periodically check the value of that Trigger input, while you don't need to set any such interval to check inputs for 'Input Processed' - it seems to fire off the program instantaneously on input change. Would this imply that, compared to 'Input Processed', there is some extra performance overhead from using 'On Interval' with Trigger due to the need to explicitly query the trigger object on a periodic basis?

As an example, say I need Program A to execute when the value of Input B changes but Input B isn't actually used in the program. The obvious way would be to use On Interval execution and set the Trigger to look at Input B. However, I would need to specify a period for the program to explicitly check that input.

However, I could also declare Input B as a VAR in the program itself and not actually use it at all. Then I can set the Execution Method to 'On Input Processed' and the program executes as soon as Input B changes without having to specify a timed interval for it to check that input.

Would the second method be more efficient in this case in terms of logic execution?

[BevanWeiss · Accepted solution]

Hi @rlao ,

First off, are you able to change the label you've assigned to this from 'scripting' to 'logic'.  They are drastically different environments within ClearSCADA / Geo SCADA Expert, and it is vitally important not to confuse the two.

So...

'Input Processed' results in the execution of the Logic being queued every time an 'Input Processed' event occurs.  Depending on the driver associated with the inputs, this might be quite a few times in very quick succession.

A good example is with DNP3/KF drivers.  If you use the LoggedValue property of a point within your logic, and a bunch of historical records are received, then your logic will be queued for execution for every single historical record received.  This might be several hundred (theoretically it could be infinite...) times.

However, if you use 'On Interval with Trigger', then your execution will only occur AT MOST at the 'On Interval' rate.  But this will only occur if the property that you have linked to in the 'Trigger' has changed from the last execution of the logic.  This is similar to the 'Input Change Detection' tick box, but specific to a single Object Property (that may not actually be referenced within the logic itself).  In our example from before, if we had the Interval set to 1s, and we received all those millions of LoggedValue changes, our logic would execute ONCE... not millions of times.

In your example, it would depend on the rate at which Input B is changing.  If Input B is updating more frequently than your Interval, then having it in the Input list and using 'On Input Processed' would be more intensive, since the logic would be executing more frequently than if you had of used 'On Interval' and had Input B in the Trigger property, however you would also not have executed your logic for EVERY change of Input B, only the ones that happened to be caught by the Interval / Interval Offset periodicity.

There are certain situations when it's obvious that you need one or the other.

a) If you need the logic to run EVERYTIME that Input B changes, then you need Input B to be within the Inputs of the Logic Routine, and to use 'On Input Processed'.

b) If you need the logic to always run at some interval, then you DO NOT want to use the 'Trigger' or the 'Input Change Detection' properties.  Likewise, you also probably want to configure the Update Outputs property to 'All Outputs (Always)'.

c) If neither of the above apply, AND you want to minimise your system load, then you should use 'On Interval' with a nice long period, enable the 'Input Change Detection' and the 'Trigger', and set Update Outputs to 'Changed Outputs Only' (assuming your logic is non-trivial).

I've never really gone with option (c), because there's other more beneficial ways to reduce the system load than this kind of thing.  Remember that pre-mature optimisation is foolish.  Make sure that you have some kind of statistics to back up such a performance 'optimisation'.

[rlao]

Hi Bevan,

Thanks for the detailed answer, I have corrected my label to 'Logic' as you have advised.

The distinction of having the logic execute for each input change regardless of frequency is not something I had considered and, for the application I was looking at, this would be a necessary requirement and would trump any efficiency considerations that I initially had in mind. So it seems pretty clear now due to this limitation that 'On Input Processed' is the way forward for me.

Just for my own curiosity, do you know through what mechanism the 'On Input Processed' logic get queued due to an input change? It sounds like some publish-subscribe messaging to me. Compare this to 'Input Change Detection' and 'On Interval with Trigger' which both require specifying a timed interval for the logic driver to explicitly check the input(s) and determine if the logic is to be executed.

Hopefully, I am using the correct terminology here...

[BevanWeiss]

The exact details of the Logic 'On Input Processed' 'engine' are a bit hidden, since it's not really exposed in the Simple DDK, or in many other locations.

The .NET API does have the concept of a 'Register For Property Updates', so I believe it would be based upon this concept. The Logic Routine object when 'compiled' would subscribe to the Properties it can identify as inputs from the VAR lists, this would exclude 'tricky' combinations line Indirect addressing, or SQL queries, or similar.  The subscription event would be quite simple, take a copy of the input values as at the time the Input Processed was fired, and queue the logic for execution.

The 'On Interval' would be similar queuing logic (since they both share the ability to have the Input Change Detection / Trigger associations), but would only be executed on a timer (or perhaps more accurately a schedule).

Here's the .NET API information for the Subscription 'stuff'

[Anonymous user]

Input Processed is the same mechanism that triggers the ProcessCount and ProcessCountTime OPC tags. This is the explanation from a colleague that I quote:

The logic "On Input Processed" mechanism is only supported by the following types of object:

• Points - means point was updated, this includes 'hand controlling' an internal point, or applying/releasing an override. Only current and logged point updates are processed.
• Accumulators - means accumulator was updated, reset, initialised or value was set.
• Variables - means value of variable was set, or for array variables an element of the array was set.
• Random Generators - means it generated new random values.

Other types of object, including outstations, are not supported and therefore won't trigger the execution of the logic program.

[BevanWeiss]

Thanks @Anonymous user ,

I don't suppose you'd know if there is a difference in the Logic 'On Input Processed' "subscription" and the .NET API reference that I provided?

My expectation has always been that the RegisterForPropertyUpdates method would be limited to the same things that you mentioned, and wouldn't truly be for any property.  Although I haven't actually tested this to confirm, and the .NET API help for this method doesn't mention any limitations.

[Anonymous user]

I don't know for sure, but I wouldn't have thought it would be related. One is a server side mechanism and relates to the object, the other is a client method and relates to property/ies.

RegisterForPropertyUpdates should be for any database field.

[AndrewScott]

Logic 'On Input Processed" and RegisterForPropertyUpdates method are not related.

RegisterForPropertyUpdates uses the interest mechanism, the same as OPC-DA which is used for mimic animations and will work with any property on any object.

As noted above On Input Processed can only be used with certain properties of selected objects.

[rlao]

Thanks, definitely good to keep in mind the limitation of 'On Input Processed' only being able to be used with certain properties and not all.

[BevanWeiss]

@AndrewScott I assume this design decision was made for performance reasons?

(that the 'On Input Processed' handling is more 'efficient' overall than the Interest mechanism)

I know that the items that 'On Input Processed' can handle (i.e. not being all properties) has tripped up quite a few engineers.

if it wasn't going to kill database performance, then I would have thought having consistent behaviour here would have been the preferred solution.

So that 'On Input Processed' could be renamed as 'On Input Updated' and having the logic triggered if any of the input properties are indeed updated.

[AndrewScott]

@BevanWeiss'On Input Processed' and 'On Interval' are designed for different use cases, not for really for performance reasons. Which is more efficient will very much dependent on the specific program and its inputs.

'On Input Processed' should be used where the program must be executed each time the inputs are processed (e.g. a point is updated with new data from the outstation). For example if the program is doing some kind of accumulation calculation it must run once with each new value/quality/time for the program to calculate the correct result.

'On Interval' is basically a periodic polling mechanism.

Consider a data logger where data is retrieved once per day. This may get 100 updates for an individual point all in one go so the updates will be processed in the space of a few milliseconds. With 'On Input Processed' the program is run once for each update but 'On Interval' would likely only see the final update.

[BevanWeiss]

Thanks @AndrewScott, but I was purely talking about why the 'On Input Processed' differs from the 'Interest' system.

Why are logic routines not able to be triggered for execution when an arbitrary property changes, but only when certain (driver dependent) 'Input Processed' actions occur?

It's often incredibly desirable to have logic execute when a property changes, like when an Outstation / Channel goes from Healthy to Failed/Disconnected.  But as noted previously, this doesn't count as an 'Input Processed' event, and so won't trigger execution of logic configured for 'On Input Processed'.  However it seems that if I used the .NET API and had called RegisterForPropertyUpdate on the .State of the Outstation/Channel, I would indeed get an event triggered for this situation.

So I'm just not sure why the Logic Execution trigger doesn't also use the 'Interest' system.  It would seem to make more sense from my perspective (and most other ClearSCADA engineers I know).  UNLESS, it was considered that it would be overly detrimental to database performance...

[AndrewScott]

---

@BevanWeiss wrote:

Why are logic routines not able to be triggered for execution when an arbitrary property changes, but only when certain (driver dependent) 'Input Processed' actions occur?

@BevanWeiss Logic programs are able to do this using the Input Change Detection option in the programs configuration. See Core Reference > Coding > Logic > Define whether a Logic Program Executes when the Value of an Input Changes page in the help.

The fundamental difference between 'On Input Processed' and either 'On Interval' or RegisterForPropertyUpdate is that with 'On Input Processed' the logic program is queued for execution using the current values of the inputs as part of processing the point update, but with the other two they are just polling the database periodically looking for changes (this polling is internal to the server).

[BevanWeiss]

---

@AndrewScott wrote:

---

@BevanWeiss wrote:

Why are logic routines not able to be triggered for execution when an arbitrary property changes, but only when certain (driver dependent) 'Input Processed' actions occur?

@BevanWeiss Logic programs are able to do this using the Input Change Detection option in the programs configuration. See Core Reference > Coding > Logic > Define whether a Logic Program Executes when the Value of an Input Changes page in the help.

---

@AndrewScott As you mentioned previously, when the Input Change Detection option is used, the execution (may) only occur at the Interval rate, not at the rate at which the Input property updates.

And when configured for On Input Processed this does not include all input properties (only a select few).

So my question is:

Why does On Input Processed not work for ALL properties in the way that the .NET API RegisterForPropertyUpdates (aka the 'Interest system') works?

[Anonymous user]

Andrew will probably have a better answer, but I imagine it is because it hooks into the code path for drivers updating points in the database.

[BevanWeiss]

@Anonymous user yeah, I expect you're right that it ties into the driver code path, although that seems like it would be quite janky code flow if so.

It would seem (to me) to make a whole lot more sense if 'Interest' registration occurred on compile of the logic routine, and then the driver code simply updated properties in the database, where the database is then responsible for updating all of the 'interested' subscribers.

I expect that the property update doesn't actually trigger on EACH property update, for example to go back to my DNP3/KF advanced driver example from much earlier.  The LoggedValue updates are likely sequentially processed by the driver, but then batch processed by the database.  So whilst the driver sees these as x million individual records, the database 'interest' only sees the one LoggedValue change (to whatever the most recent LoggedValue record was from the driver batch).

This was where I mentioned that the performance likely comes in.  If the interest updates occurred for EVERY LoggedValue update this would be quite a big processing impact, even if it does push the servicing of the interest advise work into a separate lower priority (non-core) thread.

[AndrewScott · Accepted solution]

The 'On Input Processed' feature is built on top of a publisher/subscriber feature provided by the database, but there are only four specific publishers, the main one is the point processor which process all point updates from drivers, hand control, master override etc.

The interest feature scans the entire database every 500ms (by default) looking for objects that have changed (either data or configuration). Each object has a set flags indicating whether or not it has changed. Clients can subscribe to specific objects they are interested in (e.g. the rows in a list in ViewX or the tags on a mimic) and an event is sent to those clients to notify them of the change. Some clients (e.g. OPC DA and RegisterForPropertyUpdates) have additional filtering in the server to check which properties have changed against those the client is interested in.

Input Change Detection in logic is similar to the interest mechanism and periodically checks the programs inputs for changes and executes the program when a change is detected.

Points are processed by the database in batches of updates received from the drivers and therefore 'On Input Processed' will see every update but the interest feature and logic input change detection will only see the last update for each point in a batch.