[Imported] SQL call ins into STP or LOGIC programs.

[sbeadle]

>>Message imported from previous forum - Category:Scripts and Tips<<
User: ROVSCADAENGINEER, originally posted: 2019-02-07 22:39:52 Id:364
I am wanting to call in and manipulate some data in a logic program to calculate the runtime on pumps for the last year and sum that amount to then be used to set the maintenance alarm setpoint on the system. This requires querying from a data point. I do not want to manipulate the database itself. I just want to call the data in run a calculation on it and then assign that information to another digital point. The part I am stuck on is how to do this in STP by using SQL strings.

Any help or guidance appreciated.

[sbeadle · Accepted solution]

>>Responses imported from previous forum

Reply From User: adamwoodland, posted: 2019-02-10 22:37:14
Check the help, Core Reference Coding Logic ST Programs Using an ST Program to Call an SQL Query

Reply From User: ROVSCADAENGINEER, posted: 2019-02-12 01:22:37
Ok thanks ADAM. So I have generated a table and is working fine but the table when prompted with an OR statement takes an extremely long time to load. I am wondering if or why this is occurring. The code is below.

SELECT
RECORDTIME, SEVERITY, SOURCE, MESSAGE, USER, CATEGORY, FOREGROUND, SEQNO, COMMENTNO, ID
FROM
CDBEVENTJOURNAL
WHERE
SOURCE LIKE '%RTU.DI.01'
AND ( RECORDTIME BETWEEN { OPC 'H-6D' } AND { OPC 'H' } )

AND SEVERITY = 'Critical'

OR
SOURCE LIKE '%Overflow Float'
AND ( RECORDTIME BETWEEN { OPC 'H-6D' } AND { OPC 'H' } )

AND SEVERITY = 'Critical'

ORDER BY
"RecordTime" DESC, "SeqNo" DESC, "CommentNo" ASC

I have tried rearranging the code to run quicker in this way. The original code which was working but with only one LIKE function, as soon as a pulled the OR function the load time was extremely long. Is there another method that I am missing that would do this allot quicker. I am wondering because there are different name points for the digital points on the system.

Reply From User: adamwoodland, posted: 2019-02-12 03:35:47
You should run the query in QueryPad, that will give you some information on the performance and where the inefficiencies are. With those SOURCE LIKE conditions you are likely causing the processor to scan entire tables rather than using an index lookup and that is likely the cause of the significant execution length.

Also, CDBEventJournal is a disk bound table and you shouldn't really be using them in logic unless you restrict the query extremely tightly. Logic is single threaded so if you run a long query the whole logic thread stalls. My rule of thumb is that any query (or total of all queries in a logic program) that takes longer than 100mS shouldn't be in a logic program, but you also need to take into account execution frequency and number of programs.

See https://community.exchange.se.com/t5/Geo-SCADA-Knowledge-Base/bg-p/geo-scada-knowledge-base for an overview of how logic works.

Reply From User: ROVSCADAENGINEER, posted: 2019-02-13 04:36:08
Ok cool thanks adam. So I am trying to narrow it down. is there a way I can hone in on the source name from the CDBobject digital point base and then run the SQL to query that table.

Say with a JOIN function or something.

Is it possible to query a table with all the points. with specific characters in the full source name such as '%Overflow Float%' and then run the query to check based on the history points from CBDhistoric of those points? how would you join this. Is this something that you would need to do in a logic program calling in queried points.

Sorry I know what I am saying may not make much sense. I am relatively new to SQL. just trying to wrap my head around it all. Thankyou for your help this far.

Reply From User: sbeadle, posted: 2019-02-13 09:20:13
There is a guide to join optimisation here:
https://community.exchange.se.com/t5/Geo-SCADA-Knowledge-Base/bg-p/geo-scada-knowledge-base
It's complicated!
If you are 'inside' Logic, than querying history is usually not a good move, unless you can guarantee that the search is tightly specified. (e.g. a join on object id and a very short time window).
Name/field searches using '%xxxx%' are poor performers. A search based without wildcards (e.g. ParentGroupName = 'a.b.c' is fast).

Reply From User: du5tin, posted: 2019-03-06 17:48:21
How you order the WHERE clause has a bit of an effect on the speed. If you re-order your WHERE clause you might get some improvement in speed. The goal is to use wide sweeping restrictions first (by severity and time) to reduce the number of rows searched by the wildcard statements. See here:

SELECT
RECORDTIME, SEVERITY, SOURCE, MESSAGE, USER, CATEGORY, FOREGROUND, SEQNO, COMMENTNO, ID
FROM
CDBEVENTJOURNAL
WHERE
SEVERITY = 'Critical'
AND RECORDTIME BETWEEN { OPC 'H-6D' } AND { OPC 'H' }
AND (SOURCE LIKE '%RTU.DI.01' OR SOURCE LIKE '%Overflow Float')
ORDER BY
"RecordTime" DESC, "SeqNo" DESC, "CommentNo" ASC

There are some other improvements too. Like sbeadle mentioned, how you search your strings changes the speed immensely.

This runs very fast, only one expect result. DB can limit the search results.
SOURCE LIKE 'SomeName'

Runs fast because the beginning has limited options to search. In other words, the DB doesn't have to provide or search all possible results, only the ones that match the beginning part of the string.
SOURCE LIKE 'SomeName%'

Runs slow because the DB has to effectively search all possible rows to match the string.
SOURCE LIKE '%SomeName'

Reply From User: ROVSCADAENGINEER, posted: 2019-03-12 04:11:44
Thanks for the useful tips du5tin!