Aggregate by string concatenation in SQL (Geo SCADA 2020)

[rlao]

The SQL documentation only lists a few basic aggregation functions to be used with a GROUP BY statement. Unfortunately, the summarization I am after is to concatenate all the rows into a single string which does not appear to be supported.

Is there a way I can achieve this using other SQL expressions? Table in question is a dataset of <500 entries.

Thanks

[BevanWeiss · Accepted solution]

This isn't strictly a Geo SCADA Expert limitation so much as it is a limitation with the SQL-92 standard implementation used by Geo SCADA Expert (with some slight customisation for GIS / indirect operations).

You'll generally find that standard aggregates will only be operations that are commutative.  That is the order of the arguments is independent to the result.  i.e. if I have three values, RowA, RowB, and RowC.. and do a SUM(Row...), AVG(Row..), MIN(Row..), MAX(Row..), SDEV(Row..), VAR(Row..) it is the same regardless of the order of the operands.  i.e. if I swap the RowA and RowC values the aggregate is identical.  This is important, because in SQL you are not guaranteed any given ordering of the result set unless you specifically request it with an ORDER BY clause.  And this is only guaranteed to operate on the actual result set provided, not any intermediate data required to generate the result set.  As such CONCATENATE(...) simply can't be a valid aggregate, since CONCATENATE(RowA, RowB, RowC) is different than CONCATENATE(RowB, RowA, RowC).. and hence it's simply too problematic 

In general it's not well supported within SQL syntax, even proprietary vendor syntax.

MS SQL Server only recently supported it with STRING_AGGR.

The most common way to handle this in other SQL-like products would be to use a CURSOR, and index through the result set using a variable to hold the concatenation result.  Unfortunately Geo SCADA Expert doesn't support such use of CURSORs.  It also doesn't support recursive Common Table Expressions, which is an alternative method to support similar.

You could try something like this: https://stackoverflow.com/a/17308646/2535822 

But as noted in the stackoverflow post, CTEs weren't supported by SQL-92, and so it's very unlikely to work in Geo SCADA Expert. 

I would say that your best option here may be the use of ST Logic.  Which obviously imposes some additional limitations that wouldn't be in an SQL pure implementation (i.e. you'd want it to be calculated ahead of time, you'd need to store it somewhere, and due to both of these, it couldn't be as dynamic or adhoc user driven).
You could do the concat within Mimic scripting if you wanted more of the adhoc user driven stuff, but it would be cludgey to couple it back to the remainder of your SQL results.

Something like
ServerQuery(... to get all the rows that you want to concatenate)

for each row in Serverquery result

  strConcatenated = strConcatenated + "," + row.columntoconcat

endfor

ServerQuery( "SELECT " + strConcatenated + ", col2, etc FROM stuff WHERE thing=thatThingILike" )

If you wanted it to be an embedded query shown, you could have the script execute on Mimic_Navigate or something like that, and instead of the second ServerQuery it would write to the SQL property of the EmbeddedQuery object

i.e. Layers("Layer").Items("Item").SQL = "SELECT " + strConcatenated + ", col2, etc FROM... blurgh"

It still wouldn't be super dynamic though, the concatenated result wouldn't update automatically..

[rlao]

Thanks Bevan.

Yes you are right this seems to be more a SQL92 limitation, but I was hoping there was something vendor-specific that I had missed. However, you raised a good point regarding commutation that would probably make such an aggregate too messy to implement.

My alternate idea is the Mimic_Navigate solution you mentioned and loop through the query row to build the string. The lack of dynamic update while on the mimic page is not too much of a problem for me as the dataset shouldn't change that frequently over time. I just wanted to see if there was a more elegant solution I was missing.

[BevanWeiss]

In regards to SQL standards, as mentioned Geo SCADA Expert is SQL-92 (with some minor proprietary additions).

Since this 1992 standard release, the following more recent standard revisions have been released:

• 1996
• 1999
• 2003
• 2006
• 2008
• 2011
• 2016
• 2019

Many of the latter standards have quite niche enhancements, i.e. 2003 introduced XML, 2016 introduced JSON

The 1999 standard however introduced Common Table Expressions (CTEs), which would have helped you to perform the query you were after.

I'm unsure exactly how coupled the current SQL Query Parser is to the core of the Geo SCADA Expert system, but it might be worthwhile raising a feature request for an update of the supported syntax version (i.e. from SQL-92 up to SQL-99 or perhaps even more recent versions with JSON support etc).