OPC-HDA Server not providing data in a timely fashion

[du5tin]

We have a problem with a large main-standby system... 145,000 point system. OPC client is connected to the standby for server load. OPC Client is not receiving timely updates from the Geo SCADA OPC-HDA server on the ~30,000 tags. When we add new points to ClearSCADA it takes sometimes hours or days for the new data to be presented to the OPC-HDA client. 

Anyone seen anything weird with the OPC-DA and OPC-HDA servers in Geo SCADA? 

[BevanWeiss]

I suspect this is likely to be more of an issue on the client-side.

The GeoSCADA trends etc use OPC-HDA, and I've never seen anything like this.

I've also got a few customers that use OPC-HDA for their OSIsoft PI interface nodes, and have never seen this behaviour (nor had it reported by others).

When you add new points to your GeoSCADA system, how does the OPC-HDA client even know about the new data streams to start requesting data for the new topics?

So..

1. What is your OPC-HDA client using to determine what new topics to poll data for? (and how frequently is it doing this)

2. With what frequency / interval is your OPC-HDA client polling each of the topics (data streams) from GeoSCADA?

The Server Status tool has some info on OPC-HDA subscriptions.  You might want to have a look at this.  It unfortunately doesn't indicate how often your OPC-HDA client is browsing for new topics or anything like that.. but it would tell you which items are currently being requested by the OPC-HDA client.

[du5tin]

Hi Bevan, 

The OPC client is Osisoft PI. We are adding new tags into the PI database first. These topics are not updating right away after they are added. 

The OPC client is scanning once every minute right now. PI Interface is on a different machine than the ClearSCADA server. 

We are embarking on a work flow to evaluate, eliminate, and update stale or 'bad' tags in PI. Apparently there are a lot with missing ItemIds. Once those are decommissioned or properly segregated the 'stale' tags that have not updated in some time are going to be shuffled to their own interface to see if the problem moves with those tags or if the problem goes away. 

This reminds me of troubleshooting OPC with ClearSCADA as the client and getting data from an OPC server. If there are too many bad tags the response from the server is often incomplete and some data will update fine and some will only update on a forced "give me everything" refresh of the OPC client connection. Its just ClearSCADA is now the server and PI is the client. 

[BevanWeiss]

So you add the tags into OSIsoft PI Historian, and then you perform a manual restart of the interface to force it to update its own tag list?

I've never really had to look into the mechanism that the PI Interfaces use to update their tag lists.. but every time I've cared about it happening quickly, I've just restarted the interface.  Generally I like to turn off the shutdown record insertion also (since it's annoying getting those data points recorded).

I had previously written a tool that would synchronise ClearSCADA and OSIsoft PI (both in terms of PI Historian data points, and also AF templates / elements).  This generally kept things nicely in check, so there weren't lots of bad tags in the PI Interface Node... that might bias my experience however.

With lots of bad tags, it might cause other issues like slowing down the process of tags being put on scan...

Unfortunately it seems like a chicken-egg situation... your best option is to tidy up the bad tags (it should be possible to do an export from PI for the source field, and then run that against your ClearSCADA DB to identify items that don't exist.  Then you could just take them out of service in PI historian until they can get corrected.

Other than that, diving into the logging will get painful, but might reveal something there... I'd start on the PI Interface side still, since there will be less 'other stuff' dirtying up those logs.