Importing Historic Data to points in GeoSCADA: LoadDataValue( Reason, Quality, Time Stamp, Value ) Error

[Anonymous user]

I am using the Automation Interface with VB.NET and I am getting an error when trying to use the method LoadDataValue( Reason, Quality, Time Stamp, Value ) . I have managed to use the same method with Python (works perfectly).

Error: 

System.MissingMemberException:'Public member 'LoadDataValue' on type '_ComObject not found.'

Here is my code:

Dim Svr As ScxV6DbClient.ScxV6Server

Dim Obj As ScxV6DbClient.ScxV6Object

Dim his As ScxV6DbClient.ScxV6Aggregate

Svr.Connect(" ", " ", " ")

Obj=Svr.FindObject("Example Projects. New Analog Point")

his = Obj.Aggregate("Historic")

his.Enable =True                           'This Works - Enable/ Disables the historic for the point

his.Interface.LoadDataValue(1,192,"02/01/2020 10:09:05.200",1) 'Shows the above error

[sbeadle]

Hi. You will need to use a time data type, not a string.

[Hardin4019]

From the examples in the Help  I would say make sure you have Svr.Connect("ServerName", "AccountName", "AccounPassword") and his.Interface.LoadDataValue("1",192,"02/01/2020 10:09:05",1). Your was missing " " around the first value after the ( 

[AndrewScott]

The help contained some errors that were only corrected fairly recently. This included incorrect use of double quotes.

Below is the VB.NET example from the latest help:

Dim objServer = New ScxV6Server
objServer.Connect ("system name", "username", "pwd")
Dim obj = objServer.FindObject("Array Group.AG01")
Dim His = obj.Aggregate("Historic")
His.Interface.LoadDataValue(1, 192, #7/3/2016 5:22:23 PM#, 13)
 
obj = Nothing
objServer.Disconnect()

[Anonymous user]

It is giving me the same error.

[Anonymous user]

I tried putting in a timestamp variable as well. But it is giving the same error as 'LoadDataValue' on type '_ComObject' not found. Is there another reference I should try adding to the solution? Or is there a specific time stamp format? 

Dim tryDateTime As Date

tryDateTime = #01/01/2020 10:14 PM#

his.Interface.LoadDataValue(1, 192, tryDateTime, 1)

[Anonymous user]

When I look at the definition for "Interface", it shows it as a "Read Only" Object Property. The possible methods under the Property are : Equals, GetHashCode, GetType, ToString. 

[Anonymous user]

This case has been raised with support, but I wanted to put it here as well since many guys here have more experience with the APIs than I do!

This works for me:

        Dim Svr As ScxV6DbClient.ScxV6Server
        Dim Obj As ScxV6DbClient.ScxV6Object
        Dim his As ScxV6DbClient.ScxV6Aggregate

        Svr = New ScxV6DbClient.ScxV6Server                                                  'Additional line, create the server object
        Svr.Connect("Scratch", "", "")                                                                    'I assume you have working logon details
        Obj = Svr.FindObject("Example Projects.New Analog Point")

        his = Obj.Aggregate("Historic")
        his.Enable = True                                                                                        'This Works - Enable/ Disables the historic for the point
        his.Interface.LoadDataValue(1, 192, #02/01/2020 10:09:05 #, 1)              'Shows the above error

        Obj = Nothing                                                                                             'Unrelated but good practice
        Svr.Disconnect()

Only two real differences:
I've added a line to create the server object. I wouldn't have thought connect would work without! It should throw a Null Reference Exception.
I've removed a space from in front of the point's name - the leading space is not allowed config so I guess it is incorrect - but again this is strange, I wouldn't have thought the aggregate would work!

The exception you're throwing kinda makes me think you don't have the correct reference added to your project. It should be Scx V6 Automation Interface as a COM reference:

[AndrewScott]

This might be a registration issue with DBClient.dll.

You might try re-registering the DLL from an Administrator command prompt:

64-bit

regsvr32 "C:\Program Files\Schneider Electric\ClearSCADA\DBClient.dll"

32-bit

regsvr32 "C:\Program Files (x86)\Schneider Electric\ClearSCADA\DBClient.dll"

[sbeadle]

Another possible source of the problem is that this method can only be used to import data into the writable part of a point's history. Look at the Server Configuration tool, there are two relevant dates in the 'Historic Configuration' | 'Historic Data' page.

a) Data cannot be newer than the Future Data window.

b) Data cannot be older than the Archive After window.

In this example, data should be newer than 25 weeks old and older than 1 week into the future:

[Anonymous user]

@sbeadle I have faced that issue before with my Python script as well. I rechecked that, and the time is in the range.

@AndrewScott  I registered the dll with success. But I am still receiving that error.

@Anonymous user I have the "Scx V6 Automation Interface" Reference attached. And my bad for forgetting to copy the script correctly. I have created the server object in it. I am able to connect to the server and FindObject/ Write to Opc properties and Enable the Historic Aggregate. But not able to  use "LoadDataValue()" I am not able to use any methods of the Interface property specifically. 

Thank you all for trying to help.

[Anonymous user]

If you browse to ScxV6ObjectClass, does it look like this or are there things missing?

[Anonymous user]

Yes it looks exactly like that.

[Anonymous user]

I checked the definition of Interface: And this is what it shows. 

[sbeadle]

In your original post you mention that Python works for you. Please can you post your Python code here - there may be a clue in it? Thanks.

[Anonymous user]

import win32com.client

server=win32com.client.Dispatch('Serck.ScxV6Server')

server.Connect(' ',' ',' ')

AnalogObj=server.FindObject('Example Projects.Analog Point')

AnalogObj.Interface.Historic.LoadDataValue(1, 192, #01/02/2020 10:10:10#, 1)

(I am not able to use the interface property at all.

I am using Visual Studio Community 2019, should I try a different version?)

[sbeadle]

While the community considers that, here's how to do it with the .Net API:

using System;
using ClearScada.Client;            // This requires a Reference from the project to this dll
using System.Security;

namespace InsertHistoricData
{
	class Program
	{
		static void Main()
		{
			string user = "s";
			string pass = "s";

			ClearScada.Client.Simple.Connection connection;
			var node = new ClearScada.Client.ServerNode(ClearScada.Client.ConnectionType.Standard, "127.0.0.1", 5481);
			connection = new ClearScada.Client.Simple.Connection("Utility");
			connection.Connect(node);
			var spassword = new System.Security.SecureString();
			foreach (var c in pass)  spassword.AppendChar(c); 
			connection.LogOn(user, spassword);
			ClearScada.Client.Simple.DBObject PointObj = connection.GetObject("Test His Insert and Load.New Analog Point");

			Object[] p1 = new Object[4];
			p1[0] = 1;
			p1[1] = 192;
			p1[2] = DateTime.UtcNow;
			p1[3] = 1;
			PointObj.Aggregates["Historic"].InvokeMethod("LoadDataValue", p1);
		}
	}
}

[sbeadle · Accepted solution]

Going to mark this as answered. We're pretty sure there is some windows/setup issue going on as it's working for others. Please try this on a fresh PC if you can.

[sbeadle]

For completeness, a complete C# code tested and working:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using ScxV6DbClient; // Refer to program files (x86)\===\ClearScada.Client.dll

namespace InsertHistoryCOMAPI
{
	class Program
	{
		static void Main(string[] args)
		{
			ScxV6Server objServer = new ScxV6Server();
			objServer.Connect("Local", "s", "s");
			ScxV6Object obj = objServer.FindObject("Example Projects.New Analog Point");
			ScxV6Aggregate His = obj.Aggregate["Historic"];
			His.Interface.LoadDataValue(1, 192, DateTime.UtcNow, 13);
			obj = null;
			objServer.Disconnect();
		}
	}
}

[Anonymous user]

I think it is some configuration to be done. I tried on a different computer, it gave the same error. Thank you for your help. I will try the .Net API for the purpose. 

[Kan]

I have a question related to the code.

The following is my function and it works. But the issue is if the function is called multiple times with the same datetime value, then it inserts duplicated records in the historic list of the point.

For example, the list looks like this after the function is called 3 times with the same datetime value but different values.

Time                                             Value

25/05/2020 3:00:00.000 AM         3.123

25/05/2020 3:00:00.000 AM         4.866

25/05/2020 3:00:00.000 AM         3.332

What I want is the historic list only contain the value set from the latest call. How to remove the duplicates and only store the latest value? Thanks a lot.

public bool WriteValueToAnalogPoint(string pointFullName, DateTime valueDateTime, double value)

{

if (!IsConnected)

{

throw new Exception("No connection.");

}

try

{

DBObject destination = this.connection.GetObject(pointFullName);

Object[] args = new Object[4];

args[0] = 1;

args[1] = QUALITY_GOOD;

args[2] = valueDateTime;

args[3] = value;

destination.Aggregates[HISTORIC].InvokeMethod(FUNC_LOAD_DATA_VALUE, args);

return true;

}

catch(Exception ex)

{

//todo: log the error

return false;

}

}

[AdamWoodland]

Yes, that will create duplicate values with the same timestamp, even if the value changes. The method call is deliberately dumb as its about efficiency and you use those method calls to import many hundred of gigabytes of data.

A few ways I can think of:

1. If you know all the data that should be in that historic for the point (either by pre-reading it from CDBHistoric or your original data source has it) the easiest way my be to disable historic and then re-enable historic on that point and the historic will be blank giving you a clean slate. But this is a bit like a sledgehammer to a nut and with hot-standby, low bandwidth links and more complex architectures does have some possible crazy knock-on effects. If in an instance need to ensure property overrides are set. You're also not able to write into the read-only/archive period but that's the same for the next one too anyway.

2. Be surgical and read the existing data, and where existing data matches the timestamp for your new data use the delete method (new since 2017R3 I think?) to remove the old data first so when you import there will be only the values you want in its place. I'd go for this one personally but each to their own.

Both have timing issues and the need to backup things in case of failure. Test though!

Probably can't use modify value, as I think that causes a second entry to be there. Also you can't do the historic optimising methods as their values differ they aren't actually duplicates so they won't be cleaned up.

[Kan]

Thanks a lot for your reply.

Yes, I agree the best way for me is to delete the record before adding. Do you know how to call the delete function (the syntax and what sort of parameters). I can't find any help or reference on that function. I am new to ClearSCADA.

Thanks a lot.

[AndrewScott]

There are two methods for deleting historic data:

DeleteValue( Time, HistoricSourceType, Comment )

DeleteValues( StartTime, EndTime, HistoricSourceType, Comment )

The first method deletes a single record, the second method deletes all of the records in a time period.

The HistoricSourceType (integer) argument defines what type of data is removed:

1. Raw data
2. Modified data
3. Both (raw and modified)

The Comment (string) argument provides a description of why the record(s) is being deleted and is included in the event journal message that logs the deletion.

Both methods require 'system administrator' permission.

[Kan]

Thanks heaps! Very helpful.

[BevanWeiss]

It sounds like the right option for the problem directly presented would have been the ModifyValue method.

ModifyValue Modify Value Modifies a single historic value. Modify History 1.Timestamp (Time) : Timestamp of modified historic value.
2.Value (Variant) : Modified historic value.
3.Quality (Long) : Quality of modified historic value.
4.Comment (String) : Comment to be logged against the value modification in the event journal.

I would not expect it to add another equivalent historical record (and if it does, I would consider that a 'bug' since then Modify essentially performs exactly the same function as Insert).

Another approach might be to collect or coalesce the values internally before you actually report them to Geo SCADA.

If you have all of your calculated values, and then sort them by timestamp, you would know which are duplicates.. and could apply some logic (in your code) to only retain the value that you care about... which might be the most recently generated (in which case you'd need to track when you generated the timestamp), or the average, or the min/max/stddev or some other such aggregate.  Then you write one batch of values into the Historic stream, which would deliberately not have any duplicates.

If your records might come from a source where they have timestamps reported from 'all over' (i.e. you might get a record timestamped 2020-01-01 00:00:00 today... and another timestamped the same in a week's time) then my second approach wouldn't work.  But if you can assume they are infrequent, then you could do a pre-pass to identify duplicate records from your new batch (i.e. those that exist already in the historical archive) and treat them a little specially.