Welcome to the new Schneider Electric Community

It's your place to connect with experts and peers, get continuous support, and share knowledge.

Close
Important Announcement: WELCOME to the new Schneider Electric Community! Community is now no longer part of Exchange, and is now rebranded under se.com. If you have any bookmarks and links saved, we request you to update them to ensure that you continue accessing our community from this new location. For any issues that you might encounter as part of this change, please reach out to SchneiderCommunity.Support@se.com, and the team will help to get your issues resolved.
Invite a Co-worker
Send a co-worker an invite to the Exchange portal.Just enter their email address and we’ll connect them to register. After joining, they will belong to the same company.
Send Invite Cancel
82537members
350142posts

Importing Historic Data

EcoStruxure Geo SCADA Expert Forum

Find out how SCADA systems and networks, like EcoStruxure Geo SCADA Expert, help industrial organizations maintaining efficiency, processing data for smarter decision making with IoT, RTU and PLC devices.

Solved
DavidSkilbeck
Lt. Commander
Lt. Commander
0 Likes
11
1787

Importing Historic Data

Hi, I have been running through the article below.

 

https://tprojects.schneider-electric.com/GeoSCADAHelp/Geo%20SCADA%202020/Content/ServerAdministratio...

 

For the last example 2.

Initially used the data provided as below

2016,07,03,16,23,23,0

2016,07,03,16,24,23,12

and got a message below

ErrorMessageCSVImport.png

I changed the data to below to get it to work

(earliest I could go back).

2021,02,22,16,11,38,000,4.0218887
2021,02,22,16,11,53,000,3.1718216
.......
It did work, however the date in the history was today's date and not that of the file.
ErrorMessageCSV2Import.png

 

Is there a way of adjusting the time window (and if so how far back can we go).
Is there a setting for the correct date / time  to be loaded in to the history, as to that of the file ( I notice there is AM listed).

Thanks,


Accepted Solutions
AndrewScott
Commander
Commander
0 Likes
10
1782

Re: Importing Historic Data

The writable life for historic data is configured using the Server Configuration tool:

https://tprojects.schneider-electric.com/GeoSCADAHelp/Geo%20SCADA%202020/Default.htm#ServerAdministr... l

 

The timestamps in the CSV file are in UTC, but ViewX is displaying the times in local time of the client PC, which appears to be UTC+10:00, so you are probably located in Australia.


Andrew Scott, Lead Engineer, AVEVA

See Answer In Context

11 Replies 11
AndrewScott
Commander
Commander
0 Likes
10
1783

Re: Importing Historic Data

The writable life for historic data is configured using the Server Configuration tool:

https://tprojects.schneider-electric.com/GeoSCADAHelp/Geo%20SCADA%202020/Default.htm#ServerAdministr... l

 

The timestamps in the CSV file are in UTC, but ViewX is displaying the times in local time of the client PC, which appears to be UTC+10:00, so you are probably located in Australia.


Andrew Scott, Lead Engineer, AVEVA
DavidSkilbeck
Lt. Commander
Lt. Commander
0 Likes
9
1769

Re: Importing Historic Data

Hi, do you refer to this section as below

 

ErrorMessageCSV3Import.png

 

as it seems to be set to 520 weeks.

I figures it would be something about UTC / Local.

Thanks,

BevanWeiss
Spock
Spock
0 Likes
8
1761

Re: Importing Historic Data

Those sections that you have indicated only apply to 'Synchronisation Data'.. not to the data that the Main server is actively able to write to.

 

The section that applies for the historical writes is at the top.  If you look at the help for 'Archive after' you will see that this is what triggered the inability to update/add new historical data.

 

The help comments are:

Define the Archive Time for Historic Data

Alarm Summary, Configuration Changes Auditing, Event Journal or Historic Data is stored online for a specified duration (see Specify the Duration to Retain Historic Data Online), then the data is deleted. You can define a period of time after, which data that is stored online can be archived. When historic data is marked as eligible for archiving, it can be archived for example to CD, DVD and so on, (it has to be archived before the duration of the online data expires).

During the period that data is stored online, the data can be altered (by having new values reported for the time period) unless it is marked as eligible for archiving. When historic data is marked as eligible for archiving, it can no longer be altered on the disk—the new records for that period / time cannot be written to; it can only be read.

 

2021-02-24 09_18_57-Clipboard.png

 

The concept here is that once you say that a historical period is 'archivable' then it should no longer be modified.  Since it may no longer be on the computer itself, and may actually be on a CD / DVD or write-once media stored for archiving.

 

For some reason I always 'think' that having an archive after time configured will help performance.  But I'm pretty sure it doesn't.  It obviously cuts down on when re-indexing might be required, but if you're trying to insert historical data into your system, then you probably need that historical data in there, and the cost of indexing is just a burden you need to wear.  As such, I'd say for 'most' systems, probably not having the Archive after tick box ticked is a reasonable approach (I think the default is ticked and 2 weeks).  If you will archive off to DVD etc, then you will need to have this configured.  But Geo SCADA Expert archiving is a little confusing anyway... so I suspect most people aren't using it.


Lead Control Systems Engineer for Alliance Automation (VIC).
All opinions are my own and do not represent the opinions or policies of my employer, or of my cat..
geoffpatton
Commander
Commander
0 Likes
7
1750

Re: Importing Historic Data

I have always kept years of history, but never actually archived any of it.

If I am remembering correctly the Archive after setting helps when you have redundancy and are restarting one of the machines.

This could have changed over the years too. I have been using ClearSCADA since version 2005.

BevanWeiss
Spock
Spock
0 Likes
6
1726

Re: Importing Historic Data

I think this behaviour might have changed in recent versions.

I did a sync yesterday, and having the 4 week 'archive after' setting did NOT help in any way... the full 4 months of historic data was synchronised... which was quite painful over the two tin cans, and low value twine used for the inter-server connection.

 

I would have hoped that it would have done a hash over the current data blocks, and only requested sync where the data block hash didn't match... but it seems that's not the case either (or the behaviour is buggy..)


Lead Control Systems Engineer for Alliance Automation (VIC).
All opinions are my own and do not represent the opinions or policies of my employer, or of my cat..
AdamWoodland
Commander Commander
Commander
4
1724

Re: Importing Historic Data

I think the methodology of how far back it syncs was changed around 2017 R1, based on the earlier change...

 

> would have done a hash

 

It used to do a CRC comparison up to 2013R1 or R2, they changed it as the CRC was too intensive to a much faster size and last modified date. If those match it doesn't re-sync the file.

 

Does cause problems if you copy/restore files to a server and that copy changes the last modified date, as the resync will just resync everything anyway.

geoffpatton
Commander
Commander
0 Likes
0
1722

Re: Importing Historic Data

Good to know, I like your networking description. I think all of the servers I work on have lots of resources and a good gigabit connection between them, and I rarely am the one that restarts one.

BevanWeiss
Spock
Spock
0 Likes
3
1712

Re: Importing Historic Data

Hmm... thanks for the refresher on that new system @AdamWoodland .  Unfortunately doesn't match my experience from yesterday.

 

The primary server had only been off since the 20th Feb, so it should have only been missing 4 days worth of data (or the last 1 week granules needing sync).  But it definitely resynchronised the whole history.

 

I do have a scheduled historic optimise, but that's only every Sunday... I guess I'll need to keep an eye on it, and report if it happens again.


Lead Control Systems Engineer for Alliance Automation (VIC).
All opinions are my own and do not represent the opinions or policies of my employer, or of my cat..
DavidSkilbeck
Lt. Commander
Lt. Commander
0 Likes
2
1677

Re: Importing Historic Data

Hi, these are the settings for my Geo SCADA.

 

ErrorMessageCSV4Import.png

This a sample of the data I want to import
4.0218887,2020,07,22,16,11,38,010,192,8
3.1718216,2020,07,22,16,11,53,010,192,8

This is what I am getting in Geo SCADA

ErrorMessageCSV5Import.png

If I use the below it wont import historic values

dt = new DateTime(2020, 2, 26, 11, 55, 52);

If I change the 2020 to 2021 it works.

DavidSkilbeck
Lt. Commander
Lt. Commander
0 Likes
1
1644

Re: Importing Historic Data

It has been working. I just needed to select last year for it to display.

Even with offset property for the point set to "Hour - 2 Years", yo have to change search to last year.

Thanks,

 

BevanWeiss
Spock
Spock
0 Likes
0
1635

Re: Importing Historic Data

It's possible that there are bugs around the offset property (and just Historic List behaviour in certain scenarios).

It would be worth reviewing the help documentation, and if you see behaviour that differs from the help documentation then raising a case with Tech Support.  That way they know about it, and can get it fixed for the future.

 

But always good to double check what the actual query is being used for the lists.


Lead Control Systems Engineer for Alliance Automation (VIC).
All opinions are my own and do not represent the opinions or policies of my employer, or of my cat..