Help
  • Explore Community
  • Get Started
  • Ask the Community
  • How-To & Best Practices
  • Contact Support
Notifications
Login / Register
Community
Community
Notifications
close
  • Forums
  • Knowledge Center
  • Events & Webinars
  • Ideas
  • Blogs
Help
Help
  • Explore Community
  • Get Started
  • Ask the Community
  • How-To & Best Practices
  • Contact Support
Login / Register
Sustainability
Sustainability

We Value Your Feedback!
Could you please spare a few minutes to share your thoughts on Cloud Connected vs On-Premise Services. Your feedback can help us shape the future of services.
Learn more about the survey or Click here to Launch the survey
Schneider Electric Services Innovation Team!

Corrupt Historic File Alarm

EcoStruxure Geo SCADA Expert Forum

Schneider Electric support forum about installation, configuration, integration and troubleshooting of EcoStruxure Geo SCADA Expert (ClearSCADA, ViewX, WebX).

cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results forย 
Showย ย onlyย  | Search instead forย 
Did you mean:ย 
  • Home
  • Schneider Electric Community
  • Remote Operations
  • EcoStruxure Geo SCADA Expert Forum
  • Corrupt Historic File Alarm
Options
  • Mark Topic as New
  • Mark Topic as Read
  • Float this Topic for Current User
  • Bookmark
  • Subscribe
  • Mute
  • Printer Friendly Page
Invite a Co-worker
Send a co-worker an invite to the portal.Just enter their email address and we'll connect them to register. After joining, they will belong to the same company.
You have entered an invalid email address. Please re-enter the email address.
This co-worker has already been invited to the Exchange portal. Please invite another co-worker.
Please enter email address
Send Invite Cancel
Invitation Sent
Your invitation was sent.Thanks for sharing Exchange with your co-worker.
Send New Invite Close
Top Experts
User Count
sbeadle
Kirk sbeadle Kirk
308
AndrewScott
Admiral AndrewScott
98
BevanWeiss
Spock BevanWeiss
91
AdamWoodlandToo
Lt. Commander AdamWoodlandToo
37
View All
Related Products
product field
Schneider Electric
EcoStruxureโ„ข Geo SCADA Expert

Invite a Colleague

Found this content useful? Share it with a Colleague!

Invite a Colleague Invite
Back to EcoStruxure Geo SCADA Expert Forum
geoffpatton
Captain geoffpatton
Captain

Posted: โ€Ž2021-06-09 02:21 PM . Last Modified: โ€Ž2023-05-03 12:03 AM

0 Likes
10
3930
  • Mark as New
  • Bookmark
  • Subscribe
  • Mute
  • Permalink
  • Print
  • Email to a Friend
  • Report Inappropriate Content

Link copied. Please paste this link to share this article on your social media post.

Posted: โ€Ž2021-06-09 02:21 PM . Last Modified: โ€Ž2023-05-03 12:03 AM

Corrupt Historic File Alarm

One of my customers has a corrupt historic file alarm that I'll look into what's going on tomorrow, but I thought I'd see if anyone here had any advice on dealing with this. They have a hot standby pair and a permanent standby.

Labels
  • Labels:
  • SCADA
  • Tags:
  • english
  • scada
  • SCADA app
  • SCADA software
  • SCADA tutorial
  • Telemetry and SCADA
Reply
Contact Support

Link copied. Please paste this link to share this article on your social media post.

  • All forum topics
  • Previous Topic
  • Next Topic
Replies 10
AdamWoodland
AdamWoodland Schneider Alumni (Retired)
Schneider Alumni (Retired)

Posted: โ€Ž2021-06-09 02:57 PM

0 Likes
2
3922
  • Mark as New
  • Bookmark
  • Subscribe
  • Mute
  • Permalink
  • Print
  • Email to a Friend
  • Report Inappropriate Content

Link copied. Please paste this link to share this article on your social media post.

Posted: โ€Ž2021-06-09 02:57 PM

The first question is really what the corruption is:

 

a. NTFS file system corruption

b. File write corruption

c. Application level corruption

 

For a. the solution can often be chkdsk /R, but the content of the file may also be corrupt. The standby may have the data as data is streamed to the standby separately to whatever goes to disk so the standby may have all the data, however a resync may have caused problems there. The thing about a chkdsk /R is it needs a reboot so when it comes back up the other server should be the Main and so should recover that way too.

 

For b., it is possible with anti-virus, backup and similar tools, although very rare. Check the size of the HRD file (not the size on disk) is a multiple of 32 bytes... it might not be

 

For c, I've seen zeros be written as the 32 bytes a long long time ago. scx_cmd hisdump might show something or you may need to break out a hex editor and check each 32 bytes for anything odd

 

For b and c then support can also take a look, although probably need DB logs of the record being written too which may be asking a lot depending on how far back the problem happened

  • Tags:
  • english
  • scada
  • SCADA app
  • SCADA software
  • SCADA tutorial
  • Telemetry and SCADA
Reply
Contact Support

Link copied. Please paste this link to share this article on your social media post.

geoffpatton
Captain geoffpatton
Captain

Posted: โ€Ž2021-06-09 04:27 PM

In response to AdamWoodland
0 Likes
1
3919
  • Mark as New
  • Bookmark
  • Subscribe
  • Mute
  • Permalink
  • Print
  • Email to a Friend
  • Report Inappropriate Content

Link copied. Please paste this link to share this article on your social media post.

Posted: โ€Ž2021-06-09 04:27 PM

@AdamWoodlandyour comment about db logs made me go ahead and give it quick look for when they got the alarm. it was about 12 hours ago so I checked the Primary's DB log. The db log only had the same description as the Alarm in it. The good news is that it is just one file corrupt and the file is on the one running standby.

  • Tags:
  • english
  • scada
  • SCADA app
  • SCADA software
  • SCADA tutorial
  • Telemetry and SCADA
Reply
Contact Support

Link copied. Please paste this link to share this article on your social media post.

BevanWeiss
Spock BevanWeiss
Spock

Posted: โ€Ž2021-06-09 08:38 PM

In response to geoffpatton
0 Likes
0
3912
  • Mark as New
  • Bookmark
  • Subscribe
  • Mute
  • Permalink
  • Print
  • Email to a Friend
  • Report Inappropriate Content

Link copied. Please paste this link to share this article on your social media post.

Posted: โ€Ž2021-06-09 08:38 PM

I've never seen this to be due to an actual NTFS disk failure.

 

The issues that I've seen cause this have been:

  1. No disk space in my Test VM (normally the GeoSCADA DB crashes not long after)
  2. Anti-virus (Sophos / Norton) configured to real-time scan the directories in question (this normally causes a single file, sometimes a couple of files... but then sporadically, and normally at weird times of the day when you can't actively check it)
  3. Backup software which failed to apply a Shadow Copy backup, and just did a bulk lock on the disk instead (for this it generally results in much more than just a single file)

Lead Control Systems Engineer for Alliance Automation (VIC).
All opinions are my own and do not represent the opinions or policies of my employer, or of my cat..
  • Tags:
  • english
  • scada
  • SCADA app
  • SCADA software
  • SCADA tutorial
  • Telemetry and SCADA
Reply
Contact Support

Link copied. Please paste this link to share this article on your social media post.

geoffpatton
Captain geoffpatton
Captain

Posted: โ€Ž2021-06-10 01:01 PM

0 Likes
6
3891
  • Mark as New
  • Bookmark
  • Subscribe
  • Mute
  • Permalink
  • Print
  • Email to a Friend
  • Report Inappropriate Content

Link copied. Please paste this link to share this article on your social media post.

Posted: โ€Ž2021-06-10 01:01 PM

ChkDsk said everything was fine and the file size was the same as on the running Primary, so it might have already resynced it.

I still deleted it though and restarted the Standby so I know it got replaced syncing from the Primary again.

 

They are running Sophos so that is likely the cause of the problem. First time in like 7 years to have a corrupt historic file alarm though.

 

Thanks for the tips guys. It was likely Sophos interfering but I did not see any settings that would let me tell Sophos to not scan the history files. I let the server admin know since I cannot playing in his software anyway. I don't think he will change anything unless it happens again since it has not happened in the past 7 or so years.

  • Tags:
  • english
  • scada
  • SCADA app
  • SCADA software
  • SCADA tutorial
  • Telemetry and SCADA
Reply
Contact Support

Link copied. Please paste this link to share this article on your social media post.

BevanWeiss
Spock BevanWeiss
Spock

Posted: โ€Ž2021-06-10 05:31 PM

In response to geoffpatton
0 Likes
5
3887
  • Mark as New
  • Bookmark
  • Subscribe
  • Mute
  • Permalink
  • Print
  • Email to a Friend
  • Report Inappropriate Content

Link copied. Please paste this link to share this article on your social media post.

Posted: โ€Ž2021-06-10 05:31 PM

I feel that anti-virus software is largely just a random number generator to determine which files it's going to mess with.

 

I consistently have issues with non-Microsoft anti-virus and ClearSCADA/GeoSCADA.

If it's not actual historic file corruption warnings (which are typically not at all corrupt files, just files locked for a longer time than ClearSCADA likes), then it's horrible performance because of scanning of ViewX cache files, or log files..

 

On different software I've even had anti-virus cause crashes of PLC programming software.  That's one of those annoying 'tech support' calls.. where they say "Please sir to be turning off your anti-virus" and me saying "Fine... but I really don't think it's the antivirus, it's clearly a bug in your software"... and of course for the crashes to immediately stop when I disabled the anti-virus.

I still say it was a bug in their software... but also.. bloody anti-virus.


Lead Control Systems Engineer for Alliance Automation (VIC).
All opinions are my own and do not represent the opinions or policies of my employer, or of my cat..
  • Tags:
  • english
  • scada
  • SCADA app
  • SCADA software
  • SCADA tutorial
  • Telemetry and SCADA
Reply
Contact Support

Link copied. Please paste this link to share this article on your social media post.

AdamWoodland
AdamWoodland Schneider Alumni (Retired)
Schneider Alumni (Retired)

Posted: โ€Ž2021-06-16 04:31 PM

In response to BevanWeiss
0 Likes
4
3860
  • Mark as New
  • Bookmark
  • Subscribe
  • Mute
  • Permalink
  • Print
  • Email to a Friend
  • Report Inappropriate Content

Link copied. Please paste this link to share this article on your social media post.

Posted: โ€Ž2021-06-16 04:31 PM

The best way I've found to say with 99% confidence that "it is anti-virus" without running active things things like procmon is look in the config or data folder. If anti-virus is interfering with file writes and file locks then you'll see temp files persist there.

 

When the database saves it first writes to the temp files (the slow bit of the process), then deletes the old file version and renames the temp file. This causes problems with av when exclusions aren't sent, and with the amount of time this happens each minute usually captures something eventually. Normally you may see a temp file but it goes within a couple of seconds depending on how Windows Explorer is updating the file list. 

 

The other 1% is when the database crashes mid database save, but then you'll see lots of files with the same timestamp rather than random files with random timestamps.

 

Modern AV solutions like Cylance and Crowdstrike brings a whole new way of working, and the nuances of those on the database are yet to be seen.

  • Tags:
  • english
  • scada
  • SCADA app
  • SCADA software
  • SCADA tutorial
  • Telemetry and SCADA
Reply
Contact Support

Link copied. Please paste this link to share this article on your social media post.

BevanWeiss
Spock BevanWeiss
Spock

Posted: โ€Ž2021-06-21 08:50 PM

In response to AdamWoodland
0 Likes
3
3816
  • Mark as New
  • Bookmark
  • Subscribe
  • Mute
  • Permalink
  • Print
  • Email to a Friend
  • Report Inappropriate Content

Link copied. Please paste this link to share this article on your social media post.

Posted: โ€Ž2021-06-21 08:50 PM

First time I've heard of Cylance and Crowdstrike.  I'm not sure the IT world needs to have another software set with non-deterministic behaviour ๐Ÿ˜‰  We already have Windows...

 

I can see it being a good replacement for 'the dog ate my homework'.. 'the anti-virus AI must have gotten confused and thought my thesis was a virus... crazy AI you know...'


Lead Control Systems Engineer for Alliance Automation (VIC).
All opinions are my own and do not represent the opinions or policies of my employer, or of my cat..
  • Tags:
  • english
  • scada
  • SCADA app
  • SCADA software
  • SCADA tutorial
  • Telemetry and SCADA
Reply
Contact Support

Link copied. Please paste this link to share this article on your social media post.

AdamWoodland
AdamWoodland Schneider Alumni (Retired)
Schneider Alumni (Retired)

Posted: โ€Ž2021-06-27 03:58 PM

In response to BevanWeiss
0 Likes
2
3778
  • Mark as New
  • Bookmark
  • Subscribe
  • Mute
  • Permalink
  • Print
  • Email to a Friend
  • Report Inappropriate Content

Link copied. Please paste this link to share this article on your social media post.

Posted: โ€Ž2021-06-27 03:58 PM

In theory it should be better. You can still set exceptions but if an application has written similar data to the same folder 5 billion times it should have hopefully learnt pretty quickly that is 'normal' behaviour and let it continue.

  • Tags:
  • english
  • scada
  • SCADA app
  • SCADA software
  • SCADA tutorial
  • Telemetry and SCADA
Reply
Contact Support

Link copied. Please paste this link to share this article on your social media post.

BevanWeiss
Spock BevanWeiss
Spock

Posted: โ€Ž2021-06-28 10:25 PM

In response to AdamWoodland
0 Likes
1
3775
  • Mark as New
  • Bookmark
  • Subscribe
  • Mute
  • Permalink
  • Print
  • Email to a Friend
  • Report Inappropriate Content

Link copied. Please paste this link to share this article on your social media post.

Posted: โ€Ž2021-06-28 10:25 PM

That's the problem I see though... the AI learning part if not controlled.

 

For example alarms... for 10 years there might be a value of '0'... and then one day the alarm occurs, and it's a '1'.

Is the 'smart' AI going to consider that abnormal and prevent the writing?


Lead Control Systems Engineer for Alliance Automation (VIC).
All opinions are my own and do not represent the opinions or policies of my employer, or of my cat..
  • Tags:
  • english
  • scada
  • SCADA app
  • SCADA software
  • SCADA tutorial
  • Telemetry and SCADA
Reply
Contact Support

Link copied. Please paste this link to share this article on your social media post.

AdamWoodland
AdamWoodland Schneider Alumni (Retired)
Schneider Alumni (Retired)

Posted: โ€Ž2021-07-04 05:23 PM

In response to BevanWeiss
0 Likes
0
3717
  • Mark as New
  • Bookmark
  • Subscribe
  • Mute
  • Permalink
  • Print
  • Email to a Friend
  • Report Inappropriate Content

Link copied. Please paste this link to share this article on your social media post.

Posted: โ€Ž2021-07-04 05:23 PM

It should be behaviour based, so in theory it should be irrelevant what data you write, it is the why rather than the what. Not saying there might not be a problem as it is still relatively new technology but as I understand it that scenario shouldn't one of the problems.

 

COM in scripting (i.e. FileScriptingHost and WShell.Script) and changes to DLLs used and referenced due to upgrades, those are areas of higher likelihood of something being triggered as that would be a deviation from what could be normal/learnt behaviour.

 

Cylance for example is usually set to block powershell scripts. So anything like SYSTEM() calls might also be affected.

 

 

  • Tags:
  • english
  • scada
  • SCADA app
  • SCADA software
  • SCADA tutorial
  • Telemetry and SCADA
Reply
Contact Support

Link copied. Please paste this link to share this article on your social media post.

To The Top!

Forums

  • APC UPS Data Center Backup Solutions
  • EcoStruxure IT
  • EcoStruxure Geo SCADA Expert
  • Metering & Power Quality
  • Schneider Electric Wiser

Knowledge Center

Events & webinars

Ideas

Blogs

Get Started

  • Ask the Community
  • Community Guidelines
  • Community User Guide
  • How-To & Best Practice
  • Experts Leaderboard
  • Contact Support
Brand-Logo
Subscribing is a smart move!
You can subscribe to this board after you log in or create your free account.
Forum-Icon

Create your free account or log in to subscribe to the board - and gain access to more than 10,000+ support articles along with insights from experts and peers.

Register today for FREE

Register Now

Already have an account? Login

Terms & Conditions Privacy Notice Change your Cookie Settings ยฉ 2025 Schneider Electric

This is a heading

With achievable small steps, users progress and continually feel satisfaction in task accomplishment.

Usetiful Onboarding Checklist remembers the progress of every user, allowing them to take bite-sized journeys and continue where they left.

of