Help
  • Explore Community
  • Get Started
  • Ask the Community
  • How-To & Best Practices
  • Contact Support
Notifications
Login / Register
Community
Community
Notifications
close
  • Forums
  • Knowledge Center
  • Events & Webinars
  • Ideas
  • Blogs
Help
Help
  • Explore Community
  • Get Started
  • Ask the Community
  • How-To & Best Practices
  • Contact Support
Login / Register
Sustainability
Sustainability

Join our "Ask Me About" community webinar on May 20th at 9 AM CET and 5 PM CET to explore cybersecurity and monitoring for Data Center and edge IT. Learn about market trends, cutting-edge technologies, and best practices from industry experts.
Register and secure your Critical IT infrastructure

Unexpected Historic Database Logging

Geo SCADA Knowledge Base

Access vast amounts of technical know-how and pro tips from our community of Geo SCADA experts.

cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Show  only  | Search instead for 
Did you mean: 
  • Home
  • Schneider Electric Community
  • Knowledge Center
  • Geo SCADA Knowledge Base
  • Unexpected Historic Database Logging
Invite a Co-worker
Send a co-worker an invite to the portal.Just enter their email address and we'll connect them to register. After joining, they will belong to the same company.
You have entered an invalid email address. Please re-enter the email address.
This co-worker has already been invited to the Exchange portal. Please invite another co-worker.
Please enter email address
Send Invite Cancel
Invitation Sent
Your invitation was sent.Thanks for sharing Exchange with your co-worker.
Send New Invite Close
Top Labels
Top Labels
  • Alphabetical
  • database 32
  • Web Server and Client 31
  • WebX 19
  • Request Form 18
  • Lists, Events & Alarms 16
  • ViewX 15
  • Application Programming 12
  • Setup 12
  • Telemetry 8
  • Events & Alarms 7
  • Lists 7
  • Mimic Graphics 7
  • Downloads 6
  • Support 5
  • IoT 5
  • SCADA 5
  • Geo SCADA Expert 5
  • Drivers and Communications 4
  • Security 4
  • DNP 3 3
  • IEC 61131-3 Logic 3
  • Trends and Historian 2
  • Virtual ViewX 2
  • Geo Scada 1
  • ClearSCADA 1
  • Templates and Instances 1
  • Releases 1
  • Maps and GIS 1
  • Mobile 1
  • Architectures 1
  • Tools & Resources 1
  • Privacy Policy 1
  • OPC-UA 1
  • Previous
  • 1 of 4
  • Next
Latest Blog Posts
  • OPC UA - Driver and Server
  • Requirements for Generating a Valid OPC UA Server Certificate
  • Load Events Using LoadRecord and LoadRecords
  • Geo SCADA Embedded Component Licenses
  • Geo SCADA 2023 Known Issues
Related Products
product field
Schneider Electric
EcoStruxure™ Geo SCADA Expert

Invite a Colleague

Found this content useful? Share it with a Colleague!

Invite a Colleague Invite
Anonymous user
Not applicable
‎2021-06-09 12:41 PM
1 Like
0
1054
  • Bookmark
  • Subscribe
  • Email to a Friend
  • Printer Friendly Page
  • Report Inappropriate Content

Link copied. Please paste this link to share this article on your social media post.

‎2021-06-09 12:41 PM

Unexpected Historic Database Logging

Originally published on Geo SCADA Knowledge Base by Anonymous user | June 09, 2021 09:41 PM

📖 Home  Back  
If your system's historic database is being flooded with minor point value changes, it is likely that:
  • Your outstation is reporting minor point changes


Or
  • All changes in a polled PLC are being logged historically.


To prevent unnecessary point updates, you should try to filter out (compress) the data at the outstation/PLC. By compressing the data at the outstation/PLC, you reduce the amount of data that is transmitted across the SCADA communications network (which may be low bandwidth) and you also reduce the amount of data that needs to be logged in ClearSCADA. Another advantage of Historic Compression is that it reduces the amount of data that is stored in the outstation/PLC (and so there is less use of the outstation/PLC's resources).

Compressing Data at the Outstation/PLC



Depending on the features that your outstation supports, you may be able to reduce the amount of data logged (and transmitted to ClearSCADA) by configuring significant change deadbands (change deviations). The significant change deadbands set the outstation to only log values that change by a defined amount or percentage. Working in a similar way to the Historic Compression settings in ClearSCADA, the significant change deadband features are designed to reduce traffic between the outstation and master station and reduce the amount of data stored in the outstation. The significant change deadbands are set in the outstation itself or are set in ClearSCADA and are downloaded to the outstation.

Most PLCs do not support any kind of compression as they do not store historic data for later upload to the SCADA system. When using PLCs that are causing an excessive amount of historic events to be logged, you should:
  • Create a user program in the PLC to 'smooth' the value for the point (register). This is often achieved by using 2 registers, the first register containing the raw value and the second register containing the 'smoothed' value. The program then copies the value from the first register to the second register, but only when the difference between the two values exceeds a threshold. ClearSCADA is configured to read the 'smoothed' value from the second register.


If filtering logic cannot be used, you may be able to apply compression at the master station (see below).

Compressing Data at the Master Station



When you have compressed the data at the outstation/PLC, you can apply further filters at the master station by using the Historic Data Filter setting and the Historic Compression settings in ClearSCADA.

Every point update is classified as either current data or logged data (except updates for Override/Release Override actions). Some drivers support both current and logged data (where logged data is usually enabled or disabled on a per-point basis), but other drivers support only current data (such as ICMP and SNMP) or only logged data (such as Primayer, Technolog, Radcom, and Isodaq).

For each point update, there is a reason for logging, which can be:
  • Current - The point has a new current value

  • Significant Change - The point has changed value by an amount that exceeds the defined significant change deadband

  • State Change - The point has changed state

  • Timed Report - The point update is timed to occur (the update is a result of regular periodic logging of the point value)

  • End of Period - The point update is timed to occur at the end of a calculated period

  • End of Period + Reset - The point update is timed to occur at the end of a calculated period and the point's value has been reset

  • Override - The point has been overridden by a user selecting the Override action.

  • Release - The point has had its override removed by a user selecting the Release Override action.


By using the Historic Data Filter setting, you can specify which points have their updates stored historically---you set the updates to be stored according to their reason for update. For example, for DNP3 points you can choose whether static data (polled current) or event data (significant data such as state changes, crossed thresholds etc.) or both static and event data is logged in the historic database. By filtering according to the reason for update, you can reduce the amount of point data that is stored historically.

Please note that the Historic Data Filter setting is only available on certain advanced drivers.

When configuring the Historic Data Filter setting, you can usually choose which reasons for logging are to be applied. The reasons may vary according to the driver and points (and the configuration of the point itself). For more information on the Historic Data Filter settings, please refer to the documentation for the relevant driver in ClearSCADA Help.

You can also apply Historic Compression settings in ClearSCADA. The Historic Compression settings allow you to define an additional filter for point updates that are stored historically and are defined in the point configuration.

The Historic Compression settings include:
  • Apply To

ClearSCADA compresses the data that it stores historically for this point. You can choose to compress:
  • Current Data Only - ClearSCADA only compresses the current data that is stored historically for the point. It does not compress any other data that it stores historically for the point.





For example, the Historic Data Filter on a DNP3 point is set to allow its static (current) data and DNP3 event data to be logged historically (static and event data). The same point has its Historic Compression Apply To setting set to Current Data Only (i.e. static data only). This means that DNP3 events for the point are always stored historically as they are not compressed. Only the point's current data is affected by the Historic Compression settings, and so the current data for the point is compressed (not stored historically).

  • All Data

ClearSCADA compresses all data, so current data and any other data is stored historically for the point.

  • No Data

ClearSCADA does not compress any data for the point and so all of the point's data is stored historically.

Note that for points that have Logged Data storage enabled, ClearSCADA usually only stores logged data historically---see the relevant driver documentation in ClearSCADA Help for more details.

  • Significant Change

The Significant Change for Historic Compression feature allows you to set ClearSCADA so that it only stores a point value historically if the point value changes by at least the defined amount.You can choose from three different types of Significant Change:
  • Absolute

The value of the point must change by a defined amount for it to be stored historically.

  • Percentage of Span

The value of the point must change by a defined percentage of the span if it is to be stored historically. The span is the point range (Zero Scale to Full Scale). The Percentage of Span option is available for points that are, or are similar to analog points (i.e. points with a range of values).

  • Percentage of Value

The value of the point must change by a defined percentage of the last stored value if it is to be stored historically.

  • Minimum Interval

Allows you to define the minimum amount of time between the values that ClearSCADA stores historically. Any values that are reported more frequently than the defined Minimum Interval are not stored historically.

The Historic Compression settings can be used to reduce the amount of data that is stored historically. However, the Historic Compressions settings DO NOT APPLY in the following situations:
  • Compression is not applied to every point update. The following changes are exempt from compression to ensure that important changes are not discarded:

  • Change of State (including Master Station state, though Master Station state only applies to analog points that are on systems that have Master Station Alarm Limits enabled)

  • Change of Quality

  • Change of Reason for logging

  • Change of Status





So, if the point is updated, compression is only applied if the state of the point is the same as the last historically logged state. If the state of the point is different to the last historically logged state, compression is not applied, even if the Historic Compression settings are in place. For example, if an analog point crosses an alarm limit so its state changes from Normal to Low, the new data will always be logged irrespective of the Historic Compression settings.




Changes to state, quality, status, master station state and reasons for updates are usually related to important changes on your system that need to be recorded historically. That is why for these types of change, ClearSCADA ignores the Historic Compression settings - it is important that there is a record of these types of change and so they should not be filtered out by historic compression.

  • Compression is only applied to point updates that are in sequential time order. If a point updates with a time that is before the last historically logged time, the data for the point is not compressed. This is called an 'out of sequence' point update (out of sequence point updates rarely occur).


  • There is a sudden 'jump' in the point's value. When Historic Compression filters out values, the last value that was discarded is stored in the current database (but not in the historic database). If the next point update to be compressed contains a point value that has suddenly increased or decreased by a considerable amount, ClearSCADA ignores the Historic Compression settings and stores both the last discarded value and the new point update value historically. This is to ensure that when the historic data is displayed on a Trend, the trace indicates that the point value hardly changed over a period of time and then suddenly increased or decreased rapidly at a specific time. If the last discarded value was not also stored in the historic database, the trace on the Trend would indicate a gradual increase in the value from the time of the previously stored value to the value in the latest update.




UnexpectedDataLoggingExample1.pngUnexpectedDataLoggingExample1.pngUnexpectedDataLoggingExample1.png
In the Trend shown above, the trace shows the previously stored historic point value, the last value that was discarded due to historic compression (but was saved in the current database) and the new historic value. As the value that had previously been discarded by historic compression is included, the Trend is able to show the true relationship of the values i.e. that the point value had been close to 30 for a certain amount of time and then it suddenly increased to 45.




If the Historic Compression settings had been applied in this situation, the Trend would not have included the value that was previously discarded by the historic compression feature. This would mean that the trace would have shown a gradual increase from the previously stored historic value of 30 to the new historic value of 45 (as shown below). This does not represent the actual values that were reported.



UnexpectedDataLoggingExample2.pngUnexpectedDataLoggingExample2.pngUnexpectedDataLoggingExample2.png


Go: Home Back
Labels:
  • database

  • Trends and Historian

Author

Biography

Anonymous user

Link copied. Please paste this link to share this article on your social media post.

  • Back to Blog
  • Newer Article
  • Older Article
To The Top!

Forums

  • APC UPS Data Center Backup Solutions
  • EcoStruxure IT
  • EcoStruxure Geo SCADA Expert
  • Metering & Power Quality
  • Schneider Electric Wiser

Knowledge Center

Events & webinars

Ideas

Blogs

Get Started

  • Ask the Community
  • Community Guidelines
  • Community User Guide
  • How-To & Best Practice
  • Experts Leaderboard
  • Contact Support
Brand-Logo
Subscribing is a smart move!
You can subscribe to this board after you log in or create your free account.
Forum-Icon

Create your free account or log in to subscribe to the board - and gain access to more than 10,000+ support articles along with insights from experts and peers.

Register today for FREE

Register Now

Already have an account? Login

Terms & Conditions Privacy Notice Change your Cookie Settings © 2025 Schneider Electric

This is a heading

With achievable small steps, users progress and continually feel satisfaction in task accomplishment.

Usetiful Onboarding Checklist remembers the progress of every user, allowing them to take bite-sized journeys and continue where they left.

of