Help
  • Explore Community
  • Get Started
  • Ask the Community
  • How-To & Best Practices
  • Contact Support
Notifications
Login / Register
Community
Community
Notifications
close
  • Forums
  • Knowledge Center
  • Events & Webinars
  • Ideas
  • Blogs
Help
Help
  • Explore Community
  • Get Started
  • Ask the Community
  • How-To & Best Practices
  • Contact Support
Login / Register
Sustainability
Sustainability

Join our "Ask Me About" community webinar on May 20th at 9 AM CET and 5 PM CET to explore cybersecurity and monitoring for Data Center and edge IT. Learn about market trends, cutting-edge technologies, and best practices from industry experts.
Register and secure your Critical IT infrastructure

Historic Data and Event Journal File Format

Geo SCADA Knowledge Base

Access vast amounts of technical know-how and pro tips from our community of Geo SCADA experts.

cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Show  only  | Search instead for 
Did you mean: 
  • Home
  • Schneider Electric Community
  • Knowledge Center
  • Geo SCADA Knowledge Base
  • Historic Data and Event Journal File Format
Invite a Co-worker
Send a co-worker an invite to the portal.Just enter their email address and we'll connect them to register. After joining, they will belong to the same company.
You have entered an invalid email address. Please re-enter the email address.
This co-worker has already been invited to the Exchange portal. Please invite another co-worker.
Please enter email address
Send Invite Cancel
Invitation Sent
Your invitation was sent.Thanks for sharing Exchange with your co-worker.
Send New Invite Close
Top Labels
Top Labels
  • Alphabetical
  • database 32
  • Web Server and Client 31
  • WebX 19
  • Request Form 18
  • Lists, Events & Alarms 16
  • ViewX 15
  • Application Programming 12
  • Setup 12
  • Telemetry 8
  • Events & Alarms 7
  • Lists 7
  • Mimic Graphics 7
  • Downloads 6
  • Support 5
  • IoT 5
  • SCADA 5
  • Geo SCADA Expert 5
  • Drivers and Communications 4
  • Security 4
  • DNP 3 3
  • IEC 61131-3 Logic 3
  • Trends and Historian 2
  • Virtual ViewX 2
  • Geo Scada 1
  • ClearSCADA 1
  • Templates and Instances 1
  • Releases 1
  • Maps and GIS 1
  • Mobile 1
  • Architectures 1
  • Tools & Resources 1
  • Privacy Policy 1
  • OPC-UA 1
  • Previous
  • 1 of 4
  • Next
Latest Blog Posts
  • OPC UA - Driver and Server
  • Requirements for Generating a Valid OPC UA Server Certificate
  • Load Events Using LoadRecord and LoadRecords
  • Geo SCADA Embedded Component Licenses
  • Geo SCADA 2023 Known Issues
Related Products
product field
Schneider Electric
EcoStruxure™ Geo SCADA Expert

Invite a Colleague

Found this content useful? Share it with a Colleague!

Invite a Colleague Invite
Anonymous user
Not applicable
‎2021-06-09 11:46 AM
0 Likes
0
1611
  • Bookmark
  • Subscribe
  • Email to a Friend
  • Printer Friendly Page
  • Report Inappropriate Content

Link copied. Please paste this link to share this article on your social media post.

‎2021-06-09 11:46 AM

Historic Data and Event Journal File Format

Originally published on Geo SCADA Knowledge Base by Anonymous user | June 09, 2021 08:46 PM

📖 Home  Back  
ClearSCADA stores historic data as files on the computer's hard disk. The location of those files is configured in the Location section of the Server Configuration Utility. So that the ClearSCADA can find the data as required, it is stored in folders within those locations, with each folder being named according to certain rules.

When objects are created they are automatically assigned a unique ID which starts at 0, the root group object, and incrementing for each new object. If an object is removed from the database its unique ID is not reused.

Both event journal and historic data is stored in what is known as a stream. A stream (or historic group) specifies how many different folders (and files) are used for storage of historic data. When the ClearSCADA server needs to search through this historic data, it is able to search a smaller subset of data when it is split into different streams and hence performance can be improved. A stream size of 8 (the default) will mean that a maximum of 256 (2 to the power of "stream size") objects will be placed in a sub-directory. i.e. Objects with the unique IDs of 0 to 255 will go in directory JnI00000, objects in the range 256 to 511 will go in JnI00001. The number of segments will depend upon the highest object id in the database and is not a predefined number.

Event Journal



The Event Journal data files are stored in a folder named Journal. This location is user definable in the Server Configuration. Under this directory, the data directory names are made up of 8 characters. The first two characters are always 'Jn'. The third character is a capital letter in the range D to O which represents the stream size.


  Letter   Stream Size   Objects
Per Folder
  D   3   8
  E   4   16
  F   5   32
  G   6   64
  H   7   128
  I   8   256
  J   9   512
  K   10   1024
  L   11   2048
  M   12   4096
  N   13   8192
  O   14   16384



The remaining 5 letters are digits which represent the stream number. The stream number is a number generated from the Object Id. The stream size is configurable from the Server Configuration Dialog and specifies the number of objects to store in one 'stream' of data. A stream can be used to improve performance by keeping smaller subsets of data together. This means that searches for data may only need to search subsets of all data if the data required is across only one or two streams.

For new installations of ClearSCADA, the stream size field defaults to 8. A stream size of 0 is equivalent to older style event journal storage (previous versions). On changing this setting and restarting the server, the file format of the data on disk will be changed to place the data into the appropriate streams. File formats have not changed between versions, only the location of the files.

Event Journal filenames start with 'H' (for hour) followed by a decimal number that is the number of hours since 1-Jan-1601 00:00:00 UTC. The file extension is 'MWJ'.

Historic Data



The directory structure is similar to that used for the event journal described above. The root level folder is named Historic and under this are a number of folders with 8 characters in the folder name. The first two characters are always 'Hd'. The third character is a capital letter the same as that used by the journal. The remaining 5 letters are digits that represent the stream number (based on the historic group size).

For new installations of ClearSCADA, the historic group size field defaults to 8. This setting is controllable via the registry key
HKEY_LOCAL_MACHINE\Software\Serck Controls\SCX6\DB\HistoricGroupSize

If the value of this key is changed, the server will need to be restarted, and on restart, the server will rearrange the folder structure on the hard disk to match the new group size. File formats do not change between versions, only the location of the files.

On upgrading from older systems, the value of this parameter will be maintained as 0. The downside of this setting is that potentially there will be many thousands of files in a single Windows folder and it becomes very difficult to browse the folder structure as Windows struggles to browse that many files. If you need to browse the historic data file folders, then it is strongly recommended to configure this registry key to 8 and restart the server.

If performance tuning of the database is required (typically only on very large databases ie. >50k points), reasonable values for the historic group size based on the number of objects in the database are
  • 7 - 16,000

  • 8 - 65.000

  • 9 - 250,000

  • 10 - 1,000,000


It is not recommended to set this setting outside the range of 7 to 10, and strongly recommended to change this setting only on the advice of your Support Representative.

Historic filenames start with 'Wk' (for week) followed by a decimal number that is the number of weeks since 1-Jan-1601 00:00:00 UTC. There are several file extensions in use for different types of historic data:
  File Extension
  File Contents
  HRD   Historic Raw Data
  HMD   Historic Modified Data
  HAD   Historic Annotation Data
  HSD   Historic Suppression Data (introduced in ClearSCADA 2010)
  HUD   Historic Units Data (deprecated)

In most servers, almost all the historical data will be in HRD files.

Other Comments



It is not necessary for two servers in a main-standby arrangement to have the same historic and event journal stream sizes. If however the historic data is manually copied from one machine to another and the sizes do not match, ClearSCADA will reformat the historic data on startup of the second server to make the formats different based on the stream sizes. This is not a problem as far as redundancy is concerned. It simply means that the physical file structure on the two servers will be different.

Data that is archived from a server from a particular stream size can be mounted on a server with a different stream/group size without problems.

Go: Home Back
Labels:
  • Request Form

  • Trends and Historian

Author

Biography

Anonymous user

Link copied. Please paste this link to share this article on your social media post.

  • Back to Blog
  • Newer Article
  • Older Article
To The Top!

Forums

  • APC UPS Data Center Backup Solutions
  • EcoStruxure IT
  • EcoStruxure Geo SCADA Expert
  • Metering & Power Quality
  • Schneider Electric Wiser

Knowledge Center

Events & webinars

Ideas

Blogs

Get Started

  • Ask the Community
  • Community Guidelines
  • Community User Guide
  • How-To & Best Practice
  • Experts Leaderboard
  • Contact Support
Brand-Logo
Subscribing is a smart move!
You can subscribe to this board after you log in or create your free account.
Forum-Icon

Create your free account or log in to subscribe to the board - and gain access to more than 10,000+ support articles along with insights from experts and peers.

Register today for FREE

Register Now

Already have an account? Login

Terms & Conditions Privacy Notice Change your Cookie Settings © 2025 Schneider Electric

This is a heading

With achievable small steps, users progress and continually feel satisfaction in task accomplishment.

Usetiful Onboarding Checklist remembers the progress of every user, allowing them to take bite-sized journeys and continue where they left.

of