Database Stream Files

[sbeadle1]

Home  Back  

The information available in the EcoStruxure Geo SCADA Expert database is stored in different file structures called streams. This allows the separation of data on disk into conveniently sized units for good performance. Different streams are structured differently for the information that they store.

The choice of stream size for the Journal (events), Configuration Changes, and Alarm Summary is important. You should choose these before commissioning a system. Please refer to the product help for advice on how to do this (see the 'Define the Stream Size…' topic in the Server Administration Guide in the help). If you want to investigate making changes to the stream sizes after commissioning then you are advised to read the document attached at the bottom of this page, titled 'Advice on Resizing Historic Streams'.

The different stream types are listed below:

Name	Number of Streams per Server Instance	Examples	Notes
Configuration	1 per 16384 object IDs	Information that does not change often: ·       Point limits and anything shown when you click Properties. ·        Logic program code ·       Mimic content ·       Data grid structure and content ·       Data table structure (not content)	Stored in ClearSCADA.Dat, ClearSCADAConfig.Dat or ClearSCADAConfig0000.dat (and ClearSCADAConfigStr.dat) depending on your system's upgrade history Synchronized between servers Flushed to disk every minute but only when configuration has changed Saving can have a noticeable performance hit on large databases Is required to run the database Memory resident
Data	1 per 16384 object IDs	Information that changes frequently: ·       A point's CurrentValue and CurrentTime ·       A variable's value	Stored in ClearSCADAData.Dat or ClearSCADAData0000.Dat depending on your system's upgrade history Synchronized between servers Flushed to disk only when data has changed (every minute) Usually partnered with the configuration stream, however this stream is not strictly necessary to successfully start a database Memory resident
Dynamic	1	Runtime information that does not have to be retained on start-up: ·       Alarm redirection state ·       A channel’s current state	Synchronized between servers Memory resident Not stored on disk
Metadata	1	Extra properties added to database items	Stored in ClearSCADAMetadata.dat, an XML file usually edited via ViewX Flushed to disk on change Synchronized between servers A lack of file on database startup may cause some configuration to be lost Memory resident
Archive	1	The Archive volume index	Stored in ClearSCADAArchive.dat ·       Not synchronized between servers (each server has its own local index) Saved following each archive operation
Files	Multiple, 1 per 'CHisFiles' aggregate	Example: Historical Crystal Reports	Stored in the Files directory Disk resident
Historic	Multiple, 1 per 'CHistory' aggregate	Point historic data	Synchronized between servers New records are flushed to disk every minute (in batches, can take longer or be more frequent) Stored in the History directory Single-purpose stream 32 bytes per record Disk resident, with a memory cache
Journal	Multiple, many database items to one stream	Database item event data and messages	·        Synchronized between servers ·        New records are flushed to disk every minute (in batches, can take longer or be more frequent) ·        Number of streams = (current maximum object id in database/stream size) Stream size can be adjusted for larger systems Stored in the Journal directory Single-purpose stream 768 bytes per record Disk resident, with a small memory cache
DataFiles	Multiple, 1 per type of file for each relevant database item	Contents of data tables Crystal Report definitions	Synchronized between servers Stored in the DataFiles directory ·        Disk resident
ConfigChanges	Multiple, many database items to one stream	Changes to configuration properties stored historically	Disabled by default Synchronized between servers Stored in the ConfigChanges directory New records are saved to disk every minute Number of streams = (current maximum object id in database/stream size) Stream size can be adjusted for larger systems Single-purpose stream 1280 bytes per record Disk resident, with a memory cache
AlarmSummary	Multiple, many database items to one stream	Provides a combined list of current and historic alarms for an item, group of items or the whole database, creating an audit trail for alarms on the system.	Disabled by default Synchronized between servers Stored in the AlarmSummary directory New records are saved to disk every minute Number of streams = (current maximum object id in database/stream size) Stream size can be adjusted for larger systems Single-purpose stream 1280 bytes per record Disk resident, with a memory cache

You can use the Database Schema to determine if a database property is either a Data, Configuration, or Dynamic stream property. However, as a rule of thumb if the property is configurable on the Properties Form of a database item, that property is stored in the Configuration stream.

The performance impact of the Configuration stream is that the Configuration stream is bigger than the Data stream, and as such it takes longer to save and requires a longer duration of a lock on the Geo SCADA Expert database. The Data stream is kept small to allow quicker saving, whereas the Configuration stream has 'the rest of the information', usually resulting in the Configuration stream being approximately five times the size of the Data stream. Because of this impact, configuration changes should be kept to a minimum when using automated tools to change them (that is, Logic and external programs). These programs can change the configuration, often rendering the performance improvement usually seen as minimal. Rapidly changing data should be stored in data properties instead of configuration properties.

Data Grids are also stored in the Configuration stream and so they are not good as a store for constantly changing data, as a change requires a save of the entire Configuration stream for that database item, whereas Data Tables are stored in the DataFiles stream and are more efficient in their storage and transfer.

Go: Home Back