Welcome to the new Schneider Electric Community

It's your place to connect with experts and peers, get continuous support, and share knowledge.

  • Explore the new navigation for even easier access to your community.
  • Bookmark and use our new, easy-to-remember address (community.se.com).
  • Get ready for more content and an improved experience.

Contact SchneiderCommunity.Support@se.com if you have any questions.

Close
Invite a Co-worker
Send a co-worker an invite to the Exchange portal.Just enter their email address and we’ll connect them to register. After joining, they will belong to the same company.
Send Invite Cancel
84609members
353906posts

[Imported] period merge of remote clearscada historic points db with local clearscada historic db

EcoStruxure Geo SCADA Expert Forum

Find out how SCADA systems and networks, like EcoStruxure Geo SCADA Expert, help industrial organizations maintaining efficiency, processing data for smarter decision making with IoT, RTU and PLC devices.

Solved
sbeadle
Janeway Janeway
Janeway
0 Likes
1
474

[Imported] period merge of remote clearscada historic points db with local clearscada historic db

>>Message imported from previous forum - Category:ClearSCADA Software<<
User: Roughneck42, originally posted: 2019-04-17 17:23:13 Id:409
we have two pump stations running 5k single clearscada servers. approximately 3k historical points per station. our main plant is running two 25k hot standby clearscada servers with approximately 15k historical points. We would like to merge the pump station historical data up to the main plant scada servers, comms between the two may be interrupted occasionally but is normally a megabit link. I was looking at EWS but this seems to want me to add in every single pump station point individually (groan) I am also worried that ews may be too slow as it seems to use xml.

I'd appreciate some alternative suggestions thanks in advance


Accepted Solutions
sbeadle
Janeway Janeway
Janeway
0 Likes
0
473

Re: [Imported] period merge of remote clearscada historic points db with local clearscada historic db

>>Responses imported from previous forum


Reply From User: adamwoodland, posted: 2019-04-18 00:03:41
https://tprojects.schneider-electric.com/telemetry/display/CS/Comparison+of+Methods+to+Transfer+Poin... might be a good starting point for requirement discussions


Reply From User: sbeadle, posted: 2019-04-18 07:53:02
Ways to make living with a multi-tier system simpler:
1 - use a script to export/import config between tiers and convert object types, setting the properties
2 - use templates on the lower and upper tiers, similar but with the appropriate types. (You can use Template Expressions to set the EWS address property automatically).


Reply From User: du5tin, posted: 2019-04-18 17:23:01
EWS is crazy fast until it isn't. If you try to pull too much historical data it will timeout and crash, then restart. We are starting to use it more for these multi-tiered systems though. For points with minimal historic data it works great and will backfill as required.

Like sbeadle suggests, you can probably export/import, do some sort of bulk conversion to EWS points and then update the tags with a script.


Reply From User: Roughneck42, posted: 2019-04-22 15:56:59
thanks for the response. The comparison table is a good start but how much is too much data when it comes to EWS?


Reply From User: sbeadle, posted: 2019-04-23 16:22:27
'It all depends'!
Allow (say) 50 bytes per value change/time and do a calculation. (This is my guess, I have not looked but it is the right order of magnitude).
Arrange 'scanning' so the client rarely asks for more than 50K of data per point (again, a guess).


Reply From User: du5tin, posted: 2019-04-23 16:41:32
I do not have a good rule of thumb for what is the best configuration. I do know anecdotally that EWS handled syncing history on points that were polled every 30 minutes for a time span of a year without issue (and this was fairly quick!). But we had trouble with facility points that had history records every 2-3 seconds (as some HMIs have) and 6 months of history. We saw high dbserver memory usage on the source system and EWS driver timeouts/freezes on the destination system.


Reply From User: Roughneck42, posted: 2019-04-23 17:48:26
awesome, thanks for the info.

See Answer In Context

1 Reply 1
sbeadle
Janeway Janeway
Janeway
0 Likes
0
474

Re: [Imported] period merge of remote clearscada historic points db with local clearscada historic db

>>Responses imported from previous forum


Reply From User: adamwoodland, posted: 2019-04-18 00:03:41
https://tprojects.schneider-electric.com/telemetry/display/CS/Comparison+of+Methods+to+Transfer+Poin... might be a good starting point for requirement discussions


Reply From User: sbeadle, posted: 2019-04-18 07:53:02
Ways to make living with a multi-tier system simpler:
1 - use a script to export/import config between tiers and convert object types, setting the properties
2 - use templates on the lower and upper tiers, similar but with the appropriate types. (You can use Template Expressions to set the EWS address property automatically).


Reply From User: du5tin, posted: 2019-04-18 17:23:01
EWS is crazy fast until it isn't. If you try to pull too much historical data it will timeout and crash, then restart. We are starting to use it more for these multi-tiered systems though. For points with minimal historic data it works great and will backfill as required.

Like sbeadle suggests, you can probably export/import, do some sort of bulk conversion to EWS points and then update the tags with a script.


Reply From User: Roughneck42, posted: 2019-04-22 15:56:59
thanks for the response. The comparison table is a good start but how much is too much data when it comes to EWS?


Reply From User: sbeadle, posted: 2019-04-23 16:22:27
'It all depends'!
Allow (say) 50 bytes per value change/time and do a calculation. (This is my guess, I have not looked but it is the right order of magnitude).
Arrange 'scanning' so the client rarely asks for more than 50K of data per point (again, a guess).


Reply From User: du5tin, posted: 2019-04-23 16:41:32
I do not have a good rule of thumb for what is the best configuration. I do know anecdotally that EWS handled syncing history on points that were polled every 30 minutes for a time span of a year without issue (and this was fairly quick!). But we had trouble with facility points that had history records every 2-3 seconds (as some HMIs have) and 6 months of history. We saw high dbserver memory usage on the source system and EWS driver timeouts/freezes on the destination system.


Reply From User: Roughneck42, posted: 2019-04-23 17:48:26
awesome, thanks for the info.