GeoSCADA server VM migration to new host

[murrayclay]

Hi All

Our ICT department is planning to migrate our SCADA server virtual machines over to new hosts as part of our VM host machine hardware upgrades. Our SCADA system is around 250,000 point count (1000,000 object count) mainly DNP3 and Modbus. ICT has advised that this is very low risk and can happen as a hot migration with the SCADA servers running (no outage). The process takes around a minute to complete. I would prefer to do with the servers shutdown just to be on the safer side. Does anyone have any advice on anything that might be specific to GeoSCADA that we should be watching out for with this move? 

We have a hot standby pair and 2 permanent standbys – Do people think we are better to migrate all of them in quick succession to avoid having them split across new and old (current hardware was installed around 7 years ago)? Or migrating, say 2, then having a testing period before moving the other two? We are planning to carry out smokes afterwards and perform some performance testing pre and post.

ICT have already done the process on approx. 300 less critical virtual machines without issue and have been saving SCADA for last once they had proven their process.

Thanks in advance!.

[BevanWeiss · Accepted solution]

I assume you have application level redundancy (i.e. you have a hot standby server pair).

I would shut down one server, and then do the migration.  Then start it back up in the new location, then fail over to that server.

Then shutdown the second server, do the migration, start it back up and fail over to that server.

Likewise for additional permanent standby servers / web hosts etc.

Hot migration will result in hard drive locks, and such during the migration.  It will also result in a skip in time.  Geo SCADA Expert won't like either of these things.  So best to avoid it if possible.

I'd do a reasonably big smoke test on the first migration, that's when you'll catch firewall rules / VLAN rules etc that haven't been put in place.  And then just a normal failover test on the second (hot standby) server.

[murrayclay]

Thanks Bevan, Much appreciated.

[tkswathik]

Based on past experience when we migrated the VM to a new host, there are chances that license can become invalid because it is tied to machine Id and the Machine ID itself might be a resultant of VM or Host specific attributes. So please make sure you factor that in your plan because if it does end up breaking the license you have to reach out to your local support rep or Schneider Electric support to get a new license file attached to the new Machine Id. 

[JChamberlain1]

The tech that does the migration relies on the same thing that snapshotting tools do, VSS. GeoSCADA is not VSS aware (as opposed to many office and database type applications which IT teams are more familiar with), so anything that does this is a potential database corruption.

I would be shutting them down and migrating them one by one.

[BevanWeiss]

That can happen if the people doing the migration have either used passthrough MAC addresses, or have allowed the MAC address to be re-generated on the migration.

It's a reasonably common thing to be aware of with VMs and licencing in general.  The same can happen with real physical servers also, if the network interface card is changed over, the licence keys are usually tied to a combination of HDD serial numbers and MAC IDs.  Certain software also ties it to CPU ID (but I don't think this is done in VM times anymore).

Schneider are normally very good about providing updated licence keys.  But it is something to be mindful of, since it can be a gotcha.

[murrayclay]

Thanks all. I had discussed IP and mac addresses and was assured that they will not change. We will be carrying out the migration one by one with DB shutdown.