Best way for configuration and shutdown VMWARE

[knowlesy_apc]

Hi,

currently my company has the following 2 x APC Smart UPS SRT 10000 or SRT10KXL with battery packs, we have 4 pdu's coming off a STS which linsk between each of the UPS's, currently all the STS's are set to the one UPS (1) so the idea if that fails they will jump to UPS 2.

We have two VMWARe Esxi hosts that are in a cluster and are managed by a vsphere server in a different location/site which is connected through our MPLS link. 

Is it possible for the APC PC4 appliance to be able to perform a shutdown to the hosts / vms without a connection to the vsphere/vcentre server as the link granted has a 99.9% uptime I wouldnt want that to be the issue if we lost power or had an outage on the line and an automatic shutdown wouldnt be enabled. 

Another question is can the virtual appliance shutdown physical servers as well ? 

Thanks

[knowlesy_apc]

*bump*

[BillP]

Hi,

PowerChute Network Shutdown can be used. Please review Appliction Note 180. You will need to create a local users account that PowerChute will use to connect to vCenter Server and each host in the event that vCenter is down-off line (see page 16 of the application note).

[knowlesy_apc]

Hi Bill,

Thanks for your reply! some more qustions 

• How do I go about shutting physical servers down 
• How would I link an apc netbotz 570 to the ups to notify it ? 

[BillP]

Hi,

On 8/21/2015 8:39 AM, P said:

• How do I go about shutting physical servers down 

Do you mean the host? PowerChute shuts down the host as described in the application note and users guide. If you are asking about a system that is not a VMware ESXi host then to shut that system down you would install PowerChute on the individual OS.

On 8/21/2015 8:39 AM, P said:

How would I link an apc netbotz 570 to the ups to notify it ?

The Netbotz 570 does not communicate with the NMC.

[knowlesy_apc]

Hi Bill,

thanks again for your help here!

By physical servers I have two servers, server 2008r2 and 2012 that are coming from a PDU on the UPS how would I go about shutting them down, you say install powerchute on the system, this is where things get murky for me as I appear to be missing somthing,

• what version ?
• do I get that from the appliance?
• does that link into the appliance or is it seperate ?
• Can physical servers and virtual servers both be shutdown from the ups ? 
• Do i need to install it (whatever it is on both physicals or is it like another "service" and then agent's?)

As for the netbotz 570 I have previously been told by a schnieder employee in passing that if I used a AP9810 I could link the two and enable the essentialy flip a switch to alert the ups would that not be correct, In demo I have had this email me from the ups but I havent got to the pcns shutdown state just yet.

thanks  again 

[BillP]

Hi,

I apologize. Yes, you can use the AP9610 with the AP9631 to attach the Netbotz.

As for all of your questions.

On 8/21/2015 1:06 PM, P said:

what version ?

The latest version is 4.0

On 8/21/2015 1:06 PM, P said:

do I get that from the appliance?

The software is available on APC.com http://www.apc.com/products/family/index.cfm?id=127

On 8/21/2015 1:06 PM, P said:

Can physical servers and virtual servers both be shutdown from the ups ? 

A physical server is the hardware that is running a specific OS. That server can be a host of virtual machines or just a single OS. Example of a host is a server running VMware ESXi 6.0. A single server would be a system running Windows 2008. Hosts can be either stand alone i.e. not managed by VMware vCenter Server or managed by vCenter Server. Virtual servers are virtual machines running on a host.

If the servers are running Windows 2008 (not a host) you can install PowerChute Network Shutdown directly on that system and it will shut the OS down. It is possible to have a single UPS/NMC communicate with and shutdown 12700 servers. See Application Note 101 for assistance.

If the servers are ESXi host you can install the PowerChute Appliance and if configured properly it will shutdown the virtual machines and the host those virtual machines are running on. This is discussed in Application Note 180 and the PowerChute VMware user’s guide I have provided link to in past replies.

On 8/21/2015 1:06 PM, P said:

Do i need to install it (whatever it is on both physicals or is it like another "service" and then agent's?)

If you provide a diagram of your configuration I will review and provide assistance as to how to deploy PowerChute.

[knowlesy_apc]

Hi Bill,

Yet again many thanks with your help on this!

With regards to setting up the link beteren the netbotz and ups do you have any white papers or docs on it? I'm mostly concerned about the configuration on the  pcns side of things.

As for the confirmation on difference between physical and virtual cheers, from what I read I was a little concerned that you just point the ups to one central power chute network shutdown service. In which that would be the single and only point of contact to issue the shutdown command and then my understanding was from there the pcns sends out commands to shutdown.

The diagram attached/below shows how my server room is currently set up.

My concern with pcns and connecting  the virtual appliance with vcentre is that my vcentre is held off site and actually controls about 5+ other sites. When I connect it how does it effect vcentre as in what does it do, merely authenticate with it or does it install software. If it does install software what explicitly? I know you can just say esxi host option, howeevr would that work with a esxi host cluster? My worry of the vcenter loss from a downed mpls link has been put to bed after the note on the link to local option. My concern is that I don't want to damage the vcenter and cause issues for other sites. However if it isn't doing much is authenticating and adding an option tab to log in then I can't forsee an issue. 

As for my site a, all the sts switches are currently on switch a with the secondary ups on standby being switch b, would this be  a recommended config, with regards to the sts just failing all over to the secondary ups, would the total load just blow it ? On average I'm running 1500 watts on full load, soon to be rammped up to predicted 2.5k watts. Each sts has a pdu, with 2 of each in each rack, so 2 ups, 4 sts and 4 pdu in total. So each device that has a dual PSU is connected to both pdu per device per rack. Confusing haha hopefully the diagram will explain 

To confirm on the chart/diagram below the virtual hosts are in a cluster and I wouldn't be wanting to migrate the vms across the mpls, merely just shutdown the VMS on them in 3 sepetrate group/times ideally after 1 ups had failed or was drained. 

Another issue is that I can find some brilliant step by step guides of pcns 3 with screenshots etc however for 4 I'm having a hard time finding them and without a test rig I can't replicate it and document it for a change to be raised and to make myself reassured. 

The physical servers not meaning virtual hosts are a backup appliance running 2k8r2 and a CCTV serverrunning on 2k12 in which have lots of disks and will be a big drain so I ideally want them shutdown as quick as possible.

Key

Purple - network

Orange dash - mpls to external site

Green - power from ups 1

Blue - power from up

Red - power from sts to pdu to device but pdu is not shown

Orange box racks 1 or 2

FhpTp3vblAw5L2jFVkcBcw%3D%3D.png

Again Bill many thanks for your help on this, a passing thought, would a white paper be something APC would be interested in doing with me  on this? 

[BillP]

Hi,

You should Install PCNS on both Windows 2008 servers. Configure one of them to power down first. The second server configure it to support VMware and this system will go down last. The second server should be set up to communicate with vCenter Server and with Redundant UPS in the PCNS UI.  Application Note 180 has a similar configuration (page 10) except the diagram shows a single UPS where you have 2 UPS i.e. redundant UPS. Also, for your configuration you are not running PCNS at site B so that part can be disregarded in the diagram on page 10.

How this will work is

Configure Windows server 1 to power down after being on battery for 5 minutes (You can set the delay to your preference)

Configure Widows server 2 to power down after being on battery for 15 minutes (You can set the delay to your preference)

UPS 1 goes on battery nothing will happen

UPS 2 goes on battery nothing will happen

Both UPS go on battery PCNS starts the shutdown delay

5 minutes pass and AC is not restored PCNS start shutdown on Windows server 1

15 minute pass and AC is not restored PCNS start shutdown on Windows server 2

PCNS from server 2 will put both ESXi hosts into maintenance mode

PCNS from server 2 will start VM shutdown on the ESXi hosts. There is a default delay of 2 minutes for the VMs to be powered down This delay is configurable and should be change if your VMs need longer than 2 minutes to power down

After the delay PCNS from server 2 commands the hosts down

After the hosts have been commended down PCNS on server 2 starts the shutdown of itself

For help installing and setting up PCNS on Windows  you should read the PowerChute Installation Guide. Installing on Windows for VMware is described starting on page 47. Installation instruction for Windows not configure for VMware starts on page 6.

For help with the set up see PowerChute VMware User's manual

For help with PCNS on Windows see PowerChute Standard User's Guide

[knowlesy_apc]

HI Bill

Sorry to sound like I'm questioning you here, I'm just eager to piece everything together in my head

• Any particular reason to not go with the VMA appliance?
• Would using the pcns on a physical to control the virtual hosts be better overall if the mpls went down?
• With regards to physical my concern is that it will be harder to backup the config / restore where as a VM we have veeam and can have it back up and running if an issue occurs in 10 minutes along with this I would probably be installing it on the 2k12 CCTV server as the 2k8r2 will be removed in 6 months. However I prefer fewer services per server for maintenance / downtime and troubleshooting. Unless it is super easy to restore the config encase there are issues or the physical approach proves more reliable?
• From memory you can't put hosts into maintenance mode without shutting down the VMS first or migrating them which I don't want to do.
• With regards to the shutdown I'm sure I have read that I can perform commands on the VM for special shutdowns for instance we have a lotus notes server which requires commands to be executed for a clean shutdown.is this correct
• With regards to shutdown after 15 minutes can I group servers to do this imnthinking a set after 5 minutes a set after 15 and a set after 30 if we are to go the pcns route dictating the shutdown rarther than the ups time remaining route
• With your suggestion of the on battery time rarther than time remaining is that due to power utilisation in the ups in which a servrr may start a task and put a highrr load on the aups and possible issues in which the ups may have a higher predicted time than available

Again many thanks for your help with this

[BillP]

Hi,

On 8/25/2015 5:51 PM, P said:

• Any particular reason to not go with the VMA appliance?

Since you are not migrating the VMs it may work but we have not tested that scenario and therefore I am recommending a scenario that has been tested.

On 8/25/2015 5:51 PM, P said:

Would using the pcns on a physical to control the virtual hosts be better overall if the mpls went down?

This should not make any differance.

On 8/25/2015 5:51 PM, P said:

With regards to physical my concern is that it will be harder to backup the config / restore where as a VM we have veeam and can have it back up and running if an issue occurs in 10 minutes along with this I would probably be installing it on the 2k12 CCTV server as the 2k8r2 will be removed in 6 months. However I prefer fewer services per server for maintenance / downtime and troubleshooting. Unless it is super easy to restore the config encase there are issues or the physical approach proves more reliable?

PCNS uses .ini file for its configuration and allows for silent installation. If you back up pcnsconfig.ini file and create the silent install file you should be able to restore PCNS is minutes.

On 8/25/2015 5:51 PM, P said:

• From memory you can't put hosts into maintenance mode without shutting down the VMS first or migrating them which I don't want to do.

No. PowerChute put the host into maintenance mode and then starts the VM shutdown

On 8/25/2015 5:51 PM, P said:

• With regards to the shutdown I'm sure I have read that I can perform commands on the VM for special shutdowns for instance we have a lotus notes server which requires commands to be executed for a clean shutdown.is this correct

PowerChute can run a command file / script that you have written prior to shutdown.

On 8/25/2015 5:51 PM, P said:

With regards to shutdown after 15 minutes can I group servers to do this imnthinking a set after 5 minutes a set after 15 and a set after 30 if we are to go the pcns route dictating the shutdown rarther than the ups time remaining route

 No. Not in the scenario I proposed for the ESXi host. However, if you install multiple PowerChute clients on individual servers you can configure each to start the shutdown at a different time interval.

If you want to prioritize the shutdown of VMs you will be able to do that when we release PCNS 4.1. I expect ot see 4.1 on APC.com in the first or second week of September.

On 8/25/2015 5:51 PM, P said:

• With your suggestion of the on battery time rarther than time remaining is that due to power utilisation in the ups in which a servrr may start a task and put a highrr load on the aups and possible issues in which the ups may have a higher predicted time than available

It was an example. You can select run time remaining or other option. It is up to your organization to determine what works best for your needs.

[knowlesy_apc]

Hi Bill thanks again,

I think this is the part I'm getting confused on, 

On 8/26/2015 7:25 PM, Bill said:

 No. Not in the scenario I proposed for the ESXi host. However, if you install multiple PowerChute clients on individual servers you can configure each to start the shutdown at a different time interval.

If you want to prioritize the shutdown of VMs you will be able to do that when we release PCNS 4.1. I expect ot see 4.1 on APC.com in the first or second week of Septembe

is pcns not a server / client setup it is merely just a client that can plug into VMware and perform a shutdown? Ikeep thinking its a central service and the mention of agents / clients to me suggest I then deploy a small package to other servers?

With regards to the shutdown groups I was under the impression on the virtual appliance I could perform shutdown of certain VMS at certain times ? Are you suggesting it may be better to deploy full pcns to every virtual server unless I wait for version 4.11?

Thanks again

[BillP]

Hi,

On 8/26/2015 6:20 PM, P said:

is pcns not a server / client setup it is merely just a client that can plug into VMware and perform a shutdown? Ikeep thinking its a central service and the mention of agents / clients to me suggest I then deploy a small package to other servers?

PCNS is a Agent/Client application that works in conjunction with UPS and network management card. PCNS listens for information from the assigned network card and reacts to that information based on the assigned configuration.

On 8/26/2015 6:20 PM, P said:

With regards to the shutdown groups I was under the impression on the virtual appliance I could perform shutdown of certain VMS at certain times ?

With PCNS version 4.0 all VMs are commanded down at the same time. With PCNS version 4.1 you will be able to prioritize the shutdown of groups of VMs.

On 8/26/2015 6:20 PM, P said:

Are you suggesting it may be better to deploy full pcns to every virtual server unless I wait for version 4.11?

No. You should not deploy PCNS on each VM. If you want to prioritize the shut down of the VMs you should wait for PCNS version 4.1.

I anticipate version 4.1 will be available for download from APC.com on or around the week of 6 September. I will post the release to the forum.

[knowlesy_apc]

On 8/27/2015 3:26 PM, Bill said:

PCNS is a Agent/Client application that works in conjunction with UPS and network management card. PCNS listens for information from the assigned network card and reacts to that information based on the assigned configuration.

Cheers Bill, reading all the different  notes/guides etc made me a exceptionally confused at how that specifically works, cheers for clearing that one up.

On 8/27/2015 3:26 PM, Bill said:

With PCNS version 4.0 all VMs are commanded down at the same time. With PCNS version 4.1 you will be able to prioritize the shutdown of groups of VMs.

it was sequenced server shutdown that i was thinking of see attached in which you would do what you say in which you install the pcns on each server

Thanks Bill I will be playing around with this tomorrow so expect some updates then 

Thanks again 🙂 

[knowlesy_apc]

Hi Bill,

Couple of questions

• With regards to outlet group, with the setup I have shown above it looks like this isnt applicable to me as I wont be sending for the outlets on the back of the ups to shutdown, as each outlet on the back of each ups connects to a sts and that sts connects to a pdu, this is somthing I can ignore correct ? 
• I would like to test shutdown the server the PCNS is installed on however how do not put the hosts into maintenance mode, I have selected to not shutdown the virtual machines in Virtualisation settings > virtual machine shutdown/startup, I have also tried removing the hosts after the configuration steps and it goes back into setup mode. I can move all vms to one host and remove said host so i have the host with no vms on and the server with pcns to shutdown
• Where do I set for specific virtual machines to have a command file executed? or is it just a blanket across all machines?
• what is the command file just a batch file and how is it executed on the virtual or physical machines ?
• where do I set it so the pcns talks locally just in case the mpls goes offline so where not relying on the vcenter server ? 
• with regards to setting away shutdown if both ups go to power where is that explicitly configured, I went for the redundant ups option in configuration, would it be more ideal to go to advanced and create a group ? as per this document http://www2.schneider-electric.com/resources/sites/SCHNEIDER_ELECTRIC/content/live/FAQS/240000/FA240...

Thanks again 

[BillP]

Hi,

On 8/28/2015 8:57 AM, Pete said:

With regards to outlet group, with the setup I have shown above it looks like this isnt applicable to me as I wont be sending for the outlets on the back of the ups to shutdown, as each outlet on the back of each ups connects to a sts and that sts connects to a pdu, this is somthing I can ignore correct ? 

I recommend shutting down the UPS unless you need it to stay on line to power other equipment to prevent the battery from draining completely. If you have other equipment that needs to stay up then shut the outlet group down. The reason is if the UPS is shutdown it will power back up automatically when AC is restored and the ESXi host will start up automatically when AC is restored. For the UPS and the host to return on AC restore they both must be configured to do so. The setting for the UPS is on the NMC under shutdown and the setting for the host is in the system BIOS.

On 8/28/2015 8:57 AM, Pete said:

I would like to test shutdown the server the PCNS is installed on however how do not put the hosts into maintenance mode, I have selected to not shutdown the virtual machines in Virtualisation settings > virtual machine shutdown/startup, I have also tried removing the hosts after the configuration steps and it goes back into setup mode. I can move all vms to one host and remove said host so i have the host with no vms on and the server with pcns to shutdown

You can remove the host from the host protection page which will remove them from the shutdown list.

On 8/28/2015 8:57 AM, Pete said:

Where do I set for specific virtual machines to have a command file executed? or is it just a blanket across all machines?

It is blanket.

On 8/28/2015 8:57 AM, Pete said:

what is the command file just a batch file and how is it executed on the virtual or physical machines ?

A command file can have the extension .bat or .cmd. It is executed by PowerChute and will be run on the system PowerChute is running on.

On 8/28/2015 8:57 AM, Pete said:

where do I set it so the pcns talks locally just in case the mpls goes offline so where not relying on the vcenter server ?

PowerChute will attempt to communicate with vCenter Server and if it is unavailable it will use the local login credentials as discussed on page 16 of Application Note 180.

On 8/28/2015 8:57 AM, Pete said:

with regards to setting away shutdown if both ups go to power where is that explicitly configured, I went for the redundant ups option in configuration, would it be more ideal to go to advanced and create a group ? as per this document http://www2.schneider-electric.com/resources/sites/SCHNEIDER_ELECTRIC/content/live/FAQS/240000/FA240...

Since you only have 2 UPS working together the best option is to configure as redundant UPS and not use the advanced settings.

[knowlesy_apc]

Cheers Bill

On 2015-08-28 3:55 PM, Bill said:

I recommend shutting down the UPS unless you need it to stay on line to power other equipment to prevent the battery from draining completely.

I dont think this will be possible due to the configuration of the STS each ups powers the sts and due to the lack of advanced confiuration of switching off multiple outlets this wouldnt work in my scenario. IS it possible to power off outlets on smart PDU's ? that would be most benificial! 

On 2015-08-28 3:55 PM, Bill said:

You can remove the host from the host protection page which will remove them from the shutdown list.

thats a shame that i have to have at least one of the hosts in, to test it will certainly be an interesting test.

On 2015-08-28 3:55 PM, Bill said:

A command file can have the extension .bat or .cmd. It is executed by PowerChute and will be run on the system PowerChute is running on.

so in theory I could run a single batch that would have multiple lines of somthing like psexec form sysinternals to execute a special shutdown command for certain servers ? Ideally I wouldnt want to store the passwords in plain text is here a way to run the bat or cmd as a different user ? would changing the serveice that powerchute runs under effect this ? 

On 2015-08-28 3:55 PM, Bill said:

PowerChute will attempt to communicate with vCenter Server and if it is unavailable it will use the local login credentials as discussed on page 16 of Application Note 180.

great!

On 2015-08-28 3:55 PM, Bill said:

It is blanket.

will this be addressed in later versions ? 

On 2015-08-28 3:55 PM, Bill said:

Since you only have 2 UPS working together the best option is to configure as redundant UPS and not use the advanced settings.

so this is the bit i get confused at where in the setup do i state that if both ups go onto battery shut down  ? it appears to be either for all or if one from the looks of it as I cant see any specification ? 

Cheers Bill!

[knowlesy_apc]

*bump*

[BillP]

Hi,

On 8/31/2015 8:13 AM, Pete said:

I dont think this will be possible due to the configuration of the STS each ups powers the sts and due to the lack of advanced confiuration of switching off multiple outlets this wouldnt work in my scenario. IS it possible to power off outlets on smart PDU's ? that would be most benificial! 

With SRT10000KXL you can configure PowerChute to shutdown a single outlet group. SRT10000KXL should have 4 outlet groups Main, 1, 2, 3. Within PowerChute you can specify the outlet group via the web interface.

On 8/31/2015 8:13 AM, Pete said:

so in theory I could run a single batch that would have multiple lines of somthing like psexec form sysinternals to execute a special shutdown command for certain servers ? Ideally I wouldnt want to store the passwords in plain text is here a way to run the bat or cmd as a different user ? would changing the serveice that powerchute runs under effect this ? 

You can change the user the PowerChute service is run under.

Note: Windows services run in session zero so if your command file runs an application that interacts with the desktop you will not see it since as a user you are logged into Windows in session one. Example: command file opens Notepad. Since the service is running in session zero Notepad will be opened in session zero therefore not visible in the user’s session.

[BillP]

Hi Bill,

      It looks like there is a lot of great information here but none of the links are working for me. They all give me 404 errors. Can you update the links?

I found  Application note 180 

Thanks

[BillP]

Scott,

Sorry for the inconvenience. When the documents were update to include version 4.1 the links were broken. You can find all of the documents on the PowerChute Network Shutdown product web page under Manuals and Warranties and Application Notes. 

http://www.apc.com/products/family/index.cfm?id=127

[knowlesy_apc · Accepted solution]

Hi Bill,

can you shed any light onto the following ? 

On 2015-08-31 2:13 PM, Pete said:

so this is the bit i get confused at where in the setup do i state that if both ups go onto battery shut down  ? it appears to be either for all or if one from the looks of it as I cant see any specification ? 

I can see both are set as redundant however I dont want to trigger the shutdown if one says goes onto battery for whatever reason. If its redundant how is the time calculated?  

thanks