Powerchute PCNS 4.2 Advanced UPS Configuration VMware

BillP · ‎2021-07-09

I have recently been reviewing the configuration of our PCNS 4.2 as it wasn't adequate to our infrastructure.

Basically we have 4 ESXi hosts in a hyperconverged chassis that is connected to 2 x Smart-UPS 3000. And a BCP site with a similar setup (but with only two hosts in the chassis). There is a vCenter Server Appliance v6.7 at the main site and a PCNS 4.2 appliance too. Within vCenter there are two clusters per site.

I found the documentation on PCNS and on PCNS with VMware either high level or scenario orientated. So I thought I would go at running the wizard as I thought it should be setup. So I did an Advanced Configuration and put the two UPS's on each site into two groups to represent the two sites within PCNS. And later I assigned the hosts for each sites cluster to the relevant UPS group. Obviously by this point I have been warned that Advanced UPS Configuration is not supported if PowerChute is installed on a Virtual Machine which mine is. But I thought I would run through it anyway. I was able to complete it.

Now when it states that Advanced UPS Configuration is not supported does that mean it simply doesn't work or is unreliable? Is there a way to configure either PCNS or vCenter to make it work? Will a 2nd PCNS (dependant on license/cost) and two separate redundant configs do the job? I would ask if a physical vCenter is the best way but as it is deprecated in the future I will not go there.

Also the VM Shutdown/Startup settings of 240sec seem a little tight as some heavy servers could take more time and even not shutdown cleanly. What is adequate but within tolerance? Are any remaining VM's powered off at VM level? Likewise with the vCenter what is a good amount of time?

Furthermore what is the nature of the event that will send the trigger as it will be good to avoid false positives (and does it need to be a certain duration or frequency)? Furthermore at what point will other UPS connected equipment be shutdown (SAN's & Switches)?

I am already aware that although there is a test mode it actually will shutdown all VM's and devices. This is awkward as it doesn't give any workable opportunity to test a production infrastructure unless outside of hours which would still be inconvenient, especially if multiple tests were needed and multiple failovers (with offsite equipment) would occur.

Thanks in advance for any help.

BillP · ‎2021-07-09

Thanks again Bill.

I have the two virtual appliances setup as required. Together with the vCenter and local ESXi accounts.

I am trying to get a good assessment of the physical condition of the 4 UPS's before proceeding with the fine details of the configuration and any testing as one battery recently died. And of the two onsite UPS's one reports 4 mins battery life and the other 22 mins. With them all passing self tests it does make we wonder if other failures are imminent (also given they will be three years old soon).

See Answer In Context

BillP · ‎2021-07-09

Hi,

On 11/6/2019 8:48 AM, Simon said:
Basically we have 4 ESXi hosts in a hyperconverged chassis that is connected to 2 x Smart-UPS 3000. And a BCP site with a similar setup (but with only two hosts in the chassis). There is a vCenter Server Appliance v6.7 at the main site and a PCNS 4.2 appliance too. Within vCenter there are two clusters per site.

First, I recommend you install PCNS 4.3 Appliance at each site. Configure each with redundant UPS. To communicate with the single vCenter, and the local ESXi hosts. Also, you will need to create a user account on each host and vCenter that PCNS will use. See page 62 of the User's Guide for help with create a shared account. If the site that is hosting vCenter is off line PCNS at the opposite site will use the account to communicate to the local hosts and VMs.

On 11/6/2019 8:48 AM, Simon said:
Also the VM Shutdown/Startup settings of 240sec seem a little tight as some heavy servers could take more time and even not shutdown cleanly. What is adequate but within tolerance? Are any remaining VM's powered off at VM level? Likewise with the vCenter what is a good amount of time?

There is no way for anyone that has not worked with the equipment to know. You should set the durations to what you feel will be adequate and test the shutdown process then adjust as needed. If testing is not feasible you will need to base the durations on experience with the local equipment.

On 11/6/2019 8:48 AM, Simon said:
Furthermore what is the nature of the event that will send the trigger as it will be good to avoid false positives (and does it need to be a certain duration or frequency)? Furthermore at what point will other UPS connected equipment be shutdown (SAN's & Switches)?

All possible configurable events are discussed on page 144 of the User's Guide. If no events are configured within PCNS a shutdown will be triggered when the UPS reaches the low battery duration set within the NMC web interface.

As for SANs and Switches, PCNS 4.3 provide the ability to run a command file when the shutdown process is occurring, and it also provides the ability to run SSH scripts. You would write a command file or SSH script that will communicate with the SANs and or Switches to power them down.

BillP · ‎2021-07-09

Thank you so much the thorough reply. I have a few questions though.

On 11/6/2019 4:13 PM, Bill said:
To communicate with the single vCenter, and the local ESXi hosts. Also, you will need to create a user account on each host and vCenter that PCNS will use. See page 62 of the User's Guide

As I am using a vCenter Appliance is it sufficient to create accounts that are local to the VCSA and each ESXi host with identical names and passwords?

On 11/6/2019 4:13 PM, Bill said:
There is no way for anyone that has not worked with the equipment to know. You should set the durations to what you feel will be adequate and test the shutdown process then adjust as needed. If testing is not feasible you will need to base the durations on experience with the local equipment.

But the VM shutdowns are all graceful and at guest OS level? Is there a link that covers the complete shutdown process from VM to vcenter to host to UPS?

On 11/6/2019 4:13 PM, Bill said:
All possible configurable events are discussed on page 144 of the User's Guide. If no events are configured within PCNS a shutdown will be triggered when the UPS reaches the low battery duration set within the NMC web interface.

My queries regarding the events are which one(s) will cause the shutdown sequence to start? Basically I want to know what the tolerances are that I am working to as it is not unknown for minor issues to occur in a general day. And by default the Execute Virtualization Shutdown Sequence and UPS Shutdown boxes are not checked and were previously not checked. I must admit I am slightly nervous about changing that.

On 11/6/2019 4:13 PM, Bill said:
As for SANs and Switches, PCNS 4.3 provide the ability to run a command file when the shutdown process is occurring, and it also provides the ability to run SSH scripts. You would write a command file or SSH script that will communicate with the SANs and or Switches to power them down.

By running this script I assume you are referring to initiating a graceful shutdown as they will shutdown regardless when the batteries drain or UPS shuts down.

Thanks again.

BillP · ‎2021-07-09

Hi,

On 11/6/2019 10:59 AM, Simon said:
On 11/6/2019 4:13 PM, Bill said:
To communicate with the single vCenter, and the local ESXi hosts. Also, you will need to create a user account on each host and vCenter that PCNS will use. See page 62 of the User's Guide

As I am using a vCenter Appliance is it sufficient to create accounts that are local to the VCSA and each ESXi host with identical names and passwords?

Correct.

On 11/6/2019 10:59 AM, Simon said:
On 11/6/2019 4:13 PM, Bill said:
There is no way for anyone that has not worked with the equipment to know. You should set the durations to what you feel will be adequate and test the shutdown process then adjust as needed. If testing is not feasible you will need to base the durations on experience with the local equipment.

But the VM shutdowns are all graceful and at guest OS level? Is there a link that covers the complete shutdown process from VM to vcenter to host to UPS?

The User's Guide offers diagrams that show the process. See sample shutdown scenarios starting on page 101.

On 11/6/2019 10:59 AM, Simon said:
On 11/6/2019 4:13 PM, Bill said:
All possible configurable events are discussed on page 144 of the User's Guide. If no events are configured within PCNS a shutdown will be triggered when the UPS reaches the low battery duration set within the NMC web interface.

My queries regarding the events are which one(s) will cause the shutdown sequence to start? Basically I want to know what the tolerances are that I am working to as it is not unknown for minor issues to occur in a general day. And by default the Execute Virtualization Shutdown Sequence and UPS Shutdown boxes are not checked and were previously not checked. I must admit I am slightly nervous about changing that.

Each event that will trigger a shut down can be configured with a delay. Example: UPS on battery has been selected to tirgger the shut down process. You would enter a delay so that the process will only start after the UPS has run on battery for 10 minutes. So when the UPS switches to battery it informs PCNS. PCNS then wait 10 minutes and if the UPS does not report the event has cleared after 10 minutes the process begins.

You can also configure runtime remaining below threshold to trigger the process when the UPS falls below a certain runtime. Example: UPS on battery and the runtime falls below 20 minutes the shut down process is triggered

On 11/6/2019 10:59 AM, Simon said:
On 11/6/2019 4:13 PM, Bill said:
As for SANs and Switches, PCNS 4.3 provide the ability to run a command file when the shutdown process is occurring, and it also provides the ability to run SSH scripts. You would write a command file or SSH script that will communicate with the SANs and or Switches to power them down.

By running this script I assume you are referring to initiating a graceful shutdown as they will shutdown regardless when the batteries drain or UPS shuts down.

. The script can be configured to run when a specific event happens or when the shut down process is triggered. Example: set the script to run after UPS has been on battery for 5 minutes.

BillP · ‎2021-07-09

Thanks again Bill.

I have the two virtual appliances setup as required. Together with the vCenter and local ESXi accounts.

I am trying to get a good assessment of the physical condition of the 4 UPS's before proceeding with the fine details of the configuration and any testing as one battery recently died. And of the two onsite UPS's one reports 4 mins battery life and the other 22 mins. With them all passing self tests it does make we wonder if other failures are imminent (also given they will be three years old soon).