PCNS v4.2 Configuration - vCenter / ESXi

[stanj_apc]

I am trying to get a clear understanding of the setup process and settings for PCNS and the NMC.  I have exchanged emails with APC support on the VMware side and have been getting different answers. One to time I was informed to set up PCNS as standalone, another response said to set up PCNS to use vCenter, also was told the PNCS Prioritization is only supported to use the VM Prioritization option for ordering virtual machine shutdown and this does not mean that it will not work; only that it has not been tested and will not be supported.

The setup we have is as follows:

APC UPS SMT1500RMU2 with a NMC2 card

• vSphere 6 (ESXi and vCenter Server)
• Active Directory is installed on a Windows Server 2012 VM
• PCNS is installed on a Windows Server 2012 VM
• vCenter Server is in installed on a Windows Server 2012 VM
• There is a local account set up for PCNS on: vCenter Server, ESXi Server, and the PCNS VM
• About 120 VMs, some of which are off and 28 are priority VMs in our infrastructure and are always up and running.

PCNS was set up to use vCenter Server (login via the local account on vCenter).

In vCenter Server client, under the ESXi Configuration tab, selecting “Virtual Machine Startup/Shutdown” and properties, the “Shutdown Action” is set to “Suspend”.  Under the VM Startup Order, Select “edit” and then “Shutdown Settings”, perform action is “Guest Shutdown”. 

One question is will the above work settings allow PCNS to signal the vCenter VMs to shutdown and then Shutdown ESXi Server? 

Also, the settings in PCNS and NMC are as follows:

PCNS:
   Virtual Machine Shutdown/Startup
   VM Shutdown Duration 600 seconds
   VM Startup Duration 300 Seconds

vCenter Server VM Shutdown Duration 360 Seconds

NMC:

   Low Battery Duration 15 minutes
   Basic Low Battery Duration 600 seconds

   Sync Control Shutdown

   Shutdown Delay 600 Seconds

I am not sure how the timing works to allow the VMs to shutdown and then vCenter to shutdown.

I recall one test on a different setup that the VMs all did shutdown but vCenter did not and ended up being tunrned off

thanks

[BillP]

Stan,

On 4/6/2017 11:49 AM, stan said:

PCNS is installed on a Windows Server 2012 VM

We do not support installing PCNS on a Windows VM. The reasoning is PCNS does not recognize that it is a VM and therefore will be powered down along with the other VMs. If you want to install PCNS as a VM you should install the PCNS Appliance or install VMware vMA and install PCNS no that. The other option is to install PCNS on a stand alone server (none VM) and configure it to communicate with vCenter and the hosts. 

[stanj_apc]

According to the PowerChute Network shutdown v4.2 VMware guide, on page 16, vCenter can run on a VM...

"If vCenter Server is running on a VM you must configure an Active Directory account or
shared Local User account for host shutdown commands to work correctly.."  

This option worked in PCNS 3.1 per the guide..

"If vCenter Server is running on a VM, PowerChute waits until its host comes back online
before powering that VM on. PowerChute will then wait for vCenter Server to start before it
starts the rest of the VMs."

We can't use the appliance as it is ruing an unapproved OS per our security guide.

I am not sure what OS version is on the VMA but it may also be on our unapproved list.   

We used 2012 Server to install PCNS as this was approved OS per our guide.

[BillP]

Stan,

Yes. vCenter Server can run on a VM but PCNS cannot. When PCNS starts the shutdown process it first put the host into maintenance mode and then starts the VM shutdown. Since PCNS does not recognize that the system it is running on is a VM that Windows VM will be powered down before vCenter Sever and the hosts. If you install PCNS on the same Windows VM that vCenter Server is running on then it will be powered down just prior to the hosts so the hosts will not power down. 

You can install PCNS on a stand alone Windows 2012 Server and configure it to work with vCenter Sever. That is your best option since you do not want to run the appliance. 

[stanj_apc]

Bill,

I misread the previous.  I went through the PCNS Network Shutdown v4.2 install and on Page 58 see the steps to install on Windows. There is also steps to install the Virtual Appliance and on vMA, but I can't find any mention of where it states PCNS cannot be installed on  a VM.  When I see the section "VMware", that to me indicates VMs.  Maybe a note should be added or clarification about PCNS not running on a VM.

So, I need to un-install PCNS from the PCNS Windows 2012 Server VM I have, and install PCNS on the vCenter Windows 2012 Server VM?

What do you mean by the statement "You can install PCNS on a stand alone Windows 2012 Server and configure it to work with vCenter Sever".

This is what I have currently or do you mean a physical server?

Stan

[BillP]

Hi,

On 4/6/2017 3:15 PM, stan said:

What do you mean by the statement "You can install PCNS on a stand alone Windows 2012 Server and configure it to work with vCenter Sever".

A stand alone system is a physical server not a VM. When you install on the physical server you are given the option of including VMware support. See FAQ FA250921 and FA311016 for assistance. 

If you install PCNS on a Windows VM it will not work properly. 

[stanj_apc]

ok,

Since we cannot add a physical server, use the virtual appliance, or use vMA, it appears I only have the option to  install PCNS on the same Windows VM that vCenter Server is running so vCenter Server will be able to then be powered down prior to the ESXi server shutting down ?

Another question is about using the PCNS Virtual Machine Prioritization for shutting down VMs.  We do not have a cluster and will be using the vCenter Server "Virtual Machine Startup/Shutdown" order setting.  The question I have, can we use the settings in the image below to suspend some VMs and than use the individual VM setting to have the "shutdown action" set to "Guest Shutdown"?

[BillP]

Stan,

Please note that what you are suggesting is not supported. It should work to get the VMs shutdown however as I have written in the past the host will not be powered down. You will have to manually power down the host before the UPS runs out of battery life. 

The shutdown process is 

1.PowerChute reports that the UPS is on battery.
2.Shutdown delay for the On Battery event elapses. PowerChute starts a maintenance mode task on each Host. At the same time it sends a command to turn off the UPS or Outlet Group if configured to do so.
3.PowerChute starts VM shutdown followed by vApp shutdown.
4.VM/vApp shutdown durations elapse and PowerChute gracefully shutsdown the vCenterServer VM. At this point with your configuration PCNS will be powered down so the process stops. 

[stanj_apc]

Bill,

So,in our environment,  it seems we have limited options to shut down the VMs including vCenter Server VM and the ESXi Host.

1.  Leave PCNS on the Windows VM and it will shut down the VMs and ESXi host but not vCenter VM

2.  Install PCNS on vCenter VM and it will shutdown the VMs including vCenter VM but not ESXi host (have to physically shut it down which is an issue if we lose power in the middle of the night or ona weekend) 

3.  Install PCNS on a physical serve running Windows Server2012 (not possible in our current environment)

4.  use the vMA option  (if the vMA is on our approved list).

On my other question with the screens shown, would what I suggest work ?

I was also not clear on PCNS Virtual Machine Prioritization if it is used ir shutdown is handled in vCener Server

Stan

[BillP]

Stan,

On 4/10/2017 12:12 PM, stan said:

1.  Leave PCNS on the Windows VM and it will shut down the VMs and ESXi host but not vCenter VM

No. PCNS will shutdown the VMs only. The VMs are shutdown first and since PCNS is on one of those VMs the PCNS VM will be powered down and therefore will not power down vCenter or the hosts.

On 4/10/2017 12:12 PM, stan said:

2.  Install PCNS on vCenter VM and it will shutdown the VMs including vCenter VM but not ESXi host (have to physically shut it down which is an issue if we lose power in the middle of the night or ona weekend) 

Correct.

On 4/10/2017 12:12 PM, stan said:

3.  Install PCNS on a physical serve running Windows Server2012 (not possible in our current environment)

How about on an approved Windows Desktop OS? PCNS is supported on Windows 10, 8, and 7 Enterprise & Professional. I recommend 7 or 10 but 8 will work. 

On 4/10/2017 12:12 PM, stan said:

4.  use the vMA option  (if the vMA is on our approved list).

Correct.

On 4/10/2017 12:12 PM, stan said:

On my other question with the screens shown, would what I suggest work ?

Not sure. Has not been tested or approved. 

On 4/10/2017 12:12 PM, stan said:

I was also not clear on PCNS Virtual Machine Prioritization if it is used ir shutdown is handled in vCener Server

We recommend you utilize PCNS prioritize for the VMs. Prioritizing on the individual host many work but it has not been tested when vCenter Server is in use.  

[stanj_apc]

Hi Bill,

On 4/10/2017 2:44 PM, Bill said:

How about on an approved Windows Desktop OS? PCNS is supported on Windows 10, 8, and 7 Enterprise & Professional. I recommend 7 or 10 but 8 will work. 

I was referring to we cannot add another physical server or PC into the racked environment.  However, there is talk about adding a physical lap top for maintenance that would run Windows 7 Enterprise.  But, I am not sure it will be permitted to stay connected.

On 4/10/2017 2:44 PM, Bill said:

We recommend you utilize PCNS prioritize for the VMs. Prioritizing on the individual host many work but it has not been tested when vCenter Server is in use.

Virtual Machine Prioritization described starting on page 38 of the PCNS v4.2 VMware guide, looks to be the process similar to what the vCenter Client "Virtual Machine Startup and Shutdown" offers (which we are using).  Are you saying we should use PCNS  Virtual Machine Prioritization and turn off/disable the vCenter Client "Virtual Machine Startup and Shutdown" order process?

also, in our current setup (PCNS on a VM) or if we move PCNS to the vCenter VM,  is there a way to  then use scripting to have the vCenter VM and the ESXi Serve shutdown in an orderly manner?

[BillP]

Stan,

I have been giving this a lot of thought. As I have written many many times your configuration is NOT SUPPORTED! However, I may have a solution. This solution is also not official support by Schneider Electric technical support. 

First, install PCNS 4.2 on a Windows VM. Do not install on the VM that is hosting vCenter. It should be its own VM and that VM needs to be running on the same host as the vCenter VM. Then configure VM prioritization in PCNS and prioritize all VMs except the VM PCNS is running on. Leave the PCNS VM as un-prioritized. Set the un-prioritized VM shutdown duration to zero. When the un-prioritized group is set to zero the shutdown will be skipped so the VM will not be powered down with the other VMs. On each of the hosts disable virtual machine start-up and shutdown with the exception of the host the the PCNS VM is running on. On that host set the PCNS Vm to be powered down and started up by the host.

In PCNS configure host protection and set the host that is signified as running vCenter to power down last. 

In the servers BIOS configure each system to power on when AC is restored, in PCNS select turn off UPS, and in the NMC select restart when power is restored. 

What should happen is when PCNS start the shutdown process the VMs will be powered down. Then vCenter Server VM will be powered down. Then the hosts. When the final host is powering down it will tell the PCNS VM to power down and then UPS will power down.

The UPS will be commanded down when the VM shutdown process begins so you need to configure the low battery duration and if the UPS has switched outlet groups the outlet group power off delay to be 2 minutes greater than the amount of time needed to complete the total shutdown process.

As an example, the VMs have a total shutdown duration of 5 minutes, vCenter shutdown is 4 minutes, and the PCNS VM delay is 1 minute so the low battery duration and the outlet group power down delay needs to be 12 minutes. 

When AC returns the UPS will restore power to the outlets and the host will power up. Then PCNS VM will be power up by the host. Once PCNS is up it will start vCenter and then the VMs. 

As I said earlier this is not a supported configuration and I have not tested it but I'm fairly certain it will work. Note: You will need to adjust the delays based on the amount of time your systems need. 

[stanj_apc]

On 4/13/2017 9:41 AM, Bill said:

However, I may have a solution.

Thanks Bill for the effort and continued supporting of our issue.  

If we cannot use the physical laptop running windows 7 ent, this is the option to use.

I will read this over the post thoroughly and look to make the desired changes.  It may be in the next few weeks as i make the changes and then need to schedule down time for a test.

I have PCNS installed on the Windows 2012 VM, so that should eliminate needing to reinstall on a new OS.

I will report back if i have any questions or issues once I test this out.

Stan

[stanj_apc]

Hello  Bill,

Our sec team is  checking to see if we can add a physical laptop to the  environment that would be running Windows 7 Enterprise SP1. 

If this is the case, and I may be able to install the PCNS software on the laptop and configure it for vCenter.

Would i still need to use PCNS Prioritization or can I use the vCenter "Virtual Machine Sartup and Shutdown" settings?

Stan

[BillP]

Stan,

I recommend that you use the PowerChute prioritization and disable the startup and shutdown on the hosts. 

[stanj_apc]

Hi Bill,

We received permission to install PCNS on a laptop.  We just tried a test wit mixed results.. 

The VMs look like they were shut down, but vCenter did not shut down and the ESXi Host did not shut down.

Below is some of the log with errors..

Not sure why the laptop that PCNS is running on tried to shut down so early in the process..

Thanks

Stan

09:33:42   Shutdown sequence started on Hosts

09:33:42   Starting Maintenance Mode Task on Host

09:34:03   Shutting down VMs on Host

09:35:43   Available runtime has been exceeded

09:36:26  Shutdown Host failed for Host

09:36:26   Shutting down physical machine that PowerChute is running on

09:36:52   Available runtime is sufficient

09:41:42  ERROR: The ini contains an invalid value for localHostAddress in section Networking

09:41:42    Cannot connect to http://ip address of NMC:80

09:41:40   Cannot connect to vCenter Server. PowerChute may not be able to issue commands to Virtual Machines or Hosts.

09:42:42   PowerChute cannot communicate with Network Communication Card

09:43:55  Cannot connect to http://ip address of NMC:80

10:01:00   ERROR: The ini contains an invalid value for localHostAddress in section Networking

[BillP]

Hi,

There are a few things that need to be corrected. First there is an issue with the localHostAddress that was saved in the ini file. Is the laptop using DHCP and if so has the IP address changed since PCNS was installed? You can correct the IP by stopping the PCNS service (from command prompt logged in a admin enter net stop pcns1. The go to C:\Program Files\APC\group1 (assuming the system is 64 bit and PCNS was installed to the default path). You can open the ini file with a text editor and enter the proper IP then restart PCNS. You should also log into the NMC and verify the proper IP address of the PCSN client has been added to the PowerChute Client list. 

After correcting those things follow the instructions in FAQ FA230514 and put PCNS into debug mode. Run the shutdown test again. Once done let me know and I will create a BOX folder for you to drop the needed files. I will review them or one of our other reps will review them and post back what we found. 

[stanj_apc]

Thanks Bill,

I will teak a look at this later today or next week.  

I will need to check if I can remove files from the system.  If not, what i the next alternative to troubleshoot?

Stan

[stanj_apc]

Bill,

I just checked the .ini file and it has the correct IP address for the laptop.

Not sure then what caused the error.

The NMC PowerChute Client list has the IP of the laptop and an IP of the PCNS appliance we tried a while back

Stan

[BillP]

Hi,

"I will need to check if I can remove files from the system.  If not, what i the next alternative to troubleshoot?"

You can review the files yourself. Troubleshooting for communication and shutdown issues are listed in the User's Guide starting on page 128.

[stanj_apc · Accepted solution]

ok,,,

not sure what the issue is on the INI file ,,, I checked the INI file and the Ip address is correct, so not sure what the issue is with the localHostAddress setting

Stan