PowerChute Network Shutdown 4.1 Not shutting down host

[Anonymous user]

Hello all,

I'm having the following situation:
I have 2 Esxi 6.0 Hosts with a few VM's and a VCSA 6.0 and a PowerChute Network Shutdown appliance running.

I got everything configured but now I cant get everything to shut down properly.
The problem is this, first the PwoerChute appliance tries to put the hosts in maintenance mode. All VM's excetp VCSA and the powerchut appliance shut down. Then the Maintenance mode request is cancelled and the VCSA is shut down. The PowerChute appliance sees this and after the configured time the host with VCSA is shut down. But then the PowerChute Appliance is shut down and I'm left with 1 host running no VM's?!

I followed the manual and made a global user on vcenter and the hosts. With that account I can login so it works. I dont' have an AD. I dont see errors in error.log..

Or is the solution to run PowerChute on each host and shutdown that host? That seems to work but itsn't the best solution I think...

Who can help me fix this issue?

Thanks in advance!

Here the last eventlog, here I tried without VM shut down configured in PowerChute but configured in ESXI hosts

10/14/2015    10:59:48    UPS has switched to battery power.
10/14/2015    11:00:48    UPS critical event: On Battery.
10/14/2015    11:00:48    Shutdown sequence started on Hosts 10.10.1.10, 10.10.1.11 in response to UPS critical event: On Battery.
10/14/2015    11:00:51    Starting Maintenance Mode Task on Host 10.10.1.11.
10/14/2015    11:00:51    Starting Maintenance Mode Task on Host 10.10.1.10.
10/14/2015    11:00:51    Exit maintenance mode on Host 10.10.1.11.
10/14/2015    11:00:51    Exit maintenance mode on Host 10.10.1.10.
10/14/2015    11:00:52    Shutting down vCenter Server VM VCSA 6.0.
10/14/2015    11:00:53    Maintenance Mode Task cancelled on Host 10.10.1.10 as VCSA VM is shutting down.
10/14/2015    11:03:59    Cannot connect to vCenter Server. PowerChute may not be able to issue commands to Virtual Machines or Hosts.
10/14/2015    11:03:59    Shutting down vCenter Server VM Host 10.10.1.10.
10/14/2015    11:04:01    Shutting down physical machine that PowerChute is running on.
10/14/2015    13:19:48    PowerChute successfully opened UDP port 3052.
10/14/2015    13:20:15    Communication has been established.
10/14/2015    13:20:16    PowerChute Network Shutdown version 4.1.0 monitoring started.
10/14/2015    13:20:17    PowerChute successfully opened TCP port 3052.
10/14/2015    13:20:17    PowerChute successfully opened TCP port 6547.

And this eventlog with shutdown in PowerChute appliance configured and not in ESXI hosts

10/14/2015    13:20:47    Available runtime has been exceeded.
10/14/2015    13:37:18    UPS has switched to battery power.
10/14/2015    13:38:18    UPS critical event: On Battery.
10/14/2015    13:38:18    Shutdown sequence started on Hosts 10.10.1.10, 10.10.1.11 in response to UPS critical event: On Battery.
10/14/2015    13:38:20    Starting Maintenance Mode Task on Host 10.10.1.11.
10/14/2015    13:38:20    Shutting down VMs on Host 10.10.1.11.
10/14/2015    13:38:20    Starting Maintenance Mode Task on Host 10.10.1.10.
10/14/2015    13:38:20    Shutting down VMs on Host 10.10.1.10.
10/14/2015    13:40:20    Exit maintenance mode on Host 10.10.1.11.
10/14/2015    13:40:20    Exit maintenance mode on Host 10.10.1.10.
10/14/2015    13:40:21    Shutting down vCenter Server VM VCSA 6.0.
10/14/2015    13:40:21    Maintenance Mode Task cancelled on Host 10.10.1.10 as VCSA VM is shutting down.
10/14/2015    13:41:49    Cannot connect to vCenter Server. PowerChute may not be able to issue commands to Virtual Machines or Hosts.
10/14/2015    13:43:22    Starting Maintenance Mode Task on Host 10.10.1.10.
10/14/2015    13:43:23    Host 10.10.1.10 has successfully entered maintenance mode.
10/14/2015    13:43:24    Shutting down vCenter Server VM Host 10.10.1.10.
10/14/2015    13:43:25    Shutting down physical machine that PowerChute is running on.
10/14/2015    13:49:12    PowerChute successfully opened UDP port 3052.
10/14/2015    13:49:39    Communication has been established.
10/14/2015    13:49:40    PowerChute Network Shutdown version 4.1.0 monitoring started.
10/14/2015    13:49:41    PowerChute successfully opened TCP port 3052.
10/14/2015    13:49:41    PowerChute successfully opened TCP port 6547.

[BillP]

Hi,

Both test show that PCNS is attempting to power down both hosts. The log indicates PCNS appliance is running on 10.10.1.11 is this the host that is not powering down? 

10/14/2015    11:04:01    Shutting down physical machine that PowerChute is running on.

10/14/2015    13:43:25    Shutting down physical machine that PowerChute is running on.

You should run the second test again while PCNS is in debug mode. Kbase FA230514 "How to put PCNS version 3.1 and above into debug mode?" will help. Also, you should check the host log to see if it is receiving the power down command. 

[Anonymous user]

That is indeed the host that didn't shut down.

I will try one more time with debug mode and come back here and post the output.

[mchomicz_apc · Accepted solution]

Roel,

I would also check /opt/APC/PowerChute/group1/pcnsconfig.ini file and make sure this line says 'true'

isPCNSonVM = true

I have had one installation where for some mysterious reason PCNS running on a virtual appliance had  this set to FALSE and also this line was present:

shutdownCommand = /opt/APC/PowerChute/group1/bin/shutdown

Effectively, the VM installation thought it was running on a physical machine and was issuing /sbin/shutdown -h command.

Had it been running on a physical host - that would have had the effect of powering it off... Alas one node in the cluster was always staying up. The VM was powering itself off though.

Hope this helps someone!

Michael