PCNS workflow stopped

[BrianOwen_apc]

This was originally posted on APC forums on 8/25/2015

---

I had one VM that does not have VMware Tools that may have caused a stop in the workflow. All VMs and one host shutdown fine. Then this troubled VM and the vCenter Server along with two hosts did not shutdown. Why didn't PCNS power off the VM without VMware Tools? Or maybe that wasn't the issue? I am trying to piece this together to make sure this goes more smoothly next time. I would appreate any help to figure out what happened.

-Here's my setup.

• PCNS 4.0 on a physical Windows Server
• VCSA 5.5
• 3 ESXI 5.5 Hosts
• 30 VMs
• 2 vApps
• svc-or-pcns is the service account with admin access to VMware environment

-Attached is the events of the VM that would not shutdown.

-PCNS Event Log

08/21/2015    18:08:44    OR VMware/SAN UPSs: UPS's have switched to bypass in response to the UPS front-panel or a user-initiated software command, typically for maintenance.
08/21/2015    18:09:14    OR VMware/SAN UPSs: UPS critical event: Bypass: For Maintenance.
08/21/2015    18:09:15    OR VMware/SAN UPSs: UPS critical event: Bypass: For Maintenance triggered a shutdown on Hosts 10.8.3.65, 10.8.3.67, 10.8.3.66.
08/21/2015    18:09:16    Enter maintenance mode on Host 10.8.3.65.
08/21/2015    18:09:16    Enter maintenance mode on Host 10.8.3.67.
08/21/2015    18:09:16    Shutting down VMs on Host 10.8.3.65.
08/21/2015    18:09:16    Enter maintenance mode on Host 10.8.3.66.
08/21/2015    18:09:16    Shutting down VMs on Host 10.8.3.67.
08/21/2015    18:09:16    Shutting down VMs on Host 10.8.3.66.
08/21/2015    18:17:16    Shutting down vApp SharePoint in datacenter AFMS-2CON.
08/21/2015    18:17:16    Starting vApp shutdown process.
08/21/2015    18:17:16    Shutting down vApp SA in datacenter AFMS-2CON.
08/21/2015    18:25:16    Host 10.8.3.66 has successfully entered maintenance mode.
08/21/2015    18:25:17    Maintenance mode task cancelled on Host 10.8.3.65 as there are still powered on VMs. Please verify that sufficient time has been configured for VM/vApp/VCSA VM shutdown duration.
08/21/2015    18:25:17    Shutting down Host 10.8.3.65.
08/21/2015    18:25:19    Shutdown Host failed for Host 10.8.3.67.
08/21/2015    18:25:19    Shutting down Host 10.8.3.66.

[BillP]

This reply was originally posted by Bill on APC forums on 8/25/2015

---

Hi,

PowerChute and VMware host cannot shut down any VMs that do not have VMware Tools installed. In your case the VM would have crashed since the host was commanded down. You should install VMware tools on all VMs you want powered down.

The log also shows Shutdown Host failed for Host 10.8.3.67. Normally is indicates a permissions issue. You can review the error log in the PCNS install folder. It should provide insight as to why the shutdown failed.

Is vCenter Server running on a VM and if so is it running on that host?

[BrianOwen_apc]

This was originally posted on APC forums on 8/25/2015

---

Bill, thanks for your reply.

Oh. For some reason, I thought PCNS would power it off. Not a graceful shutdown. Unforuntely, I have VMs that cannot have VMware Tools installed since they are appliances from a vendor.

I just checked the error log and there are no events from the date of the shut down.

Yes, vCenter Server is running on a VM and it was on host 10.8.3.67.

[BillP]

This reply was originally posted by Bill on APC forums on 8/25/2015

---

Hi,

When you view the host protection page is host .67 signified by the Vcenter icon? below host luna.scvmmgal.net has the VM that vCenter is install on.

If not you will need to install VMware Tools on the VM vCenter is running on. I suggest after adding the Tools rerunning PCNS configuration and rediscover vCenter.

[BrianOwen_apc]

This was originally posted on APC forums on 8/25/2015

---

I verified host .67 has the vCenter Server logo next to it on the host protection page and VMware Tools is running on it.

[BillP]

This reply was originally posted by Bill on APC forums on 8/26/2015

---

Brian,

If the error log in the PowerChute folder does not list anything for that date this is tough to troubleshoot. Since PowerChute recognizes that host has vCenter another possible cause is permissions. Again, the error log would help diagnose the cause.

Since the error log is not available could you rerun the shutdown and before doing so put PowerChute into debug mode. Kbase document FA230514 describes how to change PowerChute to debug mode. When the test is done please send the error log for review.

[BrianOwen_apc]

This was originally posted on APC forums on 8/26/2015

---

Gotcha. I will try debug mode next time to get more information.

I have another question. I just used the vSphere client to log into each of the 3 hosts. All 3 have a task started every two minutes 'Find entity by IP' and it was satrted by the service account I use for PCNS. Is this normal?

[BillP]

This reply was originally posted by Bill on APC forums on 8/27/2015

---

Brian,

PCNS uses Find by IP API to discover the host/s. I find it strange that this API would be called continually. Would you provide 1 of the system logs for review?

[BrianOwen_apc · Accepted solution]

This was originally posted on APC forums on 8/27/2015

---

Sure, I would appreicate you looking at the logs. Which log in particular do you want? Or the whole log bundle of one of the ESXi hosts? Do you have somewhere I can upload it to?