PCNS 3.1 with VMware 5.5 issues

[BillP]

Hi,

I am trying to set up PCNS 3.1 with VMware 5.5.

I have a setup identical to setup A of Application Note 180: Single UPS with NMC, physical server with PCNS, two host cluster with vCenter running in a VM on the cluster. I did not install the VMA, since I don’t believe it’s necessary in this setup. Or is it?

I created an AD user with which I can login to the vCenter server or to either host using the vCenter client.

I checked that PCNS correctly detects the host on which the vCenter VM is running.

All timeouts are default.

When I test the shutdown sequence, I get the following:

10/16/2014 14:40:28 Shutting down Host 10.10.10.102.

10/16/2014 14:40:26 Shutting down Host 10.10.10.103.

10/16/2014 14:40:02 Shutting down physical machine that PowerChute is running on.

10/16/2014 14:40:00 Shutting down Host 10.10.10.103.

10/16/2014 14:40:00 Exit maintenance mode: mp-esx2.jaga.local.

10/16/2014 14:36:00 Shutting down VMs on Host mp-esx2.jaga.local.

10/16/2014 14:36:00 Enter maintenance mode: mp-esx2.jaga.local.

10/16/2014 14:35:59 Shutting down VMs on Host mp-esx3.jaga.local.

10/16/2014 14:35:59 Enter maintenance mode: mp-esx3.jaga.local.

10/16/2014 14:35:59 UPS critical event: On Battery occurred on Hosts: mp-esx3.jaga.local, mp-esx2.jaga.local.

So there’s mainly two issues:

1. No mention of a graceful shutdown of the vCenter server. In the server’s logs I get “The system has rebooted without cleanly shutting down first.”.

2. The physical PCNS server is shut down before the two VMware hosts.

I tried entering the vCenter server IP instead of its hostname, but this did not solve the issue.

Since the vCenter server wasn’t shutdown cleanly, the VMs are not automatically started when power is restored.

What am I doing wrong here?

[BillP]

Hi,

When you look at VMware Host Protection page in PCNS does it display the vCenter icon showing the host that vCenter is running on? If not PCNS will power down vCenter the same as any other VM. In most cases the cause for PCNS not to recognize vCenter is running as VM is because VMware Tools was not installed on the VM. Installing VMware Tools on the the VM vCenter is running on should resolve your issue.

If after adding VMware Tools to the VM if you are still having an issue please follow the instructions in Knowledge Base (Kbase) document FA230514 to put PCNS into debug mode and send error.log file for review after rerunning the shutdown test.

[BillP]

Hi,

Thank you for your response.

PCNS shows the vCenter's location just fine:

While troubleshooting, I even tried to move the vCenter server to the other host to see if this change was reflected in the PCNS interface, and that worked fine.

I gathered the requested error-debug.log and attached it to the case.

Best regards,

Wouter

[BillP]

Hi,

Did the host not shutdown? The error log does not show any issue with the shutdown process of the hosts. Also, may I see a copy of pcnsconfi.ini file.

[BillP]

Hi Bill,

In this case only one of the hosts (the one without the vCenter) did shut down.

vCenter server was not shut down, the host it was running on was not shut down.

During testing (this is the fifth run) I’ve had some other results as well, but the main issue seems to be that PCNS fails to shut down the vCenter server.

Please find the requested file in attachment.

Best regards,

Wouter

[BillP]

Hi,

From the logs it shows that PCNS not being able to locate the VCSA VM and its host. It is also not shutting down any of the ESXi hosts prior to local OS shutdown. The ESXi host that is being shut down is actually being shut down using a check that occurs after the local OS shutdown is triggered.

Is it possible to attach the VirtualizationFileStore.properties file?

What is the name of the VM running VSCA?

You may have a machine that is using the same IP address (even if one of them is powered off), this can prevent PCNS from locating the vCenter Server VM and its host.

There is an interface that you can use on vCenter Server called the Managed Object Browser that allows you to check if everything is set-up correctly or if there are problems. It can be accessed viahttps:///mob. You need to log in as a VMware admin.

To check if vCenter Server IP/Hostname/FQDN can be located by what is stored in pcnsconfig.ini for vCenter Server.

Once in MOB. you need to click on theContentLink. Once in the Content Link, click on the SearchIndex link.

If you have added vCenter Server using the IP address - selectFindByIP

In the FindByIP screen, clear the value under datacenter, enter the IP address and set vmSearch boolean "true".

Click Invoke Method.

If the IP address is found in the Inventory you should get a return value and a ManagedObjectReference which will show the VM (the VM name will appear beside it - note that this is the name as it appears in the vCenter Server Inventory, it may not be the same as the IP/FQDN.).

You can click on the MOR reference link  for or more information about the VM (if its found). When you get the MOR value back make sure this is the VM that the vCenter Server VM  is supposed to be not some other  VM that is still present in the Inventory but powered off.

If you added the vCenter Server using the FQDN instead of the IP you use the FindByDNSName function instead.

If the VM cannot be found the MOR value will report back as VOID.

Regards,

B

[BillP]

Hi,

My vCenter server is no VCSA, it’s a virtual machine running 2012R2. Its name is mp-vc55.

The cluster that I’m currently using only has six virtual machines and there are no duplicate IPs.

Testing through MOR was completely OK; it found the correct vCenter server.

Please find the requested file in attachment.

Best regards,

Wouter

[BillP]

Hi,

Initial diagnosis is there is an issue with DNS. Is the domain name server running as a VM?  If it is then you need to add the VMware host info to C:\Windows\System32\drivers\etc\hosts. Another option is to add the host to vCenter using their IP addresses instead of FQDN.

[BillP · Accepted solution]

Hi Bill,

DNS is running on two DC’s, both virtual machines. One of them was indeed running on the cluster that was being shut down!

I entered the VMware hosts’ IPs into the hosts file on the PCNS server and now the shutdown sequence seems OK!

Now all I have to do is change the ”on battery” event to a “battery remaining” and test the startup procedure as well.

Thanks for your help!

Best regards,

Wouter