PowerChute is not Receiving Data from the Network Management Card

[Anonymous user]

I want to preface this by saying that I've never worked with any APC products or even a Linux OS prior to the last few weeks, and I'm a little in over my head

I'm trying to install PowerChute Network Shutdown on a server running CentOS 7, but it is not receiving data from the Network Management Card, and I can't figure out why. I'm trying to get PowerChute to work on one server before I install it on the others, but I can't get them to communicate with each other.

I can access the NMC from the server through the web interface, and it can connect to the NMC long enough to register its IP address, but it won't receive any data. I checked through netstat to see if 6547 and 3052 are open and listening, and they are, and that makes sense because it automatically registered, but when I trigger a shutdown, the server does not shut down.

To test, I set up my personal Windows laptop with PCNS, and when I trigger a shutdown, my laptop will shut down, so I know the NMC is sending out a signal. Why can't I get the server to communicate? I've attached screenshots I believe might be relevant, as well as a quick diagram of my network setup (the connections are ethernet cables).

Any help would be much appreciated.

[Anonymous user]

Could you explain a bit more about what you're trying to say? Sorry, I'm a little confused

[BillP]

Hi,

When PowerChute registers with the NMC it does so over TCP port 80 or 443 depending on your preference. It then listens for NMC signals on UDP port 3052. After reviewing the screen shots I see that PowerChute is on a different network segment than the NMC. NMC IP 192.168.1.89 and PCNS is 192.168.11.234 What that means is the NMC will send a Unicast message using UDP port 3052 to the PCNS IP. You should verify Unicast traffic is not being blocked. 

[Anonymous user]

On 8/14/2017 5:27 AM, Bill said:

You should verify Unicast traffic is not being blocked. 

How would I do that? If you look at the netstat output screenshot, port 3052 is open and listening. Doesn't this mean that traffic is not being blocked?

[BillP]

Raveen,

I was able to do a quick search on the Internet with regards to Unicast issues on CentOS and I chanced upon this URL that you might want to look at. I have not personally tried it since I do not have any Linux machine on my end. Also, I suggest that you check as well what would the command do on your CentOS.

The command has never been tested and verified by Schneider Electric.

URL --> https://www.centos.org/forums/viewtopic.php?t=52970

[Anonymous user]

Unfortunately, I have already tried this. It did not work the first time, and just now, when trying again, it did not work either. According to my PC, port 3052 is already listening for both TCP and UDP packets being sent to it.

[voidstar_apc]

Here's how I'd go about it:

1) Check if your computer is listed as a Powerchute Client on the NMC. If it isn't, add it. This should've been done by PowerChute's registration step.

2) On your computer, run wireshark and listen for "udp port 3052". If you get no traffic from the NMC to your computer, then something along the path is blocking those packets. You can confirm the NMC is sending the packets by sniffing promiscuously on a hub with the NMC attached, or if using a managed switch, using port mirroring.

3) If you're getting udp port 3052 packets from the NMC to your PC, then make sure your local firewall isn't dropping them (you can kill powerchute and use netcat to listen for packets). Also make sure user name and PCNS passphrase match between the NMC and your PC.

[Anonymous user]

On 8/16/2017 9:20 AM, voidstar said:

run wireshark and listen for "udp port 3052"

Sorry for the delayed response, I was trying to do what you suggested, but honestly, I've never used Wireshark before and I don't have any idea how to do what you suggested. Do you think you could tell me how to listen for ports and sniff promiscuously? If it's too much work, that's fine; I'll keep trying on my own, and I'm thankful for the help you've already given, but even a pointer in the right direction would be a great amount of help.

[Anonymous user]

I'm honestly no closer to figuring out what's wrong.  voidstar I tried most of the things you said. My computer is mentioned as a PowerChute Client on the NMC, but when I ran Wireshark I got no traffic from the NMC to the computer. Admittedly, I have no idea how to mirror a port, so I didn't try that, but I did try disabling the firewall on my computer entirely and even so, PowerChute is still unable to communicate with the NMC. I honestly don't know what else to try.

[voidstar_apc]

Sorry for not replying sooner,

If there's no traffic from the NMC:

1) Confirm netmask setting on the NMC is correct.

2) Plug a laptop with wireshark into the same switch as your NMC. Add your laptop's IP to the list of registered PCNS clients. You should start getting traffic from the NMC -- broadcast traffic specifically because you should be on the same network segment. If yes, then move laptop to the same switch as the server that isn't getting traffic and update the laptop's IP on the NMC. Is the laptop still getting traffic? If not, can narrow down the problem by moving the laptop along the different stages of your network infrastructure.

You're probably well acquainted with wireshark by now but below I've attached a picture of how I setup the capture (Under Capture->Options):

xW8%2FKKVZNctXFIgxFEVSOg%3D%3D.png

If the capture is working, you'll see traffic every time you access an NMC webpage from the same computer as wireshark. In the image below, the first 3 packets are from accessing the NMC's webpage. The UDP 3052 packets are for PCNS, and arrive ~20-30s apart.

AXYXI58N%2B%2BfvxBkjFPWT5g%3D%3D.png

[Anonymous user]

Thanks again for your help. I plugged my personal laptop into the switch that both the gateway and the NMC are plugged into, and I successfully captured traffic from the NMC. Next, I plugged my laptop into the gateway (which the server I have been working on is plugged into), and I discovered that because my laptop isn't setup to work through the gateway, I couldn't connect to the network at all. My laptop wouldn't connect to the network, so I couldn't try to capture traffic.

If you have any questions about the setup I have been working with, it is depicted in the crude diagram I made titled "UPS Setup.PNG" in my first post. I just realized I accidentally used the term "Wi-Fi Router" when I meant "Network Switch," though.

Anyway, my inability to connect to the network through the gateway leads me to believe that what's blocking the server I have been working on from communicating with the NMC is the gateway itself. I don't know exactly how this configuration was made, nor do I have superuser privileges on anything other than the server I have been working on, but would I be correct in assuming that if I were able to allow UDP packets to pass through the gateway, my server would be able to receive them? Alternatively, I know that PCNS is designed so that when a computer receives a shutdown signal from the NMC, it rebroadcasts that signal on its own segment, so all computers with PCNS set up on them on the network shut down, regardless of whether or not the NMC directly communicates with them. Would installing PCNS on the gateway solve my problem?

[voidstar_apc]

Your network diagram is good. I don't see an uplink on the diagram -- is this an isolated network or does the gateway have an uplink to the internet? There's so many ways to set up a gateway that it's hard to say anything definitive about the setup (says the guy who made the mistake of trying to use a gateway router as a switch -- total configuration nightmare). The simpler setups have everything on a switch and then the gateway bridges between the switch and the internet. This seems like something different.

You're right that you need UDP 3052 to pass through the gateway if it's bridging the network with the NMC and the network with your server. If there's a network administrator, you could ask them about it.

Another consideration: PCNS doesn't work if the network is down. All the switches and the gateway between the the NMC and the server need to be backed up so they stay up during a power outage. It might be simpler to plug the NMC and the server into the same switch and putting that switch on the UPS' main outlet group.

[Anonymous user · Accepted solution]

On 8/28/2017 11:10 PM, voidstar said:

is this an isolated network or does the gateway have an uplink to the internet?

The switch that the gateway and NMC are plugged into has an uplink to the Internet, so the network is not isolated.

On 8/28/2017 11:10 PM, voidstar said:

The simpler setups have everything on a switch and then the gateway bridges between the switch and the internet. This seems like something different.

It's actually not different at all; there are two switches being used here, but I didn't indicate one of them. The switch I have been talking about has the Internet connection and is connected to the gateway, which is then plugged into another switch, which the servers are all plugged into. Does that make sense?

Anyway, beyond that, this project is part of an internship which ends on Friday, so I don't think I'm going to be the one to solve this problem. Talking to my mentor about the situation, he said he thought this would be an easier project than it's turned out to be, so I'm going to call it quits. Thanks for all your help!