PDU Scripted Port Shutdowns from UPS

[rstellhorn_apc]

This was originally posted on APC forums on 5/13/2008

---

Hey all, I've got a quick question.

With a bit of coaxing, my company just acquired a new Symmetra 16kva ups to power our entire server room along with a bunch of AP7901 Switched Rack PDU's to replace a tonne of aging individual rack mount UPS's. I've got the whole thing up and running with the Power Chute Network Shutdown client installed on all of the servers, with the non-critical systems set to shutdown 10 minutes into an outage and the remaining few left to shutdown on the battery low event. The concept being that if power is out for 10 minutes, set the room up to keep the critical gear up for as long as possible. But, from lessons past, I know that the chances of systems actually powering themselves off after the shutdown are 50/50 at best, and there are a few embedded systems that don't support network shutdown at all, hence getting the switched PDU's.

I wanted the PDU's to automatically power down the ports associated with the low priority servers after the systems had a chance to gracefully shutdown to ensure they wouldn't keep using power when the shouldn't. I thought this would be a built in thing as everything is made by APC and where suggested to be used together by our rep. Alas, there was no automation like this to be found... at least by me.

One of my questions is... Am I blind, is there such a feature implemented in the depths of these devices?

Now, coming up empty handed hasn't really slowed me down, at least for a first pass implementation of my evil plan. I went and loaded the NetSNMP tool on one of the critical servers set to stay up as long as possible and created a couple of quick batch files to issue the proper commands. Namely, i created a shutdown script that issue snmp commands for delayed power off to the desired ports, and a power on script that would cancel any timed event (such as a delayed shutdown in progress) and issue a power all on command to the PDU's (to turn back on anything that had been shut off). I then went into the Power Chute Network Shutdown interface loaded on that server and set it to run the startup script on line power restore (with a delay of 2 minutes), and the shutdown command on running on battery (with a delay of 10 minutes). And, finally the last step of setting the delay times on each port in the pdu interfaces to allow the servers time to shut down gracefully. I haven't had a chance to test these in their entirety yet, but i think they should do what I want... I hope.

Which leads me to my next questions. If the answer to my first question is no, then how has anyone else implemented this? I feel like I had to attack this a bit backwards, but it was the best i could do without having to write up a massively complex script to monitor snmp traps, etc...

Thoughts, questions, ridicule???

Thanks.

Robert

[TheNotoriousKMP_apc]

This was originally posted on APC forums on 5/14/2008

---

Other than creating a script, there is no feature to have the outlet turn off after the shutdown is complete. The PCNS communication is with the NMC installed in the UPS, therefore, the UPS has no idea that the PCNS Client's power source is a PDU. So long as the graceful shutdown occurs, that is all it is concerned with. However, I don't see how telling them to turn off after the server is off is going to benefit in the long run. The power draw of the PDU's outlet that's not powering anything would create such a minimal power draw from the overall draw off the UPS that your runtime would probably increase by 5 seconds at the most.

[rstellhorn_apc]

This was originally posted on APC forums on 5/14/2008

---

Ok, that's what I was thinking... but, it sure would be a nice feature to have.

As for the why question, well the reasoning is two fold.
-One, I want to make sure a mis-behaving system that either doesn't power off after doing a graceful shutdown or doesn't even respond to the network shutdown command gets shut off to preserve runtime.
-And, Two, in the normal network shutdown senerio, if the line power where to return after the shutdowns had completed but before the ups ran out of juice and shut itself down as well, the systems would never come back up as their psu's would of never lost power. But, this way, with the PDU cutting power to the system and then re-enabling the power on line power return, the system's would follow their automatic power-on after power loss settings and come back up.

I do agree with you tho, that a properly shutdown system would not really impact the runtime of the ups... I've just learned from previous emergencies that I just don't trust the systems to do the "Right Thing".

[rstellhorn_apc · Accepted solution]

This was originally posted on APC forums on 5/13/2008

---

Hey all, I've got a quick question.

With a bit of coaxing, my company just acquired a new Symmetra 16kva ups to power our entire server room along with a bunch of AP7901 Switched Rack PDU's to replace a tonne of aging individual rack mount UPS's. I've got the whole thing up and running with the Power Chute Network Shutdown client installed on all of the servers, with the non-critical systems set to shutdown 10 minutes into an outage and the remaining few left to shutdown on the battery low event. The concept being that if power is out for 10 minutes, set the room up to keep the critical gear up for as long as possible. But, from lessons past, I know that the chances of systems actually powering themselves off after the shutdown are 50/50 at best, and there are a few embedded systems that don't support network shutdown at all, hence getting the switched PDU's.

I wanted the PDU's to automatically power down the ports associated with the low priority servers after the systems had a chance to gracefully shutdown to ensure they wouldn't keep using power when the shouldn't. I thought this would be a built in thing as everything is made by APC and where suggested to be used together by our rep. Alas, there was no automation like this to be found... at least by me.

One of my questions is... Am I blind, is there such a feature implemented in the depths of these devices?

Now, coming up empty handed hasn't really slowed me down, at least for a first pass implementation of my evil plan. I went and loaded the NetSNMP tool on one of the critical servers set to stay up as long as possible and created a couple of quick batch files to issue the proper commands. Namely, i created a shutdown script that issue snmp commands for delayed power off to the desired ports, and a power on script that would cancel any timed event (such as a delayed shutdown in progress) and issue a power all on command to the PDU's (to turn back on anything that had been shut off). I then went into the Power Chute Network Shutdown interface loaded on that server and set it to run the startup script on line power restore (with a delay of 2 minutes), and the shutdown command on running on battery (with a delay of 10 minutes). And, finally the last step of setting the delay times on each port in the pdu interfaces to allow the servers time to shut down gracefully. I haven't had a chance to test these in their entirety yet, but i think they should do what I want... I hope.

Which leads me to my next questions. If the answer to my first question is no, then how has anyone else implemented this? I feel like I had to attack this a bit backwards, but it was the best i could do without having to write up a massively complex script to monitor snmp traps, etc...

Thoughts, questions, ridicule???

Thanks.

Robert