Understanding UPS / NMC / PowerChute Network Shutdown settings related to shutdown

[BillP]

I am trying to figure out how to configure settings related to running a shutdown script. They confuse me somewhat.

My basic setup is this:

One UPS (Smart-UPS RT 8000 RM XL) with an NMC protects several servers. Two of the servers (A and B) will have PowerChute Network Shutdown.

Server A needs to run a script, which basically shuts all other servers down. Server B just needs to shut itself down - but only after Server A has completed its script and shut itself down.

I cannot say precisely how long the script will take, but I can venture a guess that it will be 10-15 minutes. I will of course test this, but I cannot be sure that it will always take the same amount of time.

Because of this I would prefer if the script be triggered to run once mains power has been out for like 2 minutes. If it has been out for 2 minutes it is probably not coming back right away.

UPS usually reports about 25-26 minutes runtime remaining, so we should be in good shape for that goal.

I see these settings that I might need to adjust, but I am not sure how:

On the UPS NMC, there are many setting under "Configuration, Shutdown". I am confused as to which of these pertain to shutting down the UPS itself (this is not what I want - I think? Why would I want to turn the UPS itself off?) and which relate to shutting down servers through PowerChute Network Shutdown. 

I suspect that I need to at least look at these in UPC/NMC:

Low Battery Duration (currently set to 2 minutes)

Shutdown Delay (currently set to 20 seconds)

Maybe: Force Negotiation vs. 2 minutes

And these in the two different PowerChute Network Shutdown installations:

Shutdown Settings -> Run Command -> Duration

Do I need to enable Shutdown for any of the events in the Configure Events section?

Thanks!

[BillP]

Hi,

If the script needs 15 minutes to run and the available runtime is only 25 – 26 minutes you will need to purchase an external battery pack to increase the available runtime. The reason is the low battery threshold on the UPS/NMC needs to be 2 minutes greater than the script delay. So if you set the low battery setting to 17 minutes Server B will being shutdown when the UPS drops below 17 minutes of available runtime and server A will still be powering down.

That is assuming the UPS actually has 25 minutes of run time available. With the delays set for server A the total time it will take for it to go down would be 2 minute on battery delay + 15 minute script delay + 2 minute (none configurable delay) + 3 minutes for OS to power down for a total of 21 minutes. That is assuming the OS only takes 3 minutes to power down.

BTW: The PowerChute help file has a diagram of how this works.

What I recommend is add an external battery pack then in PowerChute configure the event UPS on battery for server A and set the delay to 2 minutes. So when the UPS has been on battery for 2minutes a shutdown will start. 

Also on server A under Shutdown Setting enter the script you want to run and set the delay to 15 minutes for it to run. When the shutdown begins the script will run and the delay will be honored so the OS will not start to shutdown until the delay time has passed. 

For Server B configure it to start shutdown after the UPS has been on battery for 25 minutes based on the new available runtime and the need for server A to be completely power down.

[BillP]

Hi Bill

Thank you for a quick and thorough reply. Permit me a few follow-ups anyway:

I hope to avoid buying extra capacity: I can tailor the script to run faster by forcing power-off of a number of non-critical virtual machines rather than shutting them down nicely. In addition, it just occurred to me that the estimated runtime is probably too conservative, since I would think it assumes constant load. But in reality, the load will be dropping quite quickly while the script is running as servers are being shut down. Towards the end only a few servers will be running. I can also improve the script to do more stuff in parallel.

Even so: I want to understand the reasoning behind your recommendations, since I am still somewhat confused.

First: Let’s say for arguments sake, that I followed your recommendations wrt. a battery pack. As I understand you, I would then only have to adjust settings in PowerChute, correct? Nothing in NMC? It makes sense to me, but I want to be sure.

Second: You say that I have too little time with my numbers, but wouldn’t I just about make it, if they hold true? 2 + 15 + 2 + 3 + 3 = 25. Like this:

2 minutes for low battery threshold

15 minutes for the script

2 minutes for non-configurable delay

3 minutes for server A to shut itself down

3 minutes for server B to shut itself down

Is this math correct?

And is your motivation for the external battery recommendation then based on the fact that I would be cutting it far too close for comfort (agreed)? Or is there another reason I am missing?

I need to know because I need to know how much time I have to shave off the script. And to understand if I am being completely unrealistic in what is possible.

Thanks!

Thomas

[BillP]

Hi,

On 7/29/2016 4:34 PM, Thomas said:

First: Let’s say for arguments sake, that I followed your recommendations wrt. a battery pack. As I understand you, I would then only have to adjust settings in PowerChute, correct? Nothing in NMC? It makes sense to me, but I want to be sure.

You should change the low battery threshold in the NMC to be 2 minutes greater than the total time needed to complete the shutdown. If you do not do that you will receive an error stating Available runtime has been exceeded. Assuming you do not configure shutdown for the warning in PowerChute nothing will happen other than the warning being present.

On 7/29/2016 4:34 PM, Thomas said:

Second: You say that I have too little time with my numbers, but wouldn’t I just about make it, if they hold true? 2 + 15 + 2 + 3 + 3 = 25. Like this:

With what I outlined above using the numbers for server A to power down 15+2+3 = 20 and add a buffer of 2 so the low battery threshold is recommended to be 22 minutes. That fits within the 25 minutes of available runtime. However, when the UPS hits low battery it report to PowerChute and PowerChute begins an immediate shutdown so when the UPS drops below 22 minutes of available runtime server B will be commanded down. To keep server B up while server A is powering down and running the script you could set the low battery setting at 5 minutes (3 minute OS shutdown and 2 extra minutes as a buffer). If you do this you will see the Available runtime has been exceeded error on server A . As long as you do not configure Shutdown for Runtime Exceeded on server A PowerChute will ignore the error. The error will be posted in the PowerChute event log on server A so users will have to ignore it also. 

Since server B will be commanded down when UPS hits low battery you do not need to configure anything in PowerChute on that server. As I said, once the UPS falls below the configured low battery threshold the NMC reports to all PowerChute clients that the battery is low and they start an immediate shutdown of the OS.

On 7/29/2016 4:34 PM, Thomas said:

But in reality, the load will be dropping quite quickly while the script is running as servers are being shut down. Towards the end only a few servers will be running.

You are correct. As the servers shutdown the runtime of the UPS will increase so if you do set the low battery threshold to 5 minutes you should be safe but the settings should be tested to be sure. 

[BillP]

Hello again

I never got around to getting this set up because I could not quite understand all of your advice. I am very new at this.

However: I have to get it done now since we have a planned power outage on Wednesday and we have decided to use the occasion as a test.

So can I ask you to be extremely clear about which settings I should configure how in NMC and in PowerChute Network Shutdown on the two servers where it is installed?

To reiterate:

Server A needs to run a script and then shut itself down. My present assumption is that the script will complete in 10-12 minutes.

Server B needs to shut itself down. No script. But it should delay as long as possible before initiating shutdown. It is not critical that Server B is live for as long as server A.

NMC lets me pick between 2 and 20 minutes for "low power duration". Please advise on a suitable value.

Please advise which event if any to configure in PowerChute on server A and B.

Please confirm that for server A I should configure a Run Command under Shutdown Settings in PowerChute and point this to my script.

Please advise which value to use for Duration under Run Command on Server A.

Do I need to do anything else?

I know that you originally advised on getting an external battery pack. Obviously I cannot do this now since I am under a tight deadline. So please take that into consideration in your answers. After the test on Wednesday I will know how long the script took and then I will make a decision on whether or not to get the battery pack.

Finally: I see a Windows Service on Server A named "PowerChute Network Shutdown Service", which runs as LocalSystem. Can I assume that my shutdown script will run under this account as well? Then I can grant permissions to that account on the network so that the script can shut down other servers.

[BillP]

Hi,

The shutdown settings will depend on the UPS model and the NMC.  What model UPS do you have and what is the model and firmware revision of the network management card? 

[BillP]

Smart-UPS RT 8000 RM XL

When I try to access the NMC webpage I get this message, which I have never seen before: "The maximum number of web connections has been reached."

[BillP · Accepted solution]

Hi,

Log into the NMC via telnet and reboot the NMC (it will not reboot the UPS). You should then be able to log in.