Two 40kW UPS's with ~100 servers - how to configure for redundancy?

[Anonymous user]

Our server room has got approx ~100 servers and other associated equipment in it. We've currently got a single 40kW APC UPS maxed out at 99-102% load (I know, baaad idea). We've just put in place a second 40kW APC UPS to bring more power to the room give our forecasted growth. Each UPS is fed from a separate power source from the building. We had to route a new power feed through the building into our room to fuel the new unit.

So here is my question. What is the smartest way to distribute the load of all our servers? It would seem to make sense to have power available in each rack to both UPS units. Since each server has two power supplies, plug one into one UPS and the other into the other UPS. This provides redundancy. The problem is, however, that if one UPS were to go down, the remaining UPS could not pick up the load of the entire room - so does that make this method futile anyway?

The other option is to plug half the servers into one UPS and the other half into the other UPS.

Just not sure which option makes the most sense. We were trying to avoid the situation in which we have some racks running off one UPS and some racks running off the other. so in a power failure it gives us more things to manage. I had hoped the two could work in tandem as one complete power source, but that does not appear to be the case.

Any thoughts would be greatly appreciated.

[Anonymous user]

100%! I hope you have a really good generator 🙂
I would suggest you to utilize a hybrid approach. That is do not load balance all of your equipment, since you do not want to overload your 1st UPS if the 2nd fails, especially if you are anticipating growth.
You would have 2 PDUs in each rack, connected to the different UPSs. First, identify your critical equipment, I am sure you already done that. The critical equipment would be connected to both of the PDUs that gives you power redundancy. The low priority equipment should be connected to only one UPS, which you will "round-robin" between your UPSs, keeping power draw in mind. One thing to note is that servers utilizing both power supplies actually pull more power than if using only one power supply. This should give you about 60% utilization on your UPSs, when both are online. That number will go up if a UPS becomes unavailable. If you do not have low-priority equipment, you should consider buying more battery cabinets. Hope this helps.

[cmarler_apc · Accepted solution]

Hey Road Hog, you can't make an 80KW UPS out of two 40KW UPS unless they can operate in parallel. It requires phase synchronization circuitry that is too expensive for this product line. They won't work in tandem as one complete power source, sorry.

One option you didn't mention is to distribute standard utility power to all your racks, to one side of the dual power supply servers B feed. Then feed the other side of the servers dual power A feed from UPS. Feed half your racks from one UPS and half your racks from the second UPS. Since you are thinking dual UPS, dual cords to plan for a UPS failure, it will give you the ability to spread your UPS risk around.

..now to make things even more complicated, well, you are doing power work anyway, might as well do it right.... if you decide this option works for you, use 3-phase rack power distribution for the new utility power B feed. That way you can plug one server's power supply into "house power" phase 1, and one power supply into UPS power phase 2... alternate phases as you deploy this power distribution strategy and you will have gained phase redundancy.

You'll also be positioned to assign physical and logical redundant servers to applications and have them on separate UPS, there's some brownie points for you.