RM6000 in Bypass Mode with 4 Power modules and 3 new batteries

[BillP]

Greetings..

We have an RM6000 in the server room.  Due to it being in bypass mode for a prolonged period of time, 3 of the batteries were trashed.  We replaced three batteries, so the unit now has 4 power modules, and 3 batteries now charged to 99%.  Problem is, when we attempt to take the unit out of bypass mode, the unit does not supply power to servers attached to it, so we manually put it back into bypass. The unit is now showing 4 power modules OFF and OK, and Battery status: 1,2 and 3: OK.  4,5 and 6 occupied by power modules.  It cannot be taken out of bypass mode via the console: it does not recognize the command. The MIM and RIM show ON and OK. Battery Frame shows Battery 1: not installed; Battery 2: OK; Battery 3 and 4: Not installed.  Warranty Alarm Present: In Bypass: For maintenance. Shows a run time of 1 hour 33 minutes.

Any help on what needs to be done next would be much appreciated.

Thanks

[BillP]

No, our unit still will not come out of bypass.  We attempted to troubleshoot according to Rob's suggestions, but nothing seems to work.  he suggested we visually check the bays for visible damage, the rear of all the modules, and tried to unplug all modules and batteries, and add one by one to see if one was actually defective...nothing worked. We might be stuck with calling APC service at $1K per visit to help us resolve...what a nightmare!

[BillP]

Just read this entire thread, but I did not spot a report on the exact state of the UPS when the maintenance bypass switch on the back is disengaged.

For example, when you flip that switch, what LEDs are illuminated on the intelligence modules? What LEDs are lit on the PowerView display? What does the UPS show as its input voltage/frequency?

It just doesn't add up that the UPS would allow the load to be powered down when the intelligence modules are otherwise OK. We would expect to see a fault if the UPS was unable to drive it's bypass contactor while the power module inverters are activated and their output verified OK.

It's behaving as if you are, per Rob's earlier comment, opening the INPUT Circuit Breaker rather than opening the Maintenance Bypass switch. That would be the only logical explanation for a UPS that is in maintenance bypass without any faults suddenly powering down the load when a switch on the rear of the UPS is flipped. Consider that while you are powering your load via the maintenance bypass switch, the UPS is also backing up that bypass with it's own logically driven bypass which stays active when the maint bypass is disengaged, awaiting confirmation that the inverter output is OK to deliver to load.

The only way to drop your equipment from maint bypass without seeing a bunch of faults is to open the input breaker.

The input breaker is in the top right and the maint bypass is below it in the middle-right of the UPS rear panel. See attached image to confirm location of these switches. You should be flipping the switch highlighted in green to the DOWN position to disengage maintenance bypass.

[BillP]

[BillP]

Thanks Rob:

Tried replying with pics but it only allows me to include one shot, and deletes my text accompanying it.

The old battery modules were removed before the new modules were installed.

When the Symmetra is taken out of Bypass manually ( via switch on back), no connected devices receive any power at all.

The unit will not respond to commands issued from the console.

I will attempt to include screenshots of the console screen.

Thanks again

[BillP]

[BillP]

Attached screenshots.

Thanks

[BillP]

Will provide raw status on all modules.  When the UPS was manually taken out of Bypass ( correct switch on back), all power modules do not spin. The Load On light never lights.  When the bypass switch is flipped ON again, power is now supplied to the devices attached to the APC. I'll get some shots of what exactly happens when it is taken out of Bypass in a few days, but for now, I will include pix of the console status of each module. We have tried taking out, and re installing power modules, however, the load On led never lights we taken out of bypass using the correct switch on the back of the unit. I thought what should happen, according to Joemamma back in 2009:

"Off and OK means that the Power Module's inverter is OFF, but other components such as the charger and PFC could be on (if they are, the fans will spin). If the Power Module fans are not spinning, which I assume they are not, this is not a problem with the Power Modules. The reason the Power Modules' fans are not spinning is due to your exceptionally dead SYBATT battery modules. I can only assume that those battery modules were left engaged in the frame for a long period of time while the UPS was in maintenance bypass, and otherwise not operating normally. Battery modules must always be disengaged from the frame any time the UPS is not operating normally, otherwise they will discharge beyond recoverable levels. The UPS does not recognize any batteries installed due to the very low (24vDC) battery voltage. Batteries must be over 90v to be recognized, as that is the point where the Power Modules are able to start from an OFF state. Those SYBATT modules are destroyed and will need to be recycled in accordance with local regulations. You will need at least one good SYBATT module installed in the Power Array to get the UPS online. If you were to switch out of Maintenance Bypass at this time, the UPS will remain in bypass as the Power Modules cannot start without adequate battery bus voltage, and therefore there is no conditioned power to supply to the load. Only utility is available thus the UPS stays in bypass until power modules can start upon the insertion of one good battery module (assuming all dead batteries have already been removed). " So...we found out what our issue was, but even removing all damaged battery modules, replacing with 3 new batteries, the unit will still not successfully come out of Bypass mode, and the modules remain "OFF and OK in Bypass mode...

[BillP]

Raw Module data

[BillP]

Thanks, Rob.  Really nothing has changed since we had the batteries discharged.  No Devices have been added since the Symmetra unit was put into service in 2008; everything is plugged in to the proper port(s).  Unfortunately we do not have access to another system to interchange parts..

Thanks

[BillP]

Thanks, Rob.  Checked and double checked everything: everything seems to be plugged into the proper port(s), and the MIM and RIM modules  report back ON and RUNNING.  The original problem started in September, when one of 4 power modules reported as not functioning.  Didn't realize at the time that the Symmetra was not charging the batteries, so it entered bypass mode for an extended period, which discharged the batteries beyond recharge.  Remained in bypass until we got the three new batteries, installed, and have not been able to take it out of bypass since without bringing down all devices connected to it...Guess we will have to call Tech support: you havea good phone number we can use?

Thanks Again.

[BillP · Accepted solution]

Greetings..

We have an RM6000 in the server room.  Due to it being in bypass mode for a prolonged period of time, 3 of the batteries were trashed.  We replaced three batteries, so the unit now has 4 power modules, and 3 batteries now charged to 99%.  Problem is, when we attempt to take the unit out of bypass mode, the unit does not supply power to servers attached to it, so we manually put it back into bypass. The unit is now showing 4 power modules OFF and OK, and Battery status: 1,2 and 3: OK.  4,5 and 6 occupied by power modules.  It cannot be taken out of bypass mode via the console: it does not recognize the command. The MIM and RIM show ON and OK. Battery Frame shows Battery 1: not installed; Battery 2: OK; Battery 3 and 4: Not installed.  Warranty Alarm Present: In Bypass: For maintenance. Shows a run time of 1 hour 33 minutes.

Any help on what needs to be done next would be much appreciated.

Thanks