Symmetra RM 6000 problems: overheating, swelling, and leaking batteries

[BillP]

We had changed old batteries by new SYBT2(10*HRL1223WF2). They were bought here http://www.zdis.de/batterie-fuer-usv/apc/sybt2/sybt2-hq.html?listtype=search&searchparam=sybt2

After 2 years the batteries become unfit for use. 

Before accident UPS had overcharging batteries. And within some hours the batteries become destructive.

Symmerta had overcharging batteries with 147.4 V and temperature was 120F.

IM Status	: On & OK	Master Firmware Rev : 127
		Slave Firmware Rev  : 214
	Manufacture Date : 05/23/01

[Terry_Kennedy_apc]

[Bear in mind I'm just another user and don't represent APC. But I do have a bunch of these units.]

I assume this unit is out of warranty? Otherwise, I'd say contact your local APC support and let them handle it.

How many power modules do you have, and how many batteries? Is there a battery expansion chassis installed?

Tracking this sort of thing down can be a bit difficult. Regarding some of the things that had been posted in the other RM6000 thread:

• All of the battery modules (SYBT2) are in parallel. Even if the UPS completely disconnected from the battery bus, other battery modules can contribute to thermal runaway. For example, if a cell shorts out in one of the component batteries in a SYBT2, all of the other cells in that battery and the other 9 batteries in the SYBT2 will see a higher "charging" voltage as the other SYBT2's attempt to bring the defective SYBT2 up to the same voltage as the rest of the SYBT2's. This can lead to an additional cell failure, causing an even-higher voltage and so on.
• Each SYBT2 has a sense wire at the halfway point (5 batteries). This, along with the battery bus voltage and current sensed by the SYMIM, is used to determine battery health and state-of-charge. A voltage differential where half of the batteries in a SYBT2 report a different voltage than the other half is indicative of a failed battery (and is also why all 10 batteries in each SYBT2 need to be replaced as a set).
• A single SYBT2 saying "charge me" results in the UPS applying a charge to all SYBT2's in the system. If the problem is indeed a single SYBT2 needing to be charged, charging voltage should not harm the other SYBT2's as this does not persist for an indefinite period. However, a SYBT2 that is falsely reporting the need for a charge can cause the UPS to continuously apply a higher-than-desired voltage to all SYBT2's in the system. I think that the fast charge function was changed in later firmware, but your UPS has older firmware - the newest I've seen is 143 / 226 for the master / slave.
• The SYCC/SYXRCC communication cards collect the sense data and report it over an I2C bus to the SYMIM. Any problem on the I2C bus will produce very bizarre log entries in the UPS's event log - things like modules being reported removed / inserted when no-one was near the UPS and so on.

Because of the above things, I recommend replacing all batteries in all SYBT2's at the same time, even if some of them seem fine and don't report any fault status.

Any troubleshooting you can do at this point will likely involve dropping the load attached to the UPS, so I would suggest removing the UPS from the configuration. It is unlikely that you have a hard-wire kit on your UPS, so you should be able to unplug whatever is plugged into the UPS and plug it into the outlet the UPS was plugged into. Of course, this will cause a loss of power to those connected devices, so you should gracefully shut down that equipment before disconnecting the UPS. Then plug the UPS into another outlet for testing.

Reduce the configuration to the simplest possible - a single power module and a single SYBT2. You can leave both the MIM and RIM installed. Disconnect any external battery packs (both battery bus and communication bus) Measure the battery bus voltage, either via the front panel / NMC, or (if you have the equipment and skill to use it) with a meter at the battery bus connector on the rear of the unit (under a cover plate if there are no external batteries). Be aware that the voltage on the connector is likely > 120V and DC, so do not do this unless you're familiar with appropriate safety.

If this configuration does not work ("work" means charge the batteries to an appropriate voltage and then changing to float mode*), you either have a problem with the power module (excessive DC bus voltage) or with the SYBT2 (bad batteries). Swap the power module and test again. If the problem persists, use a new SYBT2 or replace the 10 batteries in the SYBT2. Per what I said above, if something bad happened to one SYBT2 it probably also affected the others. If this configuration still does not work, you either have multiple bad power modules or an intelligence module problem. If it does work, proceed to add the remaining power modules, one at a time, with testing after each. Now add SYBT2's, also one at a time, testing after each. Fill the UPS chassis first. Then reconnect a single expansion frame (without batteries) and test. Now add SYBT2's to that expansion frame, one at a time, with testing after each. Repeat this sequence with any additional expansion frames.

* The battery current is available from the raw status. For an expansion frame, it is something like "1 4 0F WH2149 2149 RT0190 I-0.0 V136.6" where I-0.0 indicates no charging (or discharging) current and V136.6 shows the battery bus voltage.

I discuss various Symmetra RM 6K topics on my blog:

New Year, new UPS batteries…

[Another] New Year, new UPS batteries…

APC Symmetra RM Front Panel Battery Replacement

APC Symmetra RM Fan Replacement

Needless to say, none of this supported / recommended by APC. User service procedures on this unit are limited to module-level replacement.

[BillP · Accepted solution]

Thanks for answer, Terry.

The symmetra is old and is not on warranty.

There were 3 sybt2 modules when this accident was happened. All of them were swelled. Accident  was 3 months ago.

I changed it all, put new 2 sybt2. Today(3 months) all is good. But of course it will happen once again.

Symmetra has 3 power modules and has no expansion chassis with batteries.

Now I have measured voltage of those bad batteries, it is 123 V on each module. I stiil do not utilized them.

We plan to change thisSymmetra by HP R5500 xr and start testingsymmetra.

But before I wil try to study about intelligent module symim3.