SURTD3000XLI Suddenly showing Battery Needs Replacing

[Anonymous user]

Hi All

Lightly used (UPS Load is 18.6%) SURTD3000XLI been running for a year, fortnightly self tests OK etc

Today, it went down, (no alert about power loss either) taking the servers with it.

Why would 216 VDC be showing with 74 mins runtime then 216 VDC with 0 mins?

Self Test passed less than 2 weeks ago now cant run for 10 seconds?

Having failed at 0153 it looped around (Battery Disconnected > Sufficient Runtime Available > Battery Reconnected > Insufficient Runtime Available > Battery Disconnected) all day and eventually got to Sufficient Run Time available at 1846 but Self test at 2112 fails?

Seems more than just the battery failing overnight going on???

Any thoughts??

Regards

Rod

Data Logs for the day show:

Battery Voltage at 216 VDC available all day (20 min intervals)

UPS Load at 18-19%

Event Logs as follows (Reverse Order):

11/03/13  21:12:45  Battery Needs Replacing    < Device Status is Replace Battery with 0mins at 216.9VDC ??

11/03/13  21:12:45  Self Test Failed 

11/03/13  21:12:42  Self Test Initiated                 < I logged on to check and showed Device Status as Online with 74mins at 216.9VDC

11/03/13  18:46:07  Sufficient Runtime Available 

11/03/13  18:46:07  Battery Reconnected

(Like this from 17:38:53 to 18:46:07 )

11/03/13  17:38:53  Insufficient Runtime Available 

11/03/13  17:38:53  Battery Disconnected 

11/03/13  17:38:53  Communication Established 

11/03/13  17:38:53  Monitoring Started                 < I Turned off power to the UPS and it stopped dead taking servers down again.

11/03/13  16:47:46  Power Restored 

11/03/13  16:47:46  Bypass Ended 

11/03/13  16:47:28  In User Initiated Bypass          < Switched to Bypass via Powerchute

(Like this from 05:58:09 to 16:47:28 )

11/03/13  05:58:09  Battery Disconnected 

11/03/13  05:57:30  Sufficient Runtime Available 

11/03/13  05:57:30  Battery Reconnected 

11/03/13  05:57:09  Insufficient Runtime Available 

11/03/13  05:57:09  Battery Disconnected 

11/03/13  01:53:17  Communication Established 

11/03/13  01:53:17  Monitoring Started                

27/02/13  00:11:10  Self Test Passed 

27/02/13  00:11:01  Self Test Initiated 

22/02/13  20:49:01  Power Restored 

22/02/13  20:49:00  Power Failed 

17/02/13  15:08:00  Power Restored 

17/02/13  14:16:08  Power Failed 

13/02/13  00:12:03  Self Test Passed 

13/02/13  00:11:54  Self Test Initiated 

30/01/13  00:12:56  Self Test Passed 

30/01/13  00:12:47  Self Test Initiated 

16/01/13  00:13:51  Self Test Passed 

16/01/13  00:13:42  Self Test Initiated

[BillP]

If you can get to the UPS serial number via remoting in using PowerChute, you should be able to see the firmware version. if you're using the PCBE console (purple GUI window), make sure that the "show advanced items" or similar is selected on the bottom left hand corner. the UPS firmware version should be listed near the serial number you found. It should be something like 5.1.

Anyway, once we verify that, I can look into it more to see if we think it is a UPS problem or batteries and try to provide an explanation for the behavior you saw.

[Anonymous user]

Hi All

Lightly used (UPS Load is 18.6%) SURTD3000XLI been running for a year, fortnightly self tests OK etc

Today, it went down, (no alert about power loss either) taking the servers with it.

Why would 216 VDC be showing with 74 mins runtime then 216 VDC with 0 mins?

Self Test passed less than 2 weeks ago now cant run for 10 seconds?

Having failed at 0153 it looped around (Battery Disconnected > Sufficient Runtime Available > Battery Reconnected > Insufficient Runtime Available > Battery Disconnected) all day and eventually got to Sufficient Run Time available at 1846 but Self test at 2112 fails?

Seems more than just the battery failing overnight going on???

Any thoughts??

Regards

Rod

Data Logs for the day show:

Battery Voltage at 216 VDC available all day (20 min intervals)

UPS Load at 18-19%

Event Logs as follows (Reverse Order):

11/03/13  21:12:45  Battery Needs Replacing    < Device Status is Replace Battery with 0mins at 216.9VDC ??

11/03/13  21:12:45  Self Test Failed 

11/03/13  21:12:42  Self Test Initiated                 < I logged on to check and showed Device Status as Online with 74mins at 216.9VDC

11/03/13  18:46:07  Sufficient Runtime Available 

11/03/13  18:46:07  Battery Reconnected

(Like this from 17:38:53 to 18:46:07 )

11/03/13  17:38:53  Insufficient Runtime Available 

11/03/13  17:38:53  Battery Disconnected 

11/03/13  17:38:53  Communication Established 

11/03/13  17:38:53  Monitoring Started                 < I Turned off power to the UPS and it stopped dead taking servers down again.

11/03/13  16:47:46  Power Restored 

11/03/13  16:47:46  Bypass Ended 

11/03/13  16:47:28  In User Initiated Bypass          < Switched to Bypass via Powerchute

(Like this from 05:58:09 to 16:47:28 )

11/03/13  05:58:09  Battery Disconnected 

11/03/13  05:57:30  Sufficient Runtime Available 

11/03/13  05:57:30  Battery Reconnected 

11/03/13  05:57:09  Insufficient Runtime Available 

11/03/13  05:57:09  Battery Disconnected 

11/03/13  01:53:17  Communication Established 

11/03/13  01:53:17  Monitoring Started                

27/02/13  00:11:10  Self Test Passed 

27/02/13  00:11:01  Self Test Initiated 

22/02/13  20:49:01  Power Restored 

22/02/13  20:49:00  Power Failed 

17/02/13  15:08:00  Power Restored 

17/02/13  14:16:08  Power Failed 

13/02/13  00:12:03  Self Test Passed 

13/02/13  00:11:54  Self Test Initiated 

30/01/13  00:12:56  Self Test Passed 

30/01/13  00:12:47  Self Test Initiated 

16/01/13  00:13:51  Self Test Passed 

16/01/13  00:13:42  Self Test Initiated

[BillP]

Can you provide the first few digits of the UPS serial number? i.e. IS1012XXXXXX? Also, what is the UPS firmware version?

[BillP]

Also, what is the environment this UPS is installed in? This could be some type of pre-mature battery failure due to a less than ideal environment.

[Anonymous user]

UPS Model                             Smart-UPS RT 3000 XL
Firmware Revision                  5.1
Manufacture Date                   4 December 2010
Serial Number                         IS1049010478 
Battery Replacement Date     12 2010

(The typical expected battery lifetime is three to five years )

The UPS & Servers (2 * HP ML150) live in an open cupboard of a non-airconditioned room.

Data Log Results from last 6 months (our summer)

(Logged at 20minute intervals, 6477 data points)

	Input Line Voltage	UPS Output Voltage	Output Frequency	Battery Voltage	UPS Load	Internal UPS Temp
	(VAC)	(VAC)	(Hz)	(VDC)	(%)	(°C)
AVERAGE	247.2	230.1	50.0	216.3	18.6	31.1
MAX	256.6	230.3	50.2	235.1	20.6	37.0
MIN	0.0	229.9	49.8	186.1	11.2	26.2

I'm guessing that an internal temp of 37°C may have been a little excessive? But apparently still within operating temp of 0-40C.

System Status before Self Test (Manual 2 days ago)                                                 Current System Status

 

[BillP]

Hi Rod,

While the acceptable range for the hardware of the UPS itself is 0°C to 40°C, the batteries should be between 20°C to 25°C to deliver their normal expected service life. The optimum operating temperature for a lead-acid battery is 25 Deg C (77 Deg F). As a guideline, every 8°C (15°F) rise in temperature will cut the battery life in half. A battery which would last for 5 years at 25°C (77°F), will only be good for 2.5 years if operated at 33 °C (95°F). Keep in mind that the battery temperature inside your UPS will always be warmer than the ambient temperature where the UPS is installed.

I have found that in overly warm environments you are more likely to see a sudden failure of the batteries. In your situation it appears that there was a sudden (and total) loss of battery capacity in the time between your biweekly self tests. I would recommend taking your batteries out of production as soon as possible and replacing them with new batteries. Going forward you can either reduce the temperature in the room or plan on proactively replacing batteries on an accelerated schedule.

Hope this helps!

Liam

[Anonymous user · Accepted solution]

Thanks Liam.

Weird... this week its reporting self tests OK.

Not 100% sure whats going on but APC are sending a new one.

Thanks for the replies.

Regards

Rod