APC Smart Ups SURT10000XLI Battery Indication Error

[Anonymous user]

Hi Everyone,

I would be very grateful if some Tech Wiz can assist me in my issue. Basically i have an smart UPS SURT 10000XLI with three external battery packs, that have been running for almost 4 years non stop with no issues.Recently we have been having issues with power drops and realized that all server just drop due to power interruption. Only then have we replaced all 128 batteries both internal and external and all hell broke lose

Simply we have realized that after replacing the batteries we had a run time of 4 minutes that went up to 6 minutes after 100% battery Recharge! Vs. the almost 4.5 -5.5 HRS previously. Then we noticed some errors with Battery Discharge error logs and power outages that made the UPS drop dead like a rock.

I have attached my erro logs and configuration and hope that some one has a solution to this issue. My well educated guess is a firmware update via serial cable but i am not a techie to go down the system command line route

Model: Smart-UPS RT 10000 XL

Position:              TOWER

Serial Number:  NS1010002855

Firmware Revision:         476.18.W

Manufacture Date:         03/04/10

Last Battery Transfer:     Due to software command or UPS's test control

Internal Temperature:   27.4°C

Runtime Remaining:       6 minutes

Power

Input Voltage:   220.3 VAC                           

Bypass Input Voltage:                                   

Output Voltage:               230.4 VAC

Current:                                              

Input Frequency:             49.39 Hz

Load

Load Power:       13.0 %Watts

Apparent Load Power:  16.0 %VA

Load Current:    7.29 Amps

Battery

Capacity:              23.0 %

Battery Voltage:               217.0 VDC

External Batteries:           3

1. 04.09.2014           12:02:09               System: Web user 'apc' logged in from 192.168.0.254 [cxxxxxxxxxxx.local].
2. 04.09.2014           11:59:59               UPS: The battery power is too low to support the load; if power fails, the UPS will be shut down immediately.
3. 04.09.2014           11:59:58               UPS: A discharged battery condition no longer exists.
4. 04.09.2014           11:59:27               UPS: The battery power is too low to support the load; if power fails, the UPS will be shut down immediately.
5. 04.09.2014           11:59:25               UPS: A discharged battery condition no longer exists.
6. 04.09.2014           11:58:37               UPS: The battery power is too low to support the load; if power fails, the UPS will be shut down immediately.
7. 04.09.2014           11:58:36               UPS: A discharged battery condition no longer exists.
8. 04.09.2014           11:58:16               UPS: The battery power is too low to support the load; if power fails, the UPS will be shut down immediately.
9. 04.09.2014           11:58:15               UPS: A discharged battery condition no longer exists.
10. 04.09.2014           11:58:13               UPS: The battery power is too low to support the load; if power fails, the UPS will be shut down immediately.
11. 04.09.2014           11:58:11               UPS: A discharged battery condition no longer exists.
12. 04.09.2014           11:54:29               UPS: The battery power is too low to support the load; if power fails, the UPS will be shut down immediately.
13. 04.09.2014           11:54:28               UPS: A discharged battery condition no longer exists.
14. 04.09.2014           11:54:03               UPS: The battery power is too low to support the load; if power fails, the UPS will be shut down immediately.
15. 04.09.2014           11:54:01               UPS: A discharged battery condition no longer exists.
16. 04.09.2014           11:53:54               UPS: The battery power is too low to support the load; if power fails, the UPS will be shut down immediately.
17. 04.09.2014           11:53:51               UPS: A discharged battery condition no longer exists.
18. 04.09.2014           11:53:48               UPS: The battery power is too low to support the load; if power fails, the UPS will be shut down immediately.

[BillP · Accepted solution]

Can you give me the event.txt, data.txt, and config.ini from the Network Management Card? Most of what you have is event.txt? Instructions here -> How can I download Event, Data, Configuration, and Debug files from my Network Management Card? | FA...

I likely think these messages are appearing because of your lower runtime and also based on a setting you probably have set a particular way. That setting I refer to is the low battery duration in your Network Management Card. It is probably set to within 2 minutes of the available runtime of 6 minutes (or is still set to the default of 2 and the runtime is dropping down to 4 minutes) and thus, the messages show in the log to try and tell you your available runtime is quite low and when your UPS goes to battery, it will shutdown practically immediately. Essentially, you likely also have your front panel battery bar graph LEDs flashing too as the entire scenario is highlighted here -> Why are the battery charge LEDs flashing on a Smart-UPS? | FAQs | US

I think that problem is occurring because your UPS needs to be calibrated either manually or a command issued to get it to report the correct amount of 4-5 hours. That is also explained a little bit in the second knowledge base I just referenced. It also has the steps for a manual calibration there.

[Anonymous user]

HI Angela,

Well i didn't think i will be re-posting sometime soon but here i am again. If you recall i had excellent run time available right after the battery constant change of almost 6 hours with 17% load. Yesterday we had a main cable problem and work took almost two hours during which the UPC shut down all servers after 1:30 minutes and then the run time reported now is back to 1 hour and 2 minutes! Can you assist as to why this may have occurred. Do i have to reinstate the battery constant again maybe.

thanks in advance

[Anonymous user]

Dear Angela,

Thank you so much for getting back to me. Ok so i have attached the three files you requested and also a brief of what our system admin has attempted also today in an effort to get the UPS functioning normally.

First of all, we have removed all loads and accompanying devices from the UPS and connected both the serial and network to a single machine after disabling the firewall, then we tried the following:

1.       As stated in this article: http://www.rm.com/support/technicalarticle.asp?cref=tec817072 we have connected successfully using serial cable (2400 Baud, 8 data bits, 1 Stop bit, no parity, no FIFO) to the UPS through Hyperterminal, however, the UPS didn’t respond to any of the commands specified in the article!

2. Tried to connect using 3rd party tools like apcupsd.com with no success!

3.       Managed to connect through the FTP and generated the event.txt, config.txt, and data.txt attached to the message.

[BillP]

OK. You can follow the instructions in the article at your own risk but it likely won't work until you remove the Network Management Card from the UPS (which is where you retrieved the log files from ultimately). You want to be talking to the UPS directly and the Network Management Card installed will take precedence.

I see in the configuration file you gave:

[SmartUPS/Matrix]

BypassUpperVoltage=253

BypassLowerVoltage=161

OutputFrequencyRange=Auto

ReturnBatteryCapacity=50

[SmartUPS/Matrix]

BypassUpperVoltage=253

BypassLowerVoltage=161

OutputFrequencyRange=Auto

ReturnBatteryCapacity=50

LowBatteryDuration=10

ShutdownDelay=0

ReturnDelay=0

SleepTime=010

UPSName=UPS_IDEN

UPSPosition=TOWER

BatteryDate=05/04/13

AudibleAlarm=Power Fail

ExternalBatteries=3

SimpleSignalShutdown=disabled

SelfTestSchedule=Every 14 days

ShutdownDelay=0

ReturnDelay=0

SleepTime=010

UPSName=UPS_IDEN

UPSPosition=TOWER

BatteryDate=05/04/13

AudibleAlarm=Power Fail

ExternalBatteries=3

SimpleSignalShutdown=disabled

SelfTestSchedule=Every 14 days

So with 6 minutes of runtime but a low battery duration of 10 minutes, it is basically saying you have low runtime because it thinks you need 10 minutes to do a clean shutdown.

I'll give the constant value you need since you found the instructions but just remove the AP9619 from the SmartSlot, then connect with the APC serial cable. And make sure it is the APC article mentioned in the instructions because that could be your problem too. The value you need for your UPS is 7E.

[BillP]

Hi again.

What I can understand that u want us to remove the network card completely and then follow the instructions in this link: APC UPS replacement batteries may appear to have a lower capacity and not last as long and communicate directly via Hyperterminal, is that right?

Yes

In case we don't wish to remove the network card, isn't there any way to set this value?

No. It needs to be done directly on the UPS and you cannot talk to the UPS with the Network Management Card installed. It is safe to uninstall/reinstall this card while the UPS is running.

What if we want to increase the low battery duration of 10 minutes?

I only advise changing this if the scenario warrants it. Why do you want to change it?

What is attached load/equipment to the UPS? PowerChute Network Shutdown clients or other equipment? Do you know why it was set to 10? 2 is the default. And the UPS only supports certain values. It likely only supports something close to 20 minutes. Each UPS supports different values depending on the type and model.

What if the UPS currently have the constant 7E?

I bet it won't

[Anonymous user]

Dear Angela,

What I can understand that u want us to remove the network card completely and then follow the instructions in this link: APC UPS replacement batteries may appear to have a lower capacity and not last as long and communicate directly via Hyperterminal, is that right?

In case we don't wish to remove the network card, isn't there any way to set this value?

What if we want to increase the low battery duration of 10 minutes?

What if the UPS currently have the constant 7E?

Sorry to bombard you with so many questions

[Anonymous user]

Dear Angela, we have removed the network card and connected successfully to the UPS through COM, however, it still doesn't respond to terminal messages!

[Anonymous user]

Dear Angela. we finally managed to connect to the UPS through the original serial cable. However, we couldn't find any way to change the battery constant. We have navigated all the menus and nothing in battery constant terms.

[BillP]

You're likely pressing Enter/Return to get into the Smart-UPS RT Control Menu which is not what you want to do. Exit out of that menu and without pressing enter, press Shift Y and then follow the instructions at the link you found.

[Anonymous user]

Thank you so much for pointing us in the right direction. Ok, so i followed your steps exactly and it gave me a starting value of 18 ! ( what does this mean) Then i followed the steps to increase the value until i got 7E. Is this it.? should i run a calibration test now.

[BillP]

Hello again. Glad you got it working. These values are hex so I don't quite know what they mean - only the UPS knows. And once you set it to 7E, you're done. Press Shift R to exit. This process replaces the need to do a calibration (and should only be done when batteries are done) if at all.

Now, go ahead and remove the serial cable and re-install your Network Management Card and check the reported runtime

[Anonymous user]

Finally ! This was like a nightmare.....Thank you ever so much for your assistance we are now reporting over 7 HRS of uptime.

[BillP]

You're welcome . This value is a really good estimate based on the load and a few other characteristics. When the UPS goes on battery, either during a self test or during power events, it will continuously update over time to provide you the most accurate number it can.

[BillP]

Hi again,

In the beginning, you definitely had an issue with the constants. I know you replaced 128 batteries - did you replace these with third party batteries with a different capacity than the original APC provided ones? It seems like you replaced the individual 12V batteries inside the RBC44 modules that we sell (which each RBC44 is two strings, each string with (8) 12V batteries). I believe the UPS uses (2) RBC44 and each battery pack does too. Adjusting the constants is going to base everything on a standard set up assuming the capacity, amp hours, etc included in the standard UPS internal batteries and external battery packs.

If you do a manual calibration with all communication cables disconnected and Network Management Card uninstalled, that might be your best chance at getting an accurate runtime. Though, this discharge from the power outage is effectively a calibration if you let it shutdown after running down completely and didn't have any weird involvement via Network Management Card configuration or serial/USB.

We can also look at your Network Management Card event.txt and data.txt files with the config.ini again.

[Khalil_apc]

Dear Angela,

We also have the same issue with the same model of UPS, unfortunately the instruction link mentioned above is expired and can not be access. So could you please help us updating this information. Thanks in advance for helping.

Regards

M Khalil