SUA3000RMI2U - lost battery capacity fast and ups reboot

Anonymous user · ‎2021-06-29

Hello together

The UPS with the model number "AP9631" has been running for 558 days.

But in the last weeks there happened different losses.

Here more information about the UPS and the loads:

At 14.03.2014 the System Uptime is on the state 0 for a moment.

So i think the UPS has rebooted, this was in at 2 a clock in the morning.

In the middle of the 12.03.2014 the upsAdvInputLineFailCause gone up to 7 and at the 14.03.2014 at 2 a clock down to 1 up to now.

Also the NMC has been rebooted.

At this date 12.03.2014 the overcurrent device was turned off for max. 5 min, in this moment the UPS should have bridge the power blackout.

But the battery capacity fall down from 100% to 0% inside minutes.

A colleague had told me that the UPS is fall out. But in the monitoring was the Uptime normaly, but the capacity fall down to 0% and the upsAdvInputLineFailCause fall down (06.03.2014 - 12.03.2014) from 9 to 1.

At 06.03.2014 the System Uptime is on the state 0 for a moment.

So i think the UPS has rebooted, this was in at 2 a clock in the morning.

Also the NMC has been rebooted.

At 03.12.2013 the System Uptime is on the state 0 for a moment.

So i think the UPS has rebooted, this was in at 15:10 a clock in the afternoon.

The upsAdvInputLineFailCause also fall down to 1 for 5 days.

The NMC has been rebooted too.

But the battery capacitiy didn't fall down.

I think the battery is out of date, but that would be a really short live.

That somebady has an idea?

Sorry for the bad english

In advance thanks for ideas and best regards

Fabio Martinelli

Anonymous user · ‎2021-06-29

Yes that's the date when the UPS turned off the equipment, because of the power blackout.

Here the Monitoring rrd graphic:

We have found out that one of the edge connector, with the telephone system on it, was not inserted properly (intermittent contact).

Now it's right inserted. Did this have an impact?

We haven't configured something. Whats the right configuration?

So when I look on the load, the UPS must bridge the power for more then 1 hour and 5 minutes, but there are only 8 minutes.

Ok, if I have configured the Outlet Group configuration right I'll make a simulation. I don't have done this, because of the self test fail.

See Answer In Context

BillP · ‎2021-06-29

Hi Fabio,

The Network Management Card (AP9631) can reboot and when it does, it would not affect the UPS output status. The UPS itself is the SUA3000RMI2U.

This will be a lot easier for me to assist with if you can provide the full AP9631 Network Management Card logs for me to review (event.txt, data.txt, config.ini, debug.txt, dump.txt) and I can let you know what I find.

FYI, here is what a network interface restart/coldstart means and why it may happen -> Why does my Network Management Card report warmstarts and coldstarts? | FAQs | Schneider Electric US. The messages about the outlet groups turning on is misleading and we have addressed this in a newer firmware version than you have. When the Network Management Card interface itself reboots, it is just discovering that the outlet groups are on. It does not mean they have turned off at any point.

Thus, the only issue we may have here is identifying why the Network Management Card interface is rebooting unexpectedly and the log files can confirm to me why that is and then we'll double check to make sure the UPS itself hasn't rebooted.

Anonymous user · ‎2021-06-29

Hi Angela

Thanks for the fast and halpful response.

I have found a new MIB-File (powernet410.mib), is this a firmware too or how can i insert this into my UPS?

Why the NMC not reboot periodical?

It seems that the Gateway is not reachable, i think this is turned off.

Are the outlets groups a group of interfaces to plugin devices?

In the attach files are every information.

BillP · ‎2021-06-29

The MIB is for SNMP use only. If you utilize SNMP which it sounds like you might, you can load this into your SNMP monitoring tool. It would not affect this issue though.

Looking at your log files, I don't think this UPS is truly a SUA3000RMI2U because that UPS does not contain outlet groups. Where did you see this model number?

Can you verify what is under UPS->About? Or verify the model on the sticker on the rear of the UPS? I need to know the model and firmware versions listed and I think I can figure out what the UPS model is.

I am not seeing the NMC reboot because of an issue contacting the gateway but rather something likely related to the UPS model it is in which we did make enhancements for in future Network Management Card versions.

Either way, it does not seem to be happening often and shouldn't be of major concern but I need to know the real UPS model to recommend how to proceed. But, your UPS output has not rebooted based on what's in the log and this is more of a cosmetic issue than anything.

I am not sure what you mean by "Are the outlets groups a group of interfaces to plugin devices?" - the outlet groups are switchable groups of outlets on the rear of the UPS that you can control in groups. When the NMC reboots, it realizes the outlet groups are on and reports that they have now turned on which caused confusion for users like yourself. Thus, we addressed this in v5.1.7 and higher so that when the NMC reboots, you don't see these messages anymore. You should only see them when you turn the outlet groups on or off, as you'd expect.

Anonymous user · ‎2021-06-29

I saw this on the also web shop where i bought this UPS

Here the information's about the UPS:

Okey thanks for the detour in to the outlets groups management. Now i know more about them

We found out that when we made a force self-test, the UPS couldn't bypass the power supply.

The RunTimeRemaining on UPS, in the Web GUI and the Monitoring isn't true or the UPS have a other problem.

The scan on the "upsAdvInputLineFailCause" was very long on the state "self-test", then for 10 days on "notTransfer(1)", then shortly on the state 7 and up to now on "noTransfer(1)".

This state ist normaly or not? When i offset a snmpwalk command with the new MIB then i can see the same value.

I think either we must buy a new battery or even a new UPS.

Do you have any idea?

BillP · ‎2021-06-29

Ok, SMT3000RMI2U makes more sense

I agree the runtime reported looks a little low but the UPS is also a few years old now. The runtime remaining reported while the UPS is an estimate and constantly updated when the UPS goes on battery during a power event. (A self test on this UPS is not super long and may not be enough for the UPS to update accurately).

If you're concerned about the runtime, I'd probably try a runtime calibration (either via the NMC or manually) and then see how long it actually lasts on battery. This should both update it to reflect a more accurate value or validate that 11minutes is accurate and perhaps then it is time for a battery replacement. So if anything, it may be time for a new battery in the near future but it is not likely there is an issue with the UPS. If you're interested in doing a calibration, then you can issue one from UPS->Diagnostics in the AP9631 or via the front panel LCD on the UPS. If this doesn't work, we can pursue a manual runtime calibration that I can explain to you too.

What you mentioned regarding upsAdvInputLineFailCause, this seems normal - had you rebooted the UPS at all to erase the "last transfer reason" which is what upsAdvInputLineFailCause reports? On March 6th, the AP9631 was "coldstarted" which means power cycled. This could've just been the AP9631 or the entire UPS itself. If it was the entire UPS, then that would make sense that the last transfer reason was erased. On March 12th, there also seems to be a similar power event but I can't tell if the UPS lost power completely for more than 10 minutes after it turned off which could also erase it.

Since you were asking about firmware updates, I don't know if you're interested but there is a firmware update for your UPS (you have UPS 06.5 firmware, UPS 09.1 is latest) which you could apply. You can push this file from the NMC to the UPS or you can use our UPS firmware upgrade wizard via serial or USB (if this model supports USB).

Information on updating the UPS:

The Network Management Card is also a little out of date. I'd suggest at least updating it to v5.1.7 which I can provide, with instructions, to you since it is no longer on the apc.com website. v6.X.X firmware is a major change and we are actually in the process of resolving a few critical issues with it that I would not want to subject you to. (More information here -> Network Management Card 2 Firmware v6.X.X FAQ/Upgrade Issues | FAQs | Schneider Electric US). Also, 5.1.7 I believe resolves the issue with the outlet group messages after the NMC reboots.

Anonymous user · ‎2021-06-29

Oh, yea the first three letters

Goes the bypass time of the current or the maximum capacity?

How long does a self test go?

Does the runtime callibration need a UPS restart, there are important devices on it?

We find out that the "upsBasicBatteryTimeOnBattery" shows no value and the "upsBasicBatteryStatus" is on the state "batteryNormal(2)" not on "batteryLow(3)".

No i hadn't rebooted the UPS manually, but she is inadvertend shut downed.

Right now upsAdvInputLineFailCause looks normal, but for one year or more the state was permanent on "self-test".

How we can got back the "last transfer reason", isn't he in a logfile or so?

When the power out of the box is flowing and the UPS or any component is down, why does the devices lost our power? Or is my consideration wrong.

Thanks for the information's.

So the firmware upgrade for the UPS and the NMC are not important for this case not?

BillP · ‎2021-06-29

Goes the bypass time of the current or the maximum capacity?

How long does a self test go?

Does the runtime callibration need a UPS restart, there are important devices on it?

I am not sure what you mean "Goes the bypass time of the current or the maximum capacity?"

The self test on an SMT is about 5-10 seconds long. If you look in the event.txt you provided me, you'll see the self test events in there and their timing.

Runtime calibration does not need a UPS restart. It puts the UPS on battery and discharges it down to between 10-20% (depending on model), then let's it charge back up. Only in a manual calibration do we suggest letting the UPS discharge until it turns off.

We find out that the "upsBasicBatteryTimeOnBattery" shows no value and the "upsBasicBatteryStatus" is on the state "batteryNormal(2)" not on "batteryLow(3)".

During a self test? These OIDs seem to refer to actual power events (a self test is a diagnostic) and plus, the UPS self test is very short - are you polling this definitely during the quick self test?

No i hadn't rebooted the UPS manually, but she is inadvertend shut downed.

Oh, ok. Because of a power outage it seemed because of your log.

Right now upsAdvInputLineFailCause looks normal, but for one year or more the state was permanent on "self-test". How we can got back the "last transfer reason", isn't he in a logfile or so?

This is stored on the UPS itself. The Network Management Card queries this value and can report it to you. I would not be concerned about this and let the UPS operate normally and perform it's automatic self test every week or two weeks and it should change back. What do you see under UPS->Overview or Status - the OID relates to "Last transfer reason" you see in the web too.

When the power out of the box is flowing and the UPS or any component is down, why does the devices lost our power? Or is my consideration wrong.

I am not sure what you mean here. From the log files you sent me, I only saw a legitimate power event where the UPS turned off. "The outlet groups have now turned on" messages do not indicate the outlet groups were rebooted if you see it after a "Network interface restart" message and no other "UPS on battery power" events. So, we may have a misunderstanding here. I do not see any problems with this UPS beyond the fact that your runtime remaining value is not super accurate which a runtime calibration is the first step to address.

So the firmware upgrade for the UPS and the NMC are not important for this case not?

It likely won't address the runtime remaining value being low but an NMC update will remove the messages about outlet groups turning on after only the NMC reboots for you (which could be less confusing). I don't think an update for the UPS is critical but I wasn't sure if you like to be up to date on everything to take advantage of the latest bug fixes and features.

Anonymous user · ‎2021-06-29

With the first question, i would like to find out if the "runtime remaining" is calculated with the current capacity?

It's only a control question.

I have done a runtime callibration, but the "runtime remaining" hasn't helped.

Our monitoring is measure the value every 5 minutes, i think the self test don,'t have an impact, but the first manual test.

The upsBasicBatteryStatus is permanently on "batteryNormal(2)".

Under overview the "Last Battery Transfer" is on the state "None".

When the UPS goes down, why does the devices go down too? Is it because the battery is to low. That would be strange.

Now i think a battery replacement is the next step not?

How can i make sure that everything with the ups itself is correct.

BillP · ‎2021-06-29

The runtime reported when the UPS is online is an estimate using the load percentage and capacity among likely the battery age/health.

Maybe I misunderstood previously - I did not think that the UPS turned off - I thought you were assuming it did based on the "outlet group turned on" messages in the log. Did the attached load to the UPS reboot or turn off which you have verified?

I did not see anything in your log that indicated the UPS had turned off it's attached load - just the "outlet group turned on messages." Did I miss something?

Anonymous user · ‎2021-06-29

Ok, thanks.

Oh right not the whole UPS i mean the outlet groups. The devices (attechaed load) have rebooted and the UPS capacity regenerated itself.

I have tried to install the new Firmware for the UPS and the NMC, but i havn't found the right firmware. Do you know where they are?

No i think you didn't miss something.

I think the best idea is to buy a new battery isnt't it?

BillP · ‎2021-06-29

I only saw on one instance on March 12th where it seems like there was a low battery condition due running on battery and based on the configuration of the UPS, the outlet groups starting turning off. This all looked normal to me so I wanted to clarify if this is the date that the UPS turned off the equipment? Perhaps the problem is a configuration that you had that you were unaware of that was configured?

12.03.2014 11:05:39 UPS: A discharged battery condition no longer exists. 0x0108

12.03.2014 10:54:02 System: Web user 'aspectra' logged in from 10.122.60.106. 0x0015

12.03.2014 10:53:51 System: Detected an unauthorized user attempting to access the Web interface from 10.122.60.113. 0x0006

12.03.2014 10:52:57 System: Detected an unauthorized user attempting to access the Web interface from 10.122.60.113. 0x0006

12.03.2014 10:45:13 UPS: The power for switched outlet group 1, Outlet Group 1, is now turned on. 0x0134

12.03.2014 10:45:12 UPS: The output power is now turned on. 0x0113

12.03.2014 10:45:12 UPS: The power for the main outlet group, UPS Outlets, is now turned on. 0x0134

12.03.2014 10:43:14 UPS: The power for switched outlet group 1, Outlet Group 1, is now turned off. 0x0135

12.03.2014 10:43:13 UPS: A low battery condition no longer exists. 0x0110

12.03.2014 10:43:13 UPS: The battery power is too low to support the load; if power fails, the UPS will be shut down immediately. 0x0107

12.03.2014 10:43:13 UPS: The output power is turned off. 0x0114

12.03.2014 10:43:13 UPS: No longer on battery power. 0x010A

12.03.2014 10:43:13 UPS: Switched outlet group 1, Outlet Group 1, has been commanded to shutdown, sequenced. 0x0174

12.03.2014 10:43:13 UPS: Main outlet group, UPS Outlets, has been commanded to shutdown with on delay. 0x0174

12.03.2014 10:43:13 UPS: The power for the main outlet group, UPS Outlets, is now turned off. 0x0135

12.03.2014 10:42:45 UPS: The battery power is too low to continue to support the load; the UPS will shut down if input power does not return to normal soon. 0x010F

12.03.2014 10:33:59 UPS: On battery power in response to distorted input. 0x0109

We need to review your outlet group configuration. You'll find this under the UPS tab in the web interface under Outlet Groups. (It is also under [SwitchableOutletGroups] section in the config.ini but it is more difficult to read there). There is specific configuration there that basically told your outlet groups to turn off because the UPS hit a low battery condition which is when things start turning off. Unfortunately, you're data log does not go back to this time frame so I am going off of the event log.

If I look at the data log I do have, the load is very low. In reviewing Runtime Chart for Smart-UPS for your UPS model (SMT3000RMI2U), I would've expected you to receive more than 10 minutes runtime though. When you did the runtime calibration, how long did the UPS run on battery? It might be a good idea to simulate a power failure too to see if it runs closer to what the runtime chart says. I do believe that it is feasible if we do a calibration (manually or like you did with the software) and the 11 minutes is actually what it is doing, then yes, the battery may have a decreased capacity and should be replaced (either under warranty or not).

Anonymous user · ‎2021-06-29