Battery needs replacing - causing coldstarts on all equipment.

BH_apc · ‎2021-06-28

This was originally posted on APC forums on 7/23/2009

Have a SmartUPS 3000 RM with 4 servers attached that is in need of battery replacement. The excerpt from the log file below indicates as much.

Problem is that the UPS is killing power to all the servers on an almost daily basis (usually just before working hours.. you can see all the "coldstarts" in the log..)

I understand that the battery needs to be replaced, but why would this cause the UPS to kill power to all my systems?
There have been no power events, so why doesn't the UPS just continue to provide pass-through power from the wall.

It would appear that something is wrong or misconfigured on the UPS (but I can't find it) or there is a poor design that would sacrifice the stability of all servers just when a battery is low.

Any help would be appreciated.

Thanks.

Log Excerpt Attached

BH_apc · ‎2021-06-28

This was originally posted on APC forums on 7/24/2009

The reboots happen during off hours, so there is no one to see the unit's lights or hear beeps.

In addition, the servers come up with the unexpected shutdown message, which most likely means that they had power forcibly removed and were not asked to shut down via a network message.

- And another thing regarding the events log... it seems that you are saying that the log is only showing events from the NMC and not the UPS itself (or vice-versa)? Nowhere in the log is there any indication of any type of power event other than the battery needs replacing and the coldstarts (which previous poster said was an issue with the NMC). How is one to differentiate what is an event from the NMC vs. a UPS event? It seems that there should be a more detailed log other than the event.log, but that seems to be the only one that exists.

thanks

Message was edited by: BH

See Answer In Context

BillP · ‎2021-06-28

This reply was originally posted by Jonathan on APC forums on 7/23/2009

Hi Ben, can you also include the data logs from the network management card where this UPS is listening?

Techie_apc · ‎2021-06-28

This was originally posted on APC forums on 7/23/2009

The K-base ID: 2572 says that a Coldstart event indicates that the NMC (Network Management Card) has just been powered and has completed startup.
Click on the below link for more information.
http://nam-en.apc.com/cgi-bin/nam_en.cfg/php/enduser/std_adp.php?p_faqid=2572

If possible, could you please post the data logs which were recorded when the cold start events recorded?

BH_apc · ‎2021-06-28

This was originally posted on APC forums on 7/23/2009

I'm not sure what the other logs requested are.

The log I attached is the log that shows when I connect to the Management Card's web interface and go to the Event Log.

I see nothing under there that indicates a "Data Log"

The UPS mgmt. card is at 10.10.2.5 which is where I go for web access. I can also open up the APC software on each server that is configured for network shutdown.

Can you please elaborate on what data you need?

thanks

p.s. That link that describes the NMC behavior...not sure I understand it because it states that if a device powering the NMC loses power.... but isn't the NMC powered by the UPS itself? By that I mean the NMC is installed in the UPS and derives its power from the same source, namely the wall outlet.

So I'm not sure if you are trying to indicate that we are losing power from the wall and then the servers are getting powered off because of a bad battery. If that were the case, I would expect to see a UPS event that shows on battery power if even briefly. But, perhaps I am overlooking a log file as mentioned above.

Message was edited by: BH

BillP · ‎2021-06-28

This reply was originally posted by Jonathan on APC forums on 7/23/2009

Ben,

Here is the guide that can help you retrieve the data and event logs as well as the config.ini files on your network management card.
[How can I download Event logs and Data logs and Config.ini from the Network Management card?|http://nam-en.apc.com/cgi-bin/nam_en.cfg/php/enduser/std_adp.php?p_faqid=9321&p_created=1213880132&p...]

BH_apc · ‎2021-06-28

This was originally posted on APC forums on 7/23/2009

Followed those instructions.

Only one of those files that exists is the event.txt, which is where I provided the previous excerp from.

I have attached it in its entirety.

Thanks

BH_apc · ‎2021-06-28

This was originally posted on APC forums on 7/23/2009

More info...don't know if it is relevant.

This is an AP9606 NMC. When I took over this environment, there were some issues with installing the network software that required a firmware upgrade. I couldn't find any info from APC on how to get and/or perform an upgrade. Googling led me to results which allowed me to perform the firmware upgrade. I'm not sure if this was the original site I found the info, but here are the release notes and firmware update files I used:

http://www.moscoso.org/pub/hardware/apc/hardware/webcard/firmware/sumx/v326/

I don't know if there is another newer update, as stated I couldn't find anything from APC on updating the AP9606

BillP · ‎2021-06-28

This reply was originally posted by Jonathan on APC forums on 7/23/2009

Thanks for that information Ben. We're actually helping another customer upload the updated firmware on an AP9606 on another thread -- [http://forums.apc.com/spaces/6/racks-rack-accessories-cooling/forums/general/5818/2-inrow-sc-unusual...] you might want to check it as well.

BH_apc · ‎2021-06-28

This was originally posted on APC forums on 7/23/2009

Well, as stated I had already upgraded to a pretty new version. My telnet session reports the following:

Web/SNMP Management Card AOS v3.2.5
Smart-UPS & Matrix-UPS APP v3.2.6

I will review the other thread you posted, but unless there is a known issue with the problem I describe, then the firmware probably isn't the cause.

What else can we look to, (or what would you have me try) to determine why the UPS is rebooting all the servers on an almost daily basis. Again, I'm sure replacing the battery will prevent this (and we are ordering one now), but even with an old battery, the UPS shouldn't reboot the servers.

thanks

BillP · ‎2021-06-28

This reply was originally posted by Jonathan on APC forums on 7/23/2009

Ben,

During those times wherein the servers restarted, did the UPS beeped or any LED that lit up? What was the behavior of that unit? Have you tried disconnecting the card from the network? It could be that the card is sending shutdown signal over the network.

BH_apc · ‎2021-06-28

This was originally posted on APC forums on 7/24/2009

The reboots happen during off hours, so there is no one to see the unit's lights or hear beeps.

In addition, the servers come up with the unexpected shutdown message, which most likely means that they had power forcibly removed and were not asked to shut down via a network message.

- And another thing regarding the events log... it seems that you are saying that the log is only showing events from the NMC and not the UPS itself (or vice-versa)? Nowhere in the log is there any indication of any type of power event other than the battery needs replacing and the coldstarts (which previous poster said was an issue with the NMC). How is one to differentiate what is an event from the NMC vs. a UPS event? It seems that there should be a more detailed log other than the event.log, but that seems to be the only one that exists.

thanks

Message was edited by: BH