AP9630 management cards keeps randomly rebooting

[Anonymous user]

I was reading up on several other threads regarding this same issue. Is anyone able to help me troubleshoot? This is a brand new UPS and a brand new SNMP card.

[BillP]

Hi Colby,

Yes, we can surely help you with this.

I need to know if it is rebooting constantly and you can't access it or is it rebooting constantly and depending on where the problem point in the boot process is, it might by design, stop booting after so many times and display a web page saying the application is unable to load. Can you tell me what you see or experience in relation to that? Or essentially, how do you know it is rebooting all the time?

If we can't get to it, I'll walk you through some steps to reset it.

If we can get to it enough to log in and obtain logs, refer to these instructions -> http://www.schneider-electric.us/support/index?page=content&country=US〈=en&locale=en_US&id=FA156131 . I am interested in the .tar file. If the web page is inaccessible but FTP is available, you can issue the logzip command then log in via FTP and get the debug_.tar from the /dbg directory and attach it here so I can review it. 

Here is a little more on restarts (warmstarts) in general -> http://www.schneider-electric.us/support/index?page=content&country=US〈=en&locale=en_US&id=FA156063 

In general, this can be a few different things - a weird or bad packet on the network throwing it into a tail spin, corruption somewhere, etc but we are usually able to figure it out or get it back up and running for observation.

Let me know if you have any questions. If you're in a rush and just want to get moving with it and don't care what is happening, the first thing I'd do is remove the network cable if one is attached, get a paperclip or similar and hold it to the top of the NMC reset button hole and press and hold the NMC reset button for 20+ seconds. If you're doing that properly, it will wipe ALL settings (and corruption) from the NMC and display a green "pulsing" status LED and change to orange/amber when time to let go.

[Anonymous user]

I can't even telnet to it long enough to issue any commands.

[BillP]

OK, I'd proceed with the format. Note: This will remove any TCP/IP settings though and require us to reconfigure it somehow. I'd remove the network cable (and it'd be easier to see everything) and get the paper clip and push and hold the factory reset button on the card for the 20+ seconds. As I mentioned, I find it easiest if you hold the paperclip against the top of the little hole for best alignment. If you've got a good hold on the little button, the status LED (with the checkmark and X) will be pulsing green and change to orange when its time to let go and it has been wiped. (Feels like forever when you're staring at it.) Afterwards, it will take 20-30 more seconds to finish and reboot and then keep an eye on the status LED after and note what you see for color and solid or flashing.

If you cannot afford to reconfigure the TCP/IP settings and want to try something else, try is unplugging the network cable and pressing and releasing the reset button once and observing the status LED. The status LED will be flashing green/orange rapidly when it is rebooting. You could see if it holds at solid green for a while after doing this which indicates it has stopped rebooting.

[Anonymous user]

I've already formatted it and then put the addressing in and I'm still not able get into it long enough to do anything useful.

[BillP]

That is with no network cable attached, correct? Where after format, it still does not behave?

What exact model of UPS is this?

Also, have you already pulled the card out for a few seconds and stuck it back in to power cycle it? I would try that if you haven't already.

[Anonymous user]

It seems like every time I try to web into it or telnet to it that it drops pings.

[BillP · Accepted solution]

So does it only start after you add TCP/IP settings and apply them? I want to rule out network connectivity to it as a cause is what I was getting at with my suggestions.

[glennm_apc]

Hi,

I have the same issue with one of our store UPS(SMT2200iC). We had replaced the NIC card a couple of months back due to this issue but now even the replacement card AP9630 is having the same issue.

When I try to access it, be it from DCIM or through WEB and telnet, it will reboot or lose the connection. However, when left alone it pings continuously. Is this an issue with the UPS or the cards?

Thanks,

Mark

[ChangNoiRoy]

I am experiencing the same issue.   I have 49 Smart SRT2200XLI UPS's with the NMC3 cards installed.  Out of the 49, I have had two of the NMC3 cards that continuously reboot.  I read that by design the NMC3 cars have a 9.5 minute watchdog timer where if they lose communication to the gateway, they reboot.   I have setup the gateway and verified the connections (direct on the bench) and this still happens.   It doesn't happen at the 9.5 minute mark, but rather it is random.  Sometimes it is very quick to reboot and other times it takes up to an hour (depending on the card and maybe the UPS combination).   I have reset these cards multiple times and formatted them and uploaded updated firmware many times.   The issue seems to be addressed for a short time and then returns.  Extremely frustrating and any help would be much appreciated.

[Teken]

Please call out what NMC 3 firmware this problem is seen on or provide a screen capture from the About tab.

Do both of the NMC 3 have SMTP enabled?!? If so delete all SMTP Server / User Accounts and reboot the NMC.

Report back the results of doing so along with validating on the switch the connection is 1GB and there are no port flapping error messages present.

Questions Ask . . . 👍

[ChangNoiRoy]

We utilize firmware version 2.3.1.1 on all of our NMC3 cards.  Even if we receive a replacement card with the newer 2.5.5.1 firmware, we downgrade to 2.3.1.1.    I have kept one of the cards at the latest version (2.5.5.1) and this issue still rears its head.   Regarding SMTP, we don't enable Simple Mail Transport Protocol (SMTP), but we do utilize SNMP.   Regarding a network switch, I am currently testing on my lab bench and there is no network switch.   I am connecting directly between the NMC3 card to my Lab laptop.   The ethernet connection indicates that it is at 1Gbs.

[ChangNoiRoy]

I have replied two times now and the response doesn't post.   This website seems to work just like my NMC3 cards!

[Teken]

I suggest you contact APC Technical Support to have this issue tracked and escalated for review.

In the interim you may consider uploading the diagnostic logs from the NMC while removing any PI information such as contact, location, etc.

Format the NMC to a OEM state and do not make any changes to the unit.

Make a small network with a switch, router, and monitor for any port flapping and the port remains at 1GB speed. Report back if the NMC shows warm / cold boot messages once the above is completed.

Questions Ask . . . 👍