Smart-UPS 3000VA Error during software update network adapter 9631

[Anonymous user]

Dear,

This morning I replaced an older UPS with a new one, but I also wanted to update the software of the network mnagement card.  I downloaded what I think I needed too (update for 9631 on Smart UPS), the installation program started with the boot monitor, but after that step and the restart, I can't access it anymore.

Starting Upgrade 09/04/14 15:07:10
-----------------
[09/04/14 15:07:10] 53.154.100.26: Pinging (via ICMP) device
[09/04/14 15:07:10] 53.154.100.26: Device connection passed
[09/04/14 15:07:11] 53.154.100.26: Testing FTP Log-in
[09/04/14 15:07:14] 53.154.100.26: FTP Log-in passed
[09/04/14 15:07:15] 53.154.100.26: Saving data file
[09/04/14 15:07:16] 53.154.100.26: AOS Prior to firmware transfer: Network Management Card AOS v6.0.6

[09/04/14 15:07:16] 53.154.100.26: APP Prior to firmware transfer: Smart-UPS & Matrix-UPS APP v6.0.6

[09/04/14 15:07:16] 53.154.100.26: Saving event & configuration files
[09/04/14 15:07:39] 53.154.100.26: Transferring firmware file (1/3): Boot Monitor
[09/04/14 15:07:47] 53.154.100.26: Boot Monitor transfer successful
[09/04/14 15:07:49] 53.154.100.26: Restarting network interface
[09/04/14 15:08:20] 53.154.100.26: Device connection passed
[09/04/14 15:08:22] 53.154.100.26: Testing FTP Log-in
[09/04/14 15:08:56] 53.154.100.26: ERROR --> FTP Log-in failed

If I ping the device, most of the time I get no reply, meaning that it can happen it replies (I also verified with Wireshark).  I think the card is in an "endless reboot loop", but I'd like to know if there is an option to get it working again ... preferably without shutting the UPS down...

Thanks,

Alain

[Anonymous user]

Dear,

This morning I replaced an older UPS with a new one, but I also wanted to update the software of the network mnagement card.  I downloaded what I think I needed too (update for 9631 on Smart UPS), the installation program started with the boot monitor, but after that step and the restart, I can't access it anymore.

Starting Upgrade 09/04/14 15:07:10
-----------------
[09/04/14 15:07:10] 53.154.100.26: Pinging (via ICMP) device
[09/04/14 15:07:10] 53.154.100.26: Device connection passed
[09/04/14 15:07:11] 53.154.100.26: Testing FTP Log-in
[09/04/14 15:07:14] 53.154.100.26: FTP Log-in passed
[09/04/14 15:07:15] 53.154.100.26: Saving data file
[09/04/14 15:07:16] 53.154.100.26: AOS Prior to firmware transfer: Network Management Card AOS v6.0.6

[09/04/14 15:07:16] 53.154.100.26: APP Prior to firmware transfer: Smart-UPS & Matrix-UPS APP v6.0.6

[09/04/14 15:07:16] 53.154.100.26: Saving event & configuration files
[09/04/14 15:07:39] 53.154.100.26: Transferring firmware file (1/3): Boot Monitor
[09/04/14 15:07:47] 53.154.100.26: Boot Monitor transfer successful
[09/04/14 15:07:49] 53.154.100.26: Restarting network interface
[09/04/14 15:08:20] 53.154.100.26: Device connection passed
[09/04/14 15:08:22] 53.154.100.26: Testing FTP Log-in
[09/04/14 15:08:56] 53.154.100.26: ERROR --> FTP Log-in failed

If I ping the device, most of the time I get no reply, meaning that it can happen it replies (I also verified with Wireshark).  I think the card is in an "endless reboot loop", but I'd like to know if there is an option to get it working again ... preferably without shutting the UPS down...

Thanks,

Alain

[BillP]

Hi Alain,

Yes, that is the correct file.

Are you local to the UPS/AP9631? If so, can you tell me what is going on with the LEDs on the AP9631 itself? There is the status LED (which is most important) which has a checkmark or X next to it and then the 10/100 network link LED. If it is easier, you could temporarily unplug the network cable to make sure you focus on the status LED. I need to know what color(s) it is and if it is solid or flashing. Observe it for about 2-3 minutes and let me know. That should be enough time to help me understand if it is indeed in a reboot cycle or not.

Depending on what you find, we can try a few different things.

1.) Just try to reseat/power cycle the AP9631 by removing it from the UPS SmartSlot for 5-10 seconds and sticking it back in. We'd recheck status LED behavior again for 2-3 minutes. (The card is hot swappable so you won't ever need to turn off the UPS itself.)

2.) Depending on status LED, we can try to serial into the card itself if you have the 940-0299 DB-9 to Tip Ring Sleeve 2.5mm cable that came with it.

3.) We can try formatting the card by holding the reset button for 20 seconds (which there are a few more details to that process I'll tell you if we get that far before you go ahead) but this does erase every piece of configuration on the card (TCP/IP settings, etc) so it will require some reconfiguration.

[Anonymous user]

Hi Angela,

I didn't work till this afternoon, and then the device reacted to ping again, but gave an error when I tried to access it again via the web interface.

I though it would be smart trying to do the installation again, but it wasn't, the device didn't react anymore.  The Status LED was always green (just until the reboot).  I tried the first step (removing the card for a while, and then putting it back).  Again the status led came green, but no IP-reaction ...

After a while (30 minutes?)  I saw it reacted to ping again, but the web-interface said it fails to load.  Via telnet I could get in, giving me the following "about":

Application Module
---------------
Name:                   bootmon
Version:                v1.0.8
Date:                   Apr  8 2014
Time:                   10:59:40

APC OS(AOS)
---------------
Name:                   aos
Version:                v6.0.6
Date:                   Apr  5 2013
Time:                   10:36:37

APC Boot Monitor
---------------
Name:                   bootmon
Version:                v1.0.8
Date:                   Apr  8 2014
Time:                   10:59:40

I don't think that was the idea of the update of the Application Module...

Tried ftp with filezilla, login ok, but I couldn't transfer the bin file for the aos.  Ftp manually didn't work ... until I gave the device some rest again   next attempt via ftp from the dos box worked for the aos-software, and secondly for the sumx-software as well.  "About" now shows the correct information (sumx v6.2.0 for application ; and aos v6.2.0) ; but I didn't have to do a restart of the device... (which is ok for me, as long as something works .. I think it just restarts when it recognizes the new bin file)

Also the webinterface is ok ...

But .. as I have 3 more to update, I was wondering what they would do ...

I though for the worst (but I knew the work around), but on the first it crashed a step later (so after update of bootmon & aos it stopped again, and the application module now shows it is aos v6.2.0)...

The second one had no problems, the third neither.

So, finally all is good for the management cards... Next on the planning ... update of the UPS-firmware, but as this involves a power down, this has to be scheduled a bit better... How useful is this to do, because of the impact I don't want to do just a 'nice to have'... (3 of the FJX3000RMHV2UNC-devices run on 05.8, the 4th on 06.1)? The last one could be possible next week, the others I have to be much more careful about..

[BillP]

Hmm OK. To help you understand, when you upgrade the .bin firmware files, they get stored in different spots in memory.

Bootmon goes first and I believe that goes in the application module section in flash, it reboots, then goes in its own section. Then, the AOS goes into the application's spot/location, reboots and goes in its own AOS spot in memory, and then the new application file goes in the old AOS's spot and eventually reboots and goes into the application spot. So, there is sort of an overwrite of things since the NMC does have space limitation.

Something must be failing mid firmware upgrade some place to get stuck. Usually what I see is customers with two AOS files and you just need to FTP to it and see what it says at the FTP prompt for the AOS version. I have not seen the two bootmonitor scenario but I am glad you got it working. Firmware upgrade via bootmonitor (which requires a serial connection) can always be done to recover the cards worst case. I cannot explain specifically what happened but can usually help depending on what files are present on the card.

Now, on your UPS firmware - I understand it is a hassle and you want it to be worth it. The latest for that UPS is actually UPS 06.5. I looked at the release notes for that and saw "Improve the reliability and behavior of system during a Self-Test." Doesn't mean there wasn't other fixes though For UPS 06.1, I see:

• Resolves issue where following reset to factory defaults, the Power Quality setting shows up with incorrect default value.
• Improve interoperability with software when updating the UPS Sensitivity setting.
• Improve interoperability with some HP servers.
• Improve reliability of repeated SNMP commands issued to optional network card.
• Improve the LCD behavior when a Legacy Communications Card (LCC) is installed or removed.

So of course, the UPS 06.1 updates would be in UPS 06.5 too. You can also use your NMC to update your UPS versus using the serial or USB cable to do it (which your UPS is just a variant of SMX mentioned in this article)-> How do I upgrade my SMT/SMX/SRT Smart-UPS firmware using my AP9630/AP9631 Network Management Card? |... but it does not change the fact that the output of the UPS must be off and also that you likely want to make sure that the network connection to your NMC is not powered by the UPS.

Let me know if you have any questions.

[voidstar_apc · Accepted solution]

Everything gets loaded into the app's spot, and then if not corrupt, copied into the appropriate space. That way if, for example, the AOS transfer fails, it will only overwrite the APP and the previous AOS will continue work accepting .bin files.