AP9630/31 "The Application Is Incompatible With This APC OS"

[Anonymous user]

I appear to have run into the following issue:

http://forums.apc.com/spaces/7/ups-management-devices-powerchute-software/forums/general/10822/the-a...

DpZOF76DaCZIbZ5dryXGFA%3D%3D&Rest-Api-Key=d3Bhc3F1aWw6NjM3MzU2MWEtNzYzMC00OTc5LTkyMTAtOTJjYTU0NGEwZWFl.png

I have eight of the Smart-UPS 2200RM.  Seven of them have an AP9630, one has a 9631.  Yesterday, one after another, all of them crashed and got stuck on the incompatible screen on the web interface.  I'm able to FTP and telnet to six of them; those were disabled on two.  It looks like most of them are running 6.0.6, though I haven't looked at all yet.  I've been working with one in particular.  When I try to FTP upgrade files to it, it crashes.

I've attached two of the log files I was able to get.  I downloaded event.txt as well, but it looks like it's corrupt; Notepad++ seems to think it's a binary file.  As far as I can tell it's the firmware issue, but I was hoping the logs would give me something definitive.

Is there a way to get this fixed without losing all settings, and maybe more importantly, without having to cut power?

[Anonymous user]

Small update, I read my monitoring wrong, one of them did not crash.  Unlike the others, it's running 6.5.0 and bootmon 1.08.

[BillP]

Hi Bryan,

Yes, we need to get this firmware updated and several years old. I think the behavior is likely due to something we addressed. If the application cannot boot after 6 attempts, you'll receive this error. (You can also receive it if your app file is missing but from your screenshot, yours is not. Typically this is seen in the middle of a firmware upgrade which I know also you did not do to cause this.)

Are you able to check the event log? debug.txt and dump.txt is good but we'd hopefully see a little extra in the event log too - either via eventlog command in Telnet or if you can somehow get event.txt through FTP. It may be a TCP/IP stack failure or something which the event log would show.

Some advice I have would be to potentially disable all your email recipients for email generation temporarily and see if it allows you to upgrade firmware after that. Do you have email recipients configured? Each recipient has a setting - email generation (enable/disable) and when you disable it, it allows you to keep the event configuration but just turn off the actual sending of emails. (I know of one TCP/IP stack failure relating to email that would cause this issue, hence my suggestion to avoid having to do other things you want to avoid.)

And I assume you're trying the specific FTP method and instructions for NMC2 mentioned here? http://www.apc.com/us/en/faqs/FA156047 You need to make sure you're putting the files on in the correct order.

We want to get those updated to the latest firmware. AOS 6.5.6/6.5.6 application was just released a few days ago, not sure if that is what you tried. You can download it here: http://www.apc.com/shop/us/en/products/UPS-Network-Management-Card-v6-5-6-Firmware-for-Smart-UPS-Gal... and also feel free to update your 6.5.0 device to this rev as well.

Let me know if you have any questions.

[Anonymous user]

Hi,

Yes, I've been using the FTP method from the NMC2 directions, starting with bootmon.  That so far has been unsuccessful.  When I get to 'put bootmon108.bin', I get the following:

200 PORT Command okay.
150 File status okay; about to open data connection.

Then it just crashes.

I was able to use the eventlog command, it's just a loop of network restarts:

     04/24/2018  11:16:56  Device  UPS: Restored the local network management
                                   interface-to-UPS communication.
     04/24/2018  11:16:41  System  Network service started. IPv6 address
                                   FE80::2C0:B7FF:FE58:3AA5 assigned by
                                   link-local autoconfiguration.
     04/24/2018  11:16:37  System  Network service started. System IP is
                                   10.0.0.12 from manually configured
                                   settings.
     04/24/2018  11:16:31  System  Network Interface restarted.
     04/24/2018  11:16:04  Device  UPS: Restored the local network management
                                   interface-to-UPS communication.
     04/24/2018  11:15:49  System  Network service started. IPv6 address
                                   FE80::2C0:B7FF:FE58:3AA5 assigned by
                                   link-local autoconfiguration.
     04/24/2018  11:15:46  System  Network service started. System IP is
                                   10.0.0.12 from manually configured
                                   settings.
     04/24/2018  11:15:40  System  Network Interface restarted.
     04/24/2018  11:15:12  Device  UPS: Restored the local network management
                                   interface-to-UPS communication.
     04/24/2018  11:14:57  System  Network service started. IPv6 address
                                   FE80::2C0:B7FF:FE58:3AA5 assigned by
                                   link-local autoconfiguration.
     04/24/2018  11:14:54  System  Network service started. System IP is
                                   10.0.0.12 from manually configured
                                   settings.
     04/24/2018  11:14:47  System  Network Interface restarted.

Here's the output of the 'email' command, it was previously enabled but I disabled it this morning:

apc>email
E000: Success

  Recipient:    1
  Generation:   disabled
  Address:      alerts@example.com
  Format:       long
  Language:     enUs - English
  Route:        local

[BillP]

On 4/24/2018 2:06 PM, Bryan said:

Yes, I've been using the FTP method from the NMC2 directions, starting with bootmon.  That so far has been unsuccessful.  When I get to 'put bootmon108.bin', I get the following:

200 PORT Command okay.
150 File status okay; about to open data connection.

Then it just crashes.

Hmm, when I see it hang like this, it is typically that the FTP data port is blocked or not working. I've seen this with a firewall somewhere (PC or network) or even personally when my old VPN service was running in the background on my PC. Are you able to try via a different PC or look into potentially why this would be blocked?

No TCP/IP stack failure or other obvious reason at least beyond it rebooting after your FTP transfer attempt. Which I guess aligns with what I saw in debug.txt.

Does the .exe method fail too (it uses FTP but wondering what error it tells you)? On the AP9631, you could try USB upgrade method to fix that one and bypass the FTP problem.

If you can't see about the FTP data port being blocked, then we'll have to try and use one of the other methods - XMODEM, or we could try Secure Copy if we enable SSH but this firmware is older so I am not sure if it will work well as we've fixed issues and also SCP clients changed their things on their side. I've used pscp (in the PuTTy suite) with the most success across all firmware revs.

Thoughts?

[Anonymous user]

Oddly enough, the .exe method is working now on six of them (including the one that hadn't crashed, with the newer firmware).  That seems to be getting them all back online.  It wasn't working yesterday - maybe letting them sit overnight helped, they might have been stuck rebooting last time I tried.

The USB method worked for the AP9631.  One last AP9630 is still stuck, and FTP and telnet are disabled on it.  Is serial the only option for that one?

[BillP]

Hi Bryan,

Yes, XMODEM over serial for the last AP9630 would be needed, assuming SSH was not enabled and thus, SCP would be an option.

Glad you got most of them going at least!

[Anonymous user]

Yeah, apparently nothing but HTTP was enabled on it.  I had to order the required serial cable, these were installed years ago and that probably wasn't kept.  Thanks for the help!  I'll update this when the last one is resolved.

[BillP]

No problem.

Another thing you could try is press/release the pinhole reset button once to reboot the management interface. See if it recovers and you log in via HTTP to turn the other settings on?

[Anonymous user · Accepted solution]

I should have thought of that!  It did go back into a reboot and crash cycle, but the web interface was functional long enough I was able to enable FTP.  Once it crashed enough it was on the "Incompatible" screen, the upgrade tool worked.  Thanks!  All eight of them are on the latest release now.