ACRC301S Internal Communication Error after firmware upgrade

[tolander_apc]

Hi

I have two ACRC301S units in a very small data center.
After applying firmware apc_hw06_aos632_acrc2g617_bootmon108.exe to one of the unit communications to the controller seems to have stopped.

The log reads:

Date    Time    User    Event

11/22/2017 06:48:31    tsreadonly  Undefined: Internal Communication Error.

Updated versions, Display:
Application Version v6.1.7
OS Version v6.3.2
Can't see Controller versions after upgrade

Before upgrade, Display:
Application Version v6.0.8
OS Version  v6.1.0
Controller
Firmware Revision   1.4.0
Hardware Revision   01

Any known issues or ways to resolve and restore the communications?

[BillP]

Hello. Try flashing the firmware again to the unit to make sure it went through fully. Also check the cable between the display and the board to look for any pinching or loose connection.

Thank you

[tolander_apc]

Hi

I tried rebooting the network interface and also tried flashing again, no difference.
It was working fine before the upgrade and nobody has touched the unit in between.

I am working remotely with the AC unit with network access this way.
It would be great to have a solution that saves me a trip! 

[BillP]

Hello. The only other thing over the network would be to try and downgrade the firmware to an earlier version in case something did not go fully through. unfortunately everything else requires to be on-site.

Thank you

[tolander_apc]

Where can I find previous firmware images?

What other parts is there to troubleshoot on site?

[BillP]

Hello. Please see attached fw. For the troubleshooting check the cable between the display and main board. Also try swapping displays with a different unit to see if the issue follows the display or the unit. From there please call your local tech support for any further steps.

Thank you

[tolander_apc]

Hi

Please explain how the display is related to the web interface throwing the error "Undefined: Internal Communication Error." after the upgrade.

[BillP]

The display also contains the network management card.  The Internal Communication Error is referencing the communication between the main board and the network management card.  As such, please check the cable running between the board and the display, and swap displays with an adjacent unit if possible.

Thank you.

[tolander_apc]

Downgrading does not seem to work that great:

Starting Upgrade 12/01/17 09:43:39 

-----------------

[12/01/17 09:43:39] : Pinging (via ICMP) device

[12/01/17 09:43:39] : Device connection passed

[12/01/17 09:43:40] : Testing FTP Log-in

[12/01/17 09:43:44] : FTP Log-in passed

[12/01/17 09:43:45] : Saving data file

[12/01/17 09:48:06] : Information --> Unable to verify previous version 

[12/01/17 09:48:16] : ERROR --> Incompatible HW

[12/01/17 09:48:16] : Incompatible HW: 

: Incompatible HW: 

[BillP]

Hello. To downgrade the firmware over the network try to use the FTP procedure below. This can be found in KBASE  FA156047. This unit uses a NMC2 card.

Thank you

. To perform a firmware flash with this method the Management Card must have been previously configured with a System IP, Subnet Mask, and Default Gateway and attached to the network. In addition, FTP Server must be enabled.

2. Open an MS-DOS command prompt window on a PC which is connected to the network. Change to the directory which contains the firmware upgrade files. Some example commands which need to be entered are shown in bold in order to change directories. Please note, this is just an example. You may need to enter a full path to a directory which requires different syntax.

C:\> cd apc
C:\apc>

 

3. Open an FTP client session.

C:\apc> ftp 192.168.1.100 (192.168.1.100 being the IP address of the web card)

 

4. Log in.

• NMC2 - This must be the Super User Account or an Administrator account. By default, the Super User username and password is apc. Press enter.

Typically, the password will appear on the screen as *** or not at all as you type.
 

Connected to 192.168.1.100
220- APC FTP server ready.
User (192.168.1.100 :(none)): apc
331 User name okay, need password.
Password: apc
230 User logged in, proceed.
ftp>
 

5. For NMC2, upgrade the bootmonitor file first. If the boot monitor module must be updated, a boot monitor module is included in the firmware release. Otherwise, the bootmonitor module that is installed on the card is compatible with the firmware update and can be skipped. If there is no included bootmonitor file, or you are using NMC1, upgrade the AOS first. The AOS module is the file with the AOS prefix.

 

ftp> bin (specifies binary transfer mode which is required)
200 Command okay.
ftp> hash (optional - enables hash mark printing as a way of monitoring file transfer progress)
ftp> put aosXXX.bin (XXX being the version number downloaded) Tip: With Windows Command Prompt in most OS versions, besides Vista, command prompt, you can drag and drop the file onto the command prompt window which will paste the full file path automatically here for you.
200 Command okay.
150 Opening data connection for aos301.bin

[tolander_apc]

Ok, after testing some more and going back to version 610 the communication to the controller and the network interface was restored.

Tried upgrading from 610 to 612 but then it was lost again.

Is there any know issues related to upgrading these versions?

[BillP]

Hello. I have seen a few times when using the executable, some of the information is lost over the network and the upgrade is not completed fully. If you have FTP enabled, try to push the bin files directly to the card. I have copied the procedure below. This will usually make sure the complete files get pushed to the card.

Thank you

1. To perform a firmware flash with this method the Management Card must have been previously configured with a System IP, Subnet Mask, and Default Gateway and attached to the network. In addition, FTP Server must be enabled.

2. Open an MS-DOS command prompt window on a PC which is connected to the network. Change to the directory which contains the firmware upgrade files. Some example commands which need to be entered are shown in bold in order to change directories. Please note, this is just an example. You may need to enter a full path to a directory which requires different syntax.

C:\> cd apc
C:\apc>

 

3. Open an FTP client session.

C:\apc> ftp 192.168.1.100 (192.168.1.100 being the IP address of the web card)

 

4. Log in.

• NMC1 -
  
  This must be the Administrator user account on NMC1. By default, the Administrator username and password is apc. Press enter.
• NMC2 -
  
  This must be the Super User Account or an Administrator account. By default, the Super User username and password is apc. Press enter.

Typically, the password will appear on the screen as *** or not at all as you type.
 

Connected to 192.168.1.100
220- APC FTP server ready.
User (192.168.1.100 :(none)): apc
331 User name okay, need password.
Password: apc
230 User logged in, proceed.
ftp>
 

5. For NMC2, upgrade the bootmonitor file first. If the boot monitor module must be updated, a boot monitor module is included in the firmware release. Otherwise, the bootmonitor module that is installed on the card is compatible with the firmware update and can be skipped. If there is no included bootmonitor file, or you are using NMC1, upgrade the AOS first. The AOS module is the file with the AOS prefix.

 

ftp> bin (specifies binary transfer mode which is required)
200 Command okay.
ftp> hash (optional - enables hash mark printing as a way of monitoring file transfer progress)
ftp> put aosXXX.bin (XXX being the version number downloaded)

Tip: With Windows Command Prompt in most OS versions, besides Vista, command prompt, you can drag and drop the file onto the command prompt window which will paste the full file path automatically here for you.
200 Command okay.
150 Opening data connection for aos301.bin

[tolander_apc]

That is the exact way I used for both downgrading and now upgrading from 610 to 612. Upgrading to 612 or higher results in "Internal Communication Error." between the display (network interface) and controller.

[BillP · Accepted solution]

Hello. Those are the only two ways to upgrade over the network. There is a way to upgrade through a serial connection but it requires to be at the unit. If this internal comm error keeps showing up when upgrading I will need to direct you to call into your local technical support to further troubleshoot this unit. It may be a connection or component or may need to be upgraded serially instead of over the network. You local support can be found on www.apc.com under support.

Thank you