Possible firmware issue on Network Management Card 2

[BillP]

My AP9630 arrived today. I installed it, found it with the config wizard and got it working, I then installed what I thought was the correct firmware but when I click the UPS tab I get a lost communication error.

My UPS is a "Smart-UPS 2200VA LCD 230V"

I installed firmware "apc_hw05_aos640_sumx640_bootmon108"

And basically get a lost communication error. or no UPS found.

What did I do wrong and how can I get out of this mess 😞 ?

*edit.

Sometimes it does show.

"

Model

Smart-UPS 2200

Location

Unknown

 No Alarms Present

• UPS is Online in Green Mode."

But as soon as I query the UPS it says lost communication followed by

"

 1 Critical Alarm Present

• Lost the local network management interface-to-UPS communication."

Thanks.

[BillP]

Factory Information

Hardware Factory

Model Number

AP9630

Serial Number

BA0929024138

Hardware Revision

05

Manufacture Date

07/17/2009

MAC Address

removed

Management Uptime

0 Days 0 Hours 5 Minutes

Application Module

Name

sumx

Version

v6.4.0

Date

Dec 18 2015

Time

15:14:26

APC OS (AOS)

Name

aos

Version

v6.4.0

Date

Dec 18 2015

Time

15:04:27

APC Boot Monitor

Name

bootmon

Version

v1.0.8

Date

Apr 8 2014

Time

10:59:40

Model:	Smart-UPS 2200
SKU:	SMT2200I
Serial Number:	JS1112033918
Firmware Revision:	UPS 07.1 (ID17)
Manufacture Date:	03/16/2011

[BillP]

You've probably already tried, but did you try resetting/rebooting the management interface via the CONTROL/NETWORK tab?  Also, what firmware revision level is your UPS sitting at?

[BillP]

Firmware Revision:UPS 07.1 (ID17)

So far I have tried a brain dead, re seated about 20 times,  Ieven got to the stage where I removed the accessory slot from the UPS and hooked up the cable direct. to rule out a slight misalignment issue with the connection.

I have found an older firmware (620) but it doesn't help, 640 is the same.

logs and stuff attached.

[voidstar_apc]

Hi Gareth,

What's the part number on the NMC2's PCB? It should look something like 640-1110a-Z.

[BillP]

640-1110A-Z_REV05

Thanks 🙂

[BillP]

Cards been pulled for now, It would start off working ok then slowly drop out more and more until the logs were showing it basically dropping off every minute (ish) and coming back a minute later (ish)

[voidstar_apc]

Hi Gareth,

Contact tech support. "1110A" is no good -- it is susceptible to noise on the UPS communication link. You want "1110B" or later.

http://www.apc.com/site/support/index.cfm/contact-information/

[BillP]

Hi Voidstar.  I have an NMC2, h/w rev level 640-1110A-Z_REV 05, that occasionally looses the date and time and resets itself to some incorrect point in the past.  I thought it may be the on-board battery but that measures +3.1vdc loaded and in circuit and appears to be good.

You mentioned that this "A" level card has issues and that tech support may be able to help.  Do you know if there is a fix or retrofit that can be done to the CCAs with the issue?

(EDIT) In an on-line chat with Schneider Electric, they don't seem to have knowledge of an issue with any hardware rev level of this board.

[voidstar_apc]

Hi Joe,

I'm not aware of any issues with the real time clock. The issue with "A" is on some UPSes the NMC doesn't reliably communicate with the UPS resulting in the lost communication messages Gareth described. They worked with the UPSes the card shipped with, but later on people sometimes move them into newer UPSes.

As far as the real-time clock goes, I would make sure you're running the latest firmware (as of now that's 640).

The reason I recommend talking to tech support is they generally have more knowledge of customer issues, solutions, and escalation steps.

[BillP]

Hi Voidstar,

Thanks for the clarifying message.  I think I can live with what I'm experiencing.

[BillP]

Phone support will not warranty this card as they say the warranty started with the original official reseller and not the company I purchased it from last year.

Now it seems I am stuck with a card that has a design fault if it is known to have the issues mentioned above, odd really. In the UK warranties do not apply to a design fault and products are recalled or repaired/replaced free of charge outside of any warranty.

[BillP]

I'll see what I can do about this for you. In order to do so though, we'd like to have the serial number of your UPS on file though too. Could you please get it ready and I'll have someone contact you about the NMC replacement?

Please note this is not necessarily a product "defect" in all situations but it depends on the older NMC (which why did you receive an NMC manufactured in 2009 to begin with for an order you placed today) and certain UPS models that may produce different amounts of electrical noise. We evaluate the issues on a case by case basis as this is not a widespread issue. The NMC hardware was updated to try and avoid the situation as much as possible but it can still happen depending on a few variables. And also, this internal lost communication between NMC and UPS message can be caused by a handful of other reasons too, such as a faulty UPS or problem with UPS communication bus. I don't want people to assume every time they see this message, the issue is always related to an older rev NMC and requires hardware replacement because that is certainly not the case.

In order to move forward too with your specific issue, make sure you have an NMC proof of purchase handy too for our records.

I'm not in UK support but I'll see if it can be escalated there.

[BillP]

Anglea, thanks for having a look. I have a case open with the UPS logged already from a previous ticket. 

I sent the required files for inspection but as it was already up to date firmware wise on the NMC and UPS they suggested that very little could be done on a firmware level as I was already on 6.40.

the logs fill up with this

07/17/2009 00:20:25 Device UPS: Lost the local network management interface-to-UPS communication. 0x0102
07/17/2009 00:19:17 Device UPS: Restored the local network management interface-to-UPS communication. 0x0101
07/17/2009 00:15:21 Device UPS: Lost the local network management interface-to-UPS communication. 0x0102
07/17/2009 00:14:45 Device UPS: Restored the local network management interface-to-UPS communication. 0x0101
07/17/2009 00:14:26 Device UPS: Lost the local network management interface-to-UPS communication. 0x0102
07/17/2009 00:12:37 Device UPS: Restored the local network management interface-to-UPS communication. 0x0101

Thanks.

Gareth.

[BillP]

Thanks. I think in your situation it is worth it to try a newer hardware rev NMC as voidstar mentioned but that really only applies if the card is from 2009ish, as yours is. For other customers where their event logs look similar and they may come across your discussion thread, I wouldn't think the issue exists with the NMC as a first step. Even now, I am not 100% sure but it makes sense to try it after you've checked NMC firmware and reseated the card. You could also double check for any other electrical noise makers, like having the UPS/NMC right next to something else that emits noise, such as a microwave or something less obvious than that

Please be on the lookout for follow up communications in the next day or so from your local CCC to move forward. Also, I see your cases but I do not see the UPS serial number mentioned in either of them so we'll need that either from the UPS front LCD or the white sticker on the rear of the unit. I only see NMC serial number or the serial number of a battery you had.

[BillP · Accepted solution]

Small update on this, Support emailed back and said they will replace the card so thank you Angela for sorting this & thank you Voidstar for the heads up.

And I spoke to the company who sold it me and they suggested to send up a AP9631 rev2 board that they use to test with, it arrived overnight and it has been stable for a few hours now so I know it is the card at fault 🙂