Network Mgmt card (AP9617) Warmstart, service start, restored comm, repeat

[Anonymous user]

I have a RM1500 with a AP9617 network management card.  Seems like every weekend the card starts having "fits".  The fits seem to stop, but may repeat.  As an example this past weekend it start to fit Saturday morning 10:27 through 15:19 and then did it again on Sunday starting at 9:29 through 13:42.  During these times there were multiple repeats of these episodes.  The hardware firmware version is A~0 and the application module is the latest 3.7.2.  Here is a group of the error messages.

07/13/2013	10:27:27	UPS: Restored the local network management interface-to-UPS communication.
07/13/2013	10:27:00	System: Network service started. System IP is 192.168.x.y from manually configured settings.
07/13/2013	10:27:00	System: Warmstart.

Any thoughts?  I am about ready to yank the card since it doesn't seem to be doing me much good with these type of spewing messages and try to move to the serial cable.  No devices that are connected to the UPS have any type of power issues.

[BillP]

"Official" procedure is to braindead the UPS and essentially turn it off and force it to discharge its capacitors but I never do that unless there is something wrong with the UPS to begin with. My personal recommendation for your situation is to just go ahead and remove it hot and there will be no effect to the UPS load or the UPS itself. You can also just install a new one hot too.

[Anonymous user]

I have a RM1500 with a AP9617 network management card.  Seems like every weekend the card starts having "fits".  The fits seem to stop, but may repeat.  As an example this past weekend it start to fit Saturday morning 10:27 through 15:19 and then did it again on Sunday starting at 9:29 through 13:42.  During these times there were multiple repeats of these episodes.  The hardware firmware version is A~0 and the application module is the latest 3.7.2.  Here is a group of the error messages.

07/13/2013	10:27:27	UPS: Restored the local network management interface-to-UPS communication.
07/13/2013	10:27:00	System: Network service started. System IP is 192.168.x.y from manually configured settings.
07/13/2013	10:27:00	System: Warmstart.

Any thoughts?  I am about ready to yank the card since it doesn't seem to be doing me much good with these type of spewing messages and try to move to the serial cable.  No devices that are connected to the UPS have any type of power issues.

[BillP]

Could you post the full event.txt for me to review please? (Instructions here -> How can I download Event, Data, Configuration, and Debug files from my Network Management Card? | FA... and again, I just need event.txt) You can post it as an attachment on your reply - just make sure you use the advanced editor option.

Also, I'd start by reviewing this to better understand what the message mean since it does not typically deal with any "power issues."

Why does my Network Management Card report warmstarts and coldstarts? | FAQs | Schneider Electric US

Once I review the event.txt, I can better suggest how to move forward. This could be related to times of the day where you have heavy network traffic (does that sound likely) and the management card is having trouble contacting its gateway as described in the article above?

[Anonymous user]

I have enclosed both the event and data files (just in case.)

I have reviewed the link previously and didn't see anything applicable.

Time of day nor network traffic is likely.  The weekend is the "slow" time for traffic

[BillP]

Ok I am looking at it now. I do see a few of these in there:

• System: TCP/IP stack failure. Module: tfSheapFree-assertion error line 610, file(.\treck\source\trsheap.c)

• System: TCP/IP stack failure. Module: tfUnLock-Lock is Corrupt

Let me see what I can find out more about those tomorrow. I haven't seen the second one before.

If I wasn't able to inquire further on what those specifically refer to and had to answer now, the first thing I'd do is reset the card to defaults (including or excluding TCP/IP settings depending on if you can configure those again) and reconfigure and monitor.

I'll post back tomorrow once I see what those mean if you just want to hang back.

[Anonymous user]

Any updates and/or insights based on the files?

[BillP]

Sorry for the delay drumidwest. Just re-checked on my request.

Basically - not good. Something is corrupted badly and obviously it shouldn't be. We should reset the entire card to defaults (you can back up your config.ini file to save your configuration) and monitor and if it happens again, the flash memory on the card could be faulty also and may need to be replaced.

[Anonymous user]

Thank you. I will reset it, though I believe I had to do that once before since it would not play nice with our DHCP server. If that doesn’t solve it I will presume the card is bad and move on from there.

[BillP · Accepted solution]

It might be because the DHCP Option 43 is enabled by default for this card/firmware. Do you have access to set this on your DHCP server cause its probably rejecting an address if it is not set. Alternatively, you can disable this option via a serial connection and then let it get a DHCP address. I am just guessing this is the issue you may be having though for DHCP.

Vendor Specific Information - (option 43) - this option encapsulates the APC cookie and the Boot Mode Transition setting.

The NMC uses this option in a DHCP response to determine whether the DHCP response is valid. This option contains an option in a TAG/LEN/DATA format, called the APC Cookie. This is disabled by default on NMC2 devices.

APC Cookie. Tag 1, Len 4, Data “1APC”

Option 43 communicates to the NMC that a DHCP server is configured to service devices. Following, in hexadecimal format, is an example of a Vendor Specific Information option that contains the APC cookie:

Option 43 = 0x01 0x04 0x31 0x41 0x50 0x43