TCP/IP stack failure with NMC

[BillP]

hi, the UPS did it's weekly self test last night but threw some errors listed below if anyone can help, not sure if this shows a configuration problem or just something to isgnore ( I was emailed to say the interface had regained connection to the UPS..

Network Management Card AOS v6.4.0
Smart-UPS & Matrix-UPS APP v6.4.0

 
20.02.2016 23:27:40 Device UPS: The power for switched outlet group 1, Aux, is on. 0x0185
20.02.2016 23:27:40 Device UPS: The power for the main outlet group, Main, is on. 0x0185
20.02.2016 23:27:39 Device UPS: Restored the local network management interface-to-UPS communication. 0x0101
20.02.2016 23:27:21 System Network service started. System IP is 192.168.1.240 from manually configured settings. 0x0007
20.02.2016 23:27:17 System Network Interface restarted. 0x0002
20.02.2016 23:27:09 System TCP/IP stack failure. Module: tfListRemove-assertion error line 132, file(.\treck\source\trlist.c)
0x001D
20.02.2016 23:27:09 System TCP/IP stack failure. Module: tfSheapFree-assertion error line 626, file(.\treck\source\trsheap.c)
0x001D
20.02.2016 23:26:56 Device UPS: The power for switched outlet group 1, Aux, is on. 0x0185
20.02.2016 23:26:56 Device UPS: The power for the main outlet group, Main, is on. 0x0185
20.02.2016 23:26:55 Device UPS: Restored the local network management interface-to-UPS communication. 0x0101
20.02.2016 23:26:37 System Network service started. System IP is 192.168.1.240 from manually configured settings. 0x0007
20.02.2016 23:26:33 System Network Interface restarted. 0x0002
20.02.2016 23:26:25 System TCP/IP stack failure. Module: tfListRemove-assertion error line 132, file(.\treck\source\trlist.c)
0x001D
20.02.2016 23:26:25 System TCP/IP stack failure. Module: tfSheapFree-assertion error line 626, file(.\treck\source\trsheap.c)
0x001D
20.02.2016 23:26:12 Device UPS: The power for switched outlet group 1, Aux, is on. 0x0185
20.02.2016 23:26:12 Device UPS: The power for the main outlet group, Main, is on. 0x0185
20.02.2016 23:26:12 Device UPS: Restored the local network management interface-to-UPS communication. 0x0101
20.02.2016 23:25:54 System Network service started. System IP is 192.168.1.240 from manually configured settings. 0x0007
20.02.2016 23:25:49 System Network Interface restarted. 0x0002
20.02.2016 23:25:41 System TCP/IP stack failure. Module: tfListRemove-assertion error line 132, file(.\treck\source\trlist.c)
0x001D
20.02.2016 23:25:41 System TCP/IP stack failure. Module: tfSheapFree-assertion error line 566, file(.\treck\source\trsheap.c)
0x001D
20.02.2016 23:25:41 System TCP/IP stack failure. Module: tfSheapGetFreeListBlock-assertion error line 823, file(.\treck\source\trsheap.c)
0x001D
20.02.2016 23:25:28 Device UPS: The power for switched outlet group 1, Aux, is on. 0x0185
20.02.2016 23:25:28 Device UPS: The power for the main outlet group, Main, is on. 0x0185
20.02.2016 23:25:28 Device UPS: Restored the local network management interface-to-UPS communication. 0x0101
20.02.2016 23:25:10 System Network service started. System IP is 192.168.1.240 from manually configured settings. 0x0007
20.02.2016 23:25:05 System Network Interface restarted. 0x0002
20.02.2016 23:24:58 System TCP/IP stack failure. Module: tfKernelPendCountSem-Unable to Post on uC/OS Semaphore 0x001D
20.02.2016 23:24:57 System TCP/IP stack failure. Module: tfListRemove-assertion error line 132, file(.\treck\source\trlist.c)
0x001D
20.02.2016 23:24:57 System TCP/IP stack failure. Module: tfSheapFree-assertion error line 626, file(.\treck\source\trsheap.c)
0x001D
20.02.2016 23:24:55 Device UPS: RFC1628 MIB UPS test completed. 0x3603
20.02.2016 23:24:55 Device UPS: Self-Test passed. 0x0105

[BillP]

Hi Gareth,

To help debug this, please provide the full .tar file available from About->Support in web interface. We need to look at a few log files beyond the event log you pasted.

If you need help gathering the .tar file, please refer to http://www.schneider-electric.us/en/faqs/FA156131 in the knowledge base.

[BillP]

Thanks! I need a day or two to get it looked at and I'll report back what I find.

In general, a TCP/IP stack failure means something networking related occurred which was not expected or handled. I don't see anything glaringly obviously so I need to have someone else check into it.

Though, I just noticed your configuration for a UPS self test says on UPS start-up and every 7 days after that. I see a self test happened "by automatic timer" on both the 20th and 21st...and the UPS itself did not restart either to force that.

[BillP]

Hi Gareth,

A few colleagues and I took a look at the log files. Also, one of the developers tried to replicate the issue but we could not. Have you seen the issue since we last spoke?

One suggestion was to investigate with you if you're using any RFC1628 SNMP generic UPS MIB functionality. We see those events in the log and if RFC1628 SNMP MIB stuff is configured but not in use, we could at least disable all of that and rule it out as related to this. I can tell you how to disable it if you're not using it.

Do you have any other APC NMC/UPS devices that you've seen this type of issue on at all?

[BillP · Accepted solution]

I have not seen this before or since so i will keep an eye on it and report back 🙂

Thanks for trying to replicate it.

Gareth.