[UPS Network Management] "Unknown:Unknown" Notifications

[Anonymous user]

Hello everybody,

I have a Galaxy 300 UPS inside the company where i work.

I configure notifications by email concerning alerts and sometimes i receive this notification : Unknown:Unknown....

Where is the place where i can modify this notification ?

Thanks in advance.

greenkev

[BillP]

Hi greenkev

Using a Galaxy 300, I would expect to see a pre-installed NMC AP9630

Going on that, you can look at Administration > General > Identification

In this section, you would normally see the UPS name in the “Name” section.

In the “Contact” section, it defaults to Unknown. You could change this to show your name.

[Anonymous user]

Hi,

I already did that.

You can see it in attachment.

Another idea :s ?

Thanks in advance.

greenkev

[BillP]

Hi greenkev,

You mentioned “sometimes i receive this notification : Unknown:Unknown....”

Where do you see this notification?

Could you possibly send an image of this notification without sensitive data in view.

[Anonymous user]

Hello,

Sorry for the delay.

Here are the screenshots that i had all the time :

[BillP]

Hi Kevin,

Richard asked me to take a look at this. Could you provide me the full log files available from your NMC? I need event.txt and config.ini at a minimum to see where these events are coming from and what they mean. I may need to ask someone else and we need those logs to see the full history of what is occurring and why. And, why the events say unknown and what triggers them.

Instructions on grabbing the files are here: http://www.schneider-electric.us/en/faqs/FA156131

[BillP]

Thanks Kevin, just what I need. I have to check on this one. The hex event ID code in the Unknown:Unknown is 0x0007 and in the event log itself, I see those events but the correct text shows for them. So, for some reason in the email, it does not show the right text. This part of your question and issue I need to research a little bit. Has it ever happened before? And is it happening every single time after the Network Interface Restart messages happen in the log? Did you recently do a firmware upgrade? Any info to pinpoint when you started noticing it and when you see it happen would be helpful.

Meanwhile, you got these messages in the first place because your management card did a network interface warmstart/reset as explained here: http://www.apc.com/us/en/faqs/FA156063 If I had to guess, it may have been after all of those NTP update failures that lead to it. But, then I am seeing other network interface restarts that I don't see an obvious reason for. We'll need the dump.txt and debug.txt from the /dbg directory, which I think you can grab using those same instructions I gave you, just make sure to go into /dbg directory and grab them. That may give me a bit more of an inkling why we keep seeing batches of these messages:

23.06.2016	16:09:03	UPS: Restored the local network management interface-to-UPS communication.	0x0101
23.06.2016	16:09:03	System: Set Time.	0x0021
23.06.2016	16:13:10	System: Set Date.	0x0021
22.06.2016	16:12:50	UPS: Lost the local network management interface-to-UPS communication.	0x0102
22.06.2016	16:12:46	System: FTP user 'apc' logged in from confidential (note by greenkev).	0x0016
22.06.2016	16:12:42	System: Network service started. IPv6 address FE80::2C0:B7FF:FE8F:CB79 assigned by link-local autoconfiguration.	0x0007
22.06.2016	16:12:39	System: Network service started. System IP is confidential (note by greenkev) from manually configured settings.	0x0007
22.06.2016	16:12:34	System: Network Interface restarted.	0x0002

What I also see is some of those UPS: Lost local management to UPS communication messages and then it comes back a day later. So, something is also disrupting communication between your UPS and the management card. I see this on other smaller UPSs when random cables are attached to the UPS, there is electrical noise near the cables connecting the management card to the UPS, etc so I am not sure if anyone recently touched the UPS or the management card or if you are aware of "lost communication/regained communication" messages but this should not be happening regularly either.

[Anonymous user]

Hi,

Here are the answers of your questions :

Has it ever happened before?

Yes, but at a moment, i hadn't these messages for a long time and since two days, the messages reappeared.

And is it happening every single time after the Network Interface Restart messages happen in the log?

Yes, it is.

Did you recently do a firmware upgrade?

Not at all.

You will see in attachments the files do you need : dump.txt and debug.txt

How can i modify NTP parameters if the UPS restarts all the time ? By console ? What is the command that i need to do ?

Anyone touched the UPS because it situated in a lock technical local.

Thanks for your investiguation !

Don't hesitate to contact me if you want some nformation.

[BillP]

Hi Kevin,

I am researching these issues. I see a possible NMC firmware upgrade available but I have to see if it is available for customers yet since Galaxy 300 may require a UPS firmware upgrade first in order to apply it. I will follow up to see if you'd be able to apply it and potentially fix both of these issues you're seeing.

You're not doing anything wrong here. Even if an NTP update fails, it should not necessarily cause this type of problem even though you should maybe investigate why the NTP update keeps failing - like if an invalid or inaccessible server is configured as your NTP server. In the web, you can check under Administration->General->date/time and see the NTP settings in there.

Thank you for those files sent. I will follow up with you ASAP if I find out you can get an NMC firmware upgrade and what steps are involved in that. With this UPS, your local technical support or field service may need to get involved (not sure at this point).

[BillP]

Hi Kevin,

Do you know the exact model/variant of this UPS? I just need to double check it to see if it is supported by a newer firmware. I am not sure if you have local access to it in order to check or verify the model/SKU. At a minimum, can you tell me what is listed under UPS->About menu? That might help me.

Thanks! Depending on what we find, our new NMC firmware may be available now or around the middle of July to try.

[Anonymous user]

Hello,

I found the information do you want with about command by console.

You can see information in the screenshots that they are in attachments.

[Anonymous user · Accepted solution]

Good news !

Since i modified NTP settings (i modified the IP of our NTP server by 0.0.0.0 the standard IP) and since almost one hour, i didn't receive some alerts and i can connect to the UPS without problem by web, console or SSH.

The new firmware corrects NTP settings issues ?

[BillP]

Hi Kevin,

On the "about" info you sent me - that is the "about" of the management interface. I need the "about" of the UPS itself. There may be a upsabout command or similar. Or, in the web under UPS tab->About is what I'd need.

On your question about NTP, I don't know if there is a problem on the NMC side with NTP. I would want to know if there is an issue with the NMC reaching your NTP server or we'd have to dig further to see if we can pinpoint it. I am more confident that the newer firmware would resolve the Unknown notifications and it will just be a more robust operating system with additional debugging if we need it.

[Anonymous user]

Sorry for the wrong information.

Since i resolved the NTP settings, i can go on UPS web administration so it's easy for me to retrieve good information.

You will find the picture of "about" of the UPS itself in attachment.

[BillP]

Thanks Kevin - just what I needed to confirm. So yes, we have to wait a few weeks to try the firmware. I think if you got the NTP update failures taken care of by changing it as you noted, we can see if the new firmware will resolve the Unknown issues in the email (since I am sure eventually you may see it again if the NMC reboots for any other reason beyond why it was in this scenario).

When I see the new firmware come through, I'll let you know and respond here so that we can get it upgraded and monitor from there.

Let me know if you have any questions or comments for now.

[Anonymous user]

Thanks a lot, no problem for me !

Other important information :

When NTP update failures taken, i couldn't connect to UPS web administration, i couldn't connect by SSH and by console (it's very strange for the last one) !

It's normal ?

[BillP]

Hi Kevin,

When this happens, does access ever become restored? Did you see an error message when trying to access, especially when trying the web?

My first guess is that the NTP update failures would cause the System: Network Interface Restart messages which means the NMC is rebooting and it would be inaccessible - but only for 20-30 seconds max. Plus, through the web, you'd likely see "System is booting.." If you can get into it and check the event log for the time you tried to access it, you could see what messages were reported.

In general though, NTP update failures shouldn't cause the interfaces to be inaccessible. From what I saw before, the reboots after update failures were periodic but not very frequent.

[Anonymous user]

Before restoring the access, there had continuous reboots.

When i tried to access to the web, i write my credentiels and then the website tries to load the next content but, at the final, do nothing like we don't have network (i verified this point and i succeeded to ping the UPS).

[BillP]

Can you show me an event log excerpt from when this happened? The only thing I'd expect to cause what you've described is the continuous reboots we saw and you noted but they happen so quick I'd expect the page to refresh at least once and show you "System is booting" if you waited. 

The dump.txt and debug.txt too is helpful after this type of behavior along with the event log to check again.

But, I hope this type of unexpected behavior will be resolved in the newer firmware and if its not, we should be able to debug it a little better on our newer OS.

[Anonymous user]

Here are event logs when my issue happened :

28.06.2016	11:34:32	System: Network Interface restarted.
28.06.2016	11:30:43	UPS: Restored the local network management interface-to-UPS communication.
28.06.2016	11:30:43	System: Set Time.
28.06.2016	10:41:42	System: Set Date.
28.06.2016	10:41:26	System: Console user 'apc' logged in from serial port.
28.06.2016	10:41:22	UPS: Lost the local network management interface-to-UPS communication.
28.06.2016	10:41:13	System: Network service started. IPv6 address xxxx::xxx:xxxx:xxxx:xxxx assigned by link-local autoconfiguration.
28.06.2016	10:41:11	System: Network service started. System IP is ... from manually configured settings.
28.06.2016	10:41:05	System: Network Interface restarted.
28.06.2016	11:23:45	System: NTP update failure.
28.06.2016	11:17:20	System: NTP update failure.