IT optimize. Server not responding through WMI

[DCIM_Support]

Hi team,

We have about 250 server for which we are polling CPU Utilization through WMI. 

Now 10 of the servers are are showing "Not responding" and we are not able to get the CPU utilization. 

I checked with the server team and they told me that they havent made any changes in the server.

Kindly help me troubleshoot the issue.

Best Regards,

Ivin Tomy

(CID:152570003)

[DCIM_Support]

Hello Ivin,

Is ITO service running on your ITO server?

Does the test button work from the IT Optimize entry in the external systems view in the DCO desktop client?

Can you browser to https://:8090/ and logon?

Would you be able to share the console log file from your ITO server? If yes, please logon to your ITO server, and grab the files called console.log.0 in this folder:

C:\Program Files\SchneiderElectric\ITOptimize\.log

Please note you may want/need to compress this file before sharing as it could be large, and please note there is a "." in front of the "log" folder name.

Regards

Greg Sterling

(CID:152570164)

[DCIM_Support]

Hi Greg Sterling,

Thank you for your reply.

Yes, the DCO service is running. We are getting CPU Utilization for other 250+ servers.

Yes we are able to logon on the ITO server.

I've attached the logs as requested..log.7z

Thanks

Best Regards,

Ivin

(CID:152570779)

[DCIM_Support]

Hello Ivin, thanks for the log files. They are helpful.

The log file does show the inability to poll some assets. Here are a few examples:

Poller unable to reach address 10.11.19.11/10.11.19.11 via WMI
Poller unable to reach address 10.11.16.121/10.11.16.121 via WMI
Poller unable to reach address 10.11.19.10/10.11.19.10 via WMI
Poller unable to reach address 10.11.112.71/10.11.112.71 via WMI

Poller unable to reach address 10.11.96.113/10.11.96.113 via WMI

COM Exception occured while establishing WMI connection to address 10.11.96.113: 800706ba com.viridity.windows.wmi.ComException: The RPC server is unavailable.

COM exception in WMICollector::poll - address=10.11.16.108/10.11.16.108, variable=Win32_PerfRawData_PerfOS_Processor.Name?where name = '_Total', HResult=80041017 com.viridity.windows.wmi.ComException: IDispatch error #3607

COM exception in WMICollector::poll - address=10.11.16.108/10.11.16.108, variable=Win32_PerfRawData_PerfOS_Processor.PercentProcessorTime?where name = '_Total', HResult=80041017 com.viridity.windows.wmi.ComException: IDispatch error #3607

COM exception in WMICollector::poll - address=10.11.16.108/10.11.16.108, variable=Win32_PerfFormattedData_PerfOS_System.SystemUpTime, HResult=80041017 com.viridity.windows.wmi.ComException: IDispatch error #3607

The logs also include this data.

Unexpected source for polled data. IP address may have changed for asset RTAUEPRIMSDBCL2
Unexpected source for polled data. IP address may have changed for asset RTAUEPENCSDBCL1

Unexpected source for polled data. IP address may have changed for asset RTAUEPRMSDBST2

Unexpected source for polled data. IP address may have changed for asset RTAUEPTIBEMSCL1

Unexpected source for polled data. IP address may have changed for asset RTAUEPETQODBCL2

Unexpected source for polled data. IP address may have changed for asset RTAUEPTRAODBCL1

I assume the above servers are examples of the servers which are not responding and thus show the alerts in DCO?

It seems likely the IP addresses of the above servers may have recently changed so ITO cannot poll them anymore.

To address this issue, please re-run the discoveries defined in ITO so the above servers are rediscovered. This should correct the mismatch between what ITO thinks is the old IP address for the above servers, and what their actual IP address is now and polling should resume.

Regards

Greg Sterling

(CID:152570854)

[DCIM_Support]

HI Greg Sterling,

I've tried rediscovering the servers but still am getting the same error.

Could there be any other possibility for this ? 

Best Regards,

Ivin Tomy

(CID:153092441)

[DCIM_Support]

Can you repost the most recent couple of console.log files?

That would be console.log.0, console.log.1, and console.log.2 so I can see if the error changed?

Regards

Greg Sterling

(CID:153092667)

[DCIM_Support]

Hi Greg Sterling,

Please find the attached logs. Desktop.zip

Thank you

Best Regards,

Ivin

(CID:153092812)

[DCIM_Support]

Hi Greg Sterling

Any update ?

Best Regards,

Ivin

(CID:153093611)

[DCIM_Support]

Yes, sorry about the delayed response. as I was on holiday break.

The log file shows a number of errors when discovery is running which look like this:

Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Duplicate entry '10.11.88.3' for key 'IPADDRESS'

This may be caused by several issues, but it most commonly caused in environments which rely on DHCP to assign IP addresses to servers. The above message means ITO discovered a server with IP address 10.11.88.3 which has a serial number which does not match a server which already exists in ITO with the same IP address but a different serial number.

The only way to address this issue is to delete the duplicate entries from ITO before you run the discovery. This would mean you need to unassociate those devices in DCO first, then delete the entry from the unassociated items list.

So as an example:

serverA with serial number 12345 was discovered with IP 10.11.88.3

serverA with serial number 12345 is retired or is issued a new IP address by DHCP.

serverB with serial number 67890 is assigned IP 10.11.88.3 in your DC

You discover serverB, you will get the above error at the top of this response because ITO still thinks IP address 10.11.88.3 is assigned to serial number 12345.

To fix this issue. you must unassociate serverA from its asset in DCO (assuming its associated in DCO), and then delete the serverA entry from the unassociated items list (planned->device association). Once this is completed you should be able to discover serverB.

This is a full list of the IP's which are returning the duplicate IP error.

10.11.16.235
10.11.73.22
10.11.73.23
10.11.73.24
10.11.73.82
10.11.73.83
10.11.73.86
10.11.88.108
10.11.88.112
10.11.88.124
10.11.88.129
10.11.88.130
10.11.88.131
10.11.88.152
10.11.88.154
10.11.88.172
10.11.88.173
10.11.88.174
10.11.88.175
10.11.88.196
10.11.88.215
10.11.88.216
10.11.88.235
10.11.88.236
10.11.88.247
10.11.88.248
10.11.88.249
10.11.88.3
10.11.88.30
10.11.88.34
10.11.88.35
10.11.88.37
10.11.88.48
10.11.88.63
10.11.88.69
10.11.88.95
10.11.88.96
10.11.88.97
10.11.88.99

First task should be to resolve these items to cleanup discovery. Then we can re-check the console log file for ITO to see if there are additional exceptions.

Regards

Greg Sterling

(CID:153093642)

[DCIM_Support]

This question is closed for comments. You're welcome to start a new topic if you have further comments on this issue.