ITO Discovery and Operations Connection Problem

[DCIM_Support]

Hello,

We have run a discovery in ITO and discovered around 120 devices on Operations. After 10-15 minutes we keep losing the connection to the ITO server and get the error as you may see on the screenshot. We can not work on Operaitons because we keep getting the same error. You may find the log files for ITO and operations at the links below. I think that because of too many devices were discovered, ITO server is not working properly. Can you please advice on how to proceed? 

Thanks in advance 

2015-12-03--03-07-23_Operations_Logs.7z.log_ITO_Logs.zip

(CID:105457315)

[DCIM_Support]

Hi Kaan,

Would you mind to share your ITO server hardware specs such as RAM size? and the number of the discovered devices?

According to the ITO logs it seems the server is experiencing out of memory exception and then apparently shutting down, and therefore you are seeing the communication error on DCO.

Kind regards

(CID:105457324)

[DCIM_Support]

Hello Kaan.

If you logon to your IT Optmize server, and change to the IT Optimize installation folder (default is C:\Program Files\SchneiderElectric\ITOptimize), do you see any files which have an extension .hprof (the filename is probably something like java_pidxxxx.hprof)?

If yes, can you send us the file? Its probably very large, i.e. 1GB+ in size.

The hprof file contains the details of the ITO process which ran out of memory. We can better analyze the cause of the problem and identify a solution.

Regards

Greg Sterling

(CID:105457368)

[DCIM_Support]

Hello,

Jef Faridi I have checked the specs. We have more then enough according to requirements. 4 CPUs and 16 GB memory. The discovered devices were 126. 

Greg Sterling you may find the link for the log files at the link below. it is around 180 MB. Please look into it and let me know if you need more information.

Thanks. 

http://we.tl/WVsh6DJhbJ

java_pid4476.zip 

(CID:105457757)

[DCIM_Support]

Thank you for the hprof file. I'm downloading it and will pass it along to the local engineering team for review. Regards Greg Sterling

(CID:105457801)

[DCIM_Support]

Engineering has started reviewing the hprof file. I will update you when we get results. Regards Greg Sterling

(CID:105457821)

[DCIM_Support]

Hi Kaan ( Kaan Kutlug ) Just a quick question, while Greg and the team is investigating the data, You have kindly shared the server spec's being 4 cpu's and 16GB of memory. Is this the DCO server specs? If yes, could you please also share the ITO server specs, thanks. Kind regards

(CID:105457894)

[DCIM_Support · Accepted solution]

Hello Kaan

During the polling cycles it appears ITO is attempting to poll a large number of VMware guests at which time the polling is exhausting its reserved memory space. To resolve this issue we'd like to you change a configuration parameter on the ITO server which will decrease the amount of reserved memory used by the poller when it executed.

Here's a procedure to change the number of assets ITO will attempt to simultaneously poll. This procedure will cause the poller to consume less memory. The amount of time ITO takes to complete polling of all your assets may increase.

* Logon to the IT Optimize server as an administrator user.

* Change to the IT Optimize installation folder. By default this would be: C:\Program Files\SchneiderElectric\ITOptimize\, but this destination folder can be changed during installation, so change to the proper location.

* In the "bin" sub-folder (so the full path would be C:\Program Files\SchneiderElectric\ITOptimize\bin), you'll find a file called "EnergyCenter.ini".

* Edit the EnergyCenter.ini file. The top set of lines in this file will look like this. There are more lines, I'm only showing the top of the file.

service.class=com.viridity.dcos.server.ViridityServer
service.id=EnergyCenter
service.name=Schneider Electric EnergyCenter
service.description=Data Center Energy Management Software
service.startup=auto
service.controls=stop|shutdown

process.priority=high
single.instance=process

working.directory=C:\PROGRA~1\SCHNEI~1\ITOPTI~1

vm.location=java\bin\server\jvm.dll
vm.version.min=1.6
vm.version.max=1.6
vmarg.1=-Xms128m
vmarg.2=-Xmx1024m
vmarg.3=-server
vmarg.4=-Djava.library.path=lib
vmarg.5=-Dviridity.dcos.home=C:\PROGRA~1\SCHNEI~1\ITOPTI~1
vmarg.6=-Dcom.sun.management.jmxremote.port=3333
vmarg.7=-Dcom.sun.management.jmxremote.ssl=false
vmarg.8=-Dcom.sun.management.jmxremote.authenticate=false
vmarg.9=-XX:+HeapDumpOnOutOfMemoryError
vmarg.10=-XX:MaxPermSize=256m
vmarg.11=-Xrs

Immediately after the line "vmarg.11=Xrs", add this line:

vmarg.12=-Dviridity.threadpool.poller.size=15

Save the file, and restart the IT Optimize service.

To restart the service:

* Open the windows services control panel

* Right click on "Schneider Electric EnergyCenter" service choose restart. Only the "EnergyCenter" service needs to be restarted, the "EnergyCenter Database" service does not need to be restarted.

Please let us know if the problem is then resolved.

Regards

Greg Sterling

(CID:105457922)

[DCIM_Support]

Hello Greg, we have followed the steps above and now everything works fine. Thanks for your support. Kind regards, Kaan Kutlug

(CID:105459055)

[DCIM_Support]

This question is closed for comments. You're welcome to start a new topic if you have further comments on this issue.