Welcome to the new Schneider Electric Community

It's your place to connect with experts and peers, get continuous support, and share knowledge.

  • Explore the new navigation for even easier access to your community.
  • Bookmark and use our new, easy-to-remember address (community.se.com).
  • Get ready for more content and an improved experience.

Contact SchneiderCommunity.Support@se.com if you have any questions.

Close
Invite a Co-worker
Send a co-worker an invite to the Exchange portal.Just enter their email address and we’ll connect them to register. After joining, they will belong to the same company.
Send Invite Cancel
84766members
354199posts

DCO 7.5 SP4 - Application status starting or non responding

EcoStruxure IT forum

A support forum for Data Center Operation, Data Center Expert, and EcoStruxure IT product users to share knowledge on installation, configuration, and general product use.

DCIM_Support
Picard
Picard
0 Likes
9
359

DCO 7.5 SP4 - Application status starting or non responding

This question was originally posted on DCIM Support by Mate Fekete on 2016-09-12


DCO 7.5 SP4 server was in hung up state after 50 days uptime. I was able to connect only via local console but I saw the following:

Reboot command did not worked as well therefore left behind only force reboot from hypervisor. After 30 mins server uptime Operation service is still not running. I already uploaded the application logs just please tell me who can work on that and I will share with him.

(CID:108825069)

9 Replies 9
DCIM_Support
Picard
Picard
0 Likes
0
359

Re: DCO 7.5 SP4 - Application status starting or non responding

This comment was originally posted on DCIM Support by Jef Faridi on 2016-09-12


Hi Mate, Have you checked the memory settings? it seems you setup/system has run out of memory. It says "cannot allocate memory". It would be great if you could share your server logs (and client application log) with me. Server logs can be downloaded from server webmin interface (StuxureWare DC Operation > Download Log Files > "Download log files") Kind regards

(CID:108825077)

DCIM_Support
Picard
Picard
0 Likes
0
359

Re: DCO 7.5 SP4 - Application status starting or non responding

This comment was originally posted on DCIM Support by Mate Fekete on 2016-09-12


Hi, I uploaded the logs and shared with you. Regards, Mate

(CID:108825138)

DCIM_Support
Picard
Picard
0 Likes
0
359

Re: DCO 7.5 SP4 - Application status starting or non responding

This comment was originally posted on DCIM Support by Jef Faridi on 2016-09-12


Hi Mate, Many thanks for providing the logs - will get back to you as soon as possible, Kind regards

(CID:108825142)

DCIM_Support
Picard
Picard
0 Likes
4
360

Re: DCO 7.5 SP4 - Application status starting or non responding

This answer was originally posted on DCIM Support by Jef Faridi on 2016-09-13


Hi Mate,

According to the provided log files, there is a trace of long start up, that seems to be caused by UCS integration. During start up DCO 7.5 will fetch device data for the connected UCS managers. DCO is not completely started until data is fetched and persisted (log files indicate that it hangs on persisting the data). Scheduled jobs that perform similar tasks during operation does not seem to take several minutes.

In order to debug it might make sense to have a backup to test on.

Note: In DCO 8.0 data is not fetched during startup and should not prevent the server from starting.

Did the server start eventually?

Out of memory issue. Programs seems have used 7-8 GB more memory before server was restarted compared to right after the restart. It could indicate a memory leak. We cannot deduct what the leak is from the server log files. We could use a new set of log files in order to check if memory is currently being leaked.

Also the file (jobs.log) created by the following command could help debugging the issue:

ps -faux > jobs.log

If we can see from the new set of log files, that server is still leaking, then we might need an online session in order to debug.

Kind regards

(CID:108825408)

DCIM_Support
Picard
Picard
0 Likes
0
360

Re: DCO 7.5 SP4 - Application status starting or non responding

This comment was originally posted on DCIM Support by Mate Fekete on 2016-09-14


Hi Jef, 

Thank you for the investigation and feedback.

Yes, we have couple of UCS infrastructure which are using dedicated Managers. It would be welcome if I can use UCS Central integration instead of individual UCS Manager 😀

Did the server start eventually? No, I did not apply manual server start/reboot. It had 50 days uptime before I faced issue with the memory leakage problem. Just an additional information: DCO virtual machine can use 32GB allocated RAM and 4 vCPU.

I know about 8.0 and we already test it in our Lab and I am also planning to upgrade it in this year.

I uploaded "ps -faux" output however the file export was not worked for some reason. You can also see the latest DCO backup on appbox.

Regards,

Mate

 

(CID:108825467)

DCIM_Support
Picard
Picard
0 Likes
0
360

Re: DCO 7.5 SP4 - Application status starting or non responding

This comment was originally posted on DCIM Support by Jef Faridi on 2016-09-14


Hi Mate, You are welcome & many thanks for the info. I will get the files and get back to asap. Kind regards

(CID:108825468)

DCIM_Support
Picard
Picard
0 Likes
0
360

Re: DCO 7.5 SP4 - Application status starting or non responding

This comment was originally posted on DCIM Support by Mate Fekete on 2016-09-15


Do you have any update? Most of the users reported that facing performance issue with in DCO client which is unresponsive for 30-40 seconds and makes impossible the daily operation. I am open for a WebEx session anytime for further investigation.

I already uploaded one client log to the same store. I can see it is full with the following errors:

Error (Connection lost, response code:null) ExternalWebServiceService.getCollectedMetricDataFor() @ http://<DCO_IP_address>:80/isxo/core/external/ExternalWebService

(CID:108825655)

DCIM_Support
Picard
Picard
0 Likes
0
360

Re: DCO 7.5 SP4 - Application status starting or non responding

This comment was originally posted on DCIM Support by Jef Faridi on 2016-09-16


Hi Mate, Sorry for the delay, The setup contains a lot of UCS manager integration's and from thread logs it seems the server is doing a lot of (Hibernate) flushing during startup which can explain the long startup time. I had a brief discussion with our main developer, apparently the reason the start behaves differently from a normal UCS manager is because they are all done in the same go. In that way up to 10000 external items end up in the Hibernate 1st level cache, and may end with a lot of flushing. This is not an issue on DCO 8.0 since each job will do less flushing. So it is highly recommended to upgrade the product to the latest version (which contains many enhancements). Kind regards

(CID:108825662)

DCIM_Support
Picard
Picard
0 Likes
0
360

🔒 Closed

This question is closed for comments. You're welcome to start a new topic if you have further comments on this issue.