DCO 8.3.2 - Cisco UCS Manager integration is broken

[DCIM_Support]

Opened support case: #66927771

I’m running StruxureWare Operation version 8.3.2 and using Cisco UCS Manager integration in External System Configuration.

Sometime we do physical changes on Cisco UCS side:
- moving blade server from one chassis slot to another
- adding new blade servers
- replacing motherboard due to hardware failure

I think above mentioned changes are quite regular in a blade server environment.

In UCS Manager all above’s are updating correctly: I am able to see the appropriate blades’ serial number in the chassis slots after the physical changes.

But not in StruxureWare Operation! Even if the External System configuration is correct StruxureWare Operation is not showing the blades in the right physical locations.

It does not mean that none of the blade information is in DCO but after a physical movement that integration is not working as expected.

This problem generates another issue in my configuration. I also use VMware vCenter integration to follow up virtual machine information.
But for that I rely on the blade serial number information which is invalid based on above mentioned issue. Consequently the virtual machine location details from vCenter are referencing to an incorrect physical blade.

I already raised this problem a few years! ago but there was no any real solution for that. Here are some community posts:
https://community.ecostruxureit.com/questions/110005183/dco---ucs-manager-integration-is-not-updatin... [14/11/2016]
https://community.ecostruxureit.com/questions/128757950/dco---could-not-communicate-with-the-externa... [18/02/2018]
https://community.ecostruxureit.com/questions/147194440/dco---ucs-manager-list-discrepancy [11/07/2019]

Now I kindly ask the support / developers look into this issue and find a solution because the complete third party external system integration (VMware and Cisco) is useless in my environment.

Thank you in advance.

(CID:153944124)

[DCIM_Support]

Hi Mate,

I do recall the previously raised problems, issue seemed to be related to the external system syncing task not be executed and needed to be restarted either by rebooting or restarting the services. 

1- Do you know if rebooting the system or restarting the services "sudo systemctl restart operations" helps?

If it's okay with you, we can collect and investigate these updated data:

2- I guess this a clean 8.3.2 installation followed by restoring a backup file (from previous setup), right?

3- Which external system(s) do you now have issues with? please provide the host name and its software version details.

4- I recall you have many external sys. integrations, do you have issue with certain version(s) of those systems/Cisco UCS?

5- Can you provide a copy of the latest backup file (from DCO 8.3.2)

6- Can you provide screen captures (both from the external system and also from DCO) illustrating the issue (so we can find it in your setup).

7- Can you provide complete server log files

8- As an example. when (date and time) do you think there have been changes in real model (physical changes on Cisco UCS) that may not have been updated/reflected in DCO (this may help us to investigate the log files).

I will send you an invite to my =S= box shortly, so the data safely can be shared with me, thanks.

Kind regards 

(CID:153944586)

[DCIM_Support]

Hi Jef,

1 - Yes, I applied application restarts and server reboots which are did not solve the problem. I was also let the system running after the upgrade which obviously initiated minimum an application restart but did not change anything.

2 - No, I upgraded from 8.2.12 and used internet connection option (if you remember there was another question on the community about that)

3 - I provide to you a comprehensive data in excel. I exported all UCS blade details from DCO and we have a 100% reliable database about the blades' physical location. (We also use Cisco API and pullout a report in every 24 hours.)

4 - The excel also contains the running infra firmware versions. In nutshell: we have main version 2, 3 and 4's too so we are using all the available major versions from Cisco. I'm also involved into UCS management and I can tell to you those basic API calls (serial number, blade location) what DCO is using did not change in the different versions.

5 - Sure, uploaded.

6 - Sure, uploaded. But excel spreadsheet provides more accurate information.

7 - Sure, uploaded.

8 - It is hard to remember. I try to check with more details but the forthcoming will be also good for test: we are moving 50+ servers physically so I will have day and serials number what I can provide to you.

Let me do a bit "reverse engineering" and react on the last question: Do you see from the logs when was the last update / sync between UCS Manager and DCO?

As I recall your developer was not able to see it from the log last time. In my opinion it is the key and we have to go into direction: make the DCO capable to show when was the last update received from any external system in the logs. Even better in the GUI and in case any intermittent send an email notification?

I'm open for further discussion and let me know if you need further information regarding the uploaded data.

Thanks,

Mate

(CID:153945365)

[DCIM_Support]

Hi Mate,

Many thanks, it may take some time to investigate the data, but I will get back to you as soon as possible. 

Kind regards

(CID:154402914)

[DCIM_Support]

This question is closed for comments. You're welcome to start a new topic if you have further comments on this issue.