Welcome to the new Schneider Electric Community

It's your place to connect with experts and peers, get continuous support, and share knowledge.

  • Explore the new navigation for even easier access to your community.
  • Bookmark and use our new, easy-to-remember address (community.se.com).
  • Get ready for more content and an improved experience.

Contact SchneiderCommunity.Support@se.com if you have any questions.

Close
Invite a Co-worker
Send a co-worker an invite to the Exchange portal.Just enter their email address and we’ll connect them to register. After joining, they will belong to the same company.
Send Invite Cancel
84325members
353490posts

AP7724 and Phase Sync:Out of sync

APC UPS Data Center & Enterprise Solutions Forum

Schneider Electric support forum for our Data Center and Business Power UPS, UPS Accessories, Software, Services, and associated commercial products designed to share knowledge, installation, and configuration.

Solved
hachete_apc
Crewman
Crewman
0 Likes
3
620

AP7724 and Phase Sync:Out of sync

This was originally posted on APC forums on 5/23/2016


Hello everyone,

We have two different AP7724 ATSs which show the "Phase Sync: Out of sync" message.

This is the setup:

ATS feed 1: Smart-UPS RT 8000 RM XL  with this data:

Date  Time  Vmin  Vmax  Vout  Iout  %Wout  Freq  %Cap  Vbat  TupsC  %VAout  TIambC  %HumI 

  Smart-UPS RT 8000 RM XL Integrated EMU
Date  Time  Vmin  Vmax  Vout  Iout  %Wout  Freq  %Cap  Vbat  TupsC  %VAout  TIambC  %HumI 
 23.05.2016   11:53:12   221.7   224.6   231.6   10.17   32.0   50.01   100.0   216.0   32.4   29.0         
 23.05.2016   11:43:12   221.7   224.6   231.6   10.28   32.0   49.98   100.0   216.0   31.9   29.0         
 23.05.2016   11:33:12   221.7   226.0   231.6   10.07   33.0   50.00   100.0   214.9   32.4   29.0         
 23.05.2016   11:23:12   223.2   224.6   231.6   10.07   32.0   50.00   100.0   216.0   31.9   29.0         
 23.05.2016   11:13:12   221.7   226.0   231.6   10.07   33.0   49.99   100.0   216.0   32.4   29.0         
 23.05.2016   11:03:12   221.7   226.0   231.6   10.17   32.0   49.97   100.0   216.0   32.4   29.0         
 23.05.2016   10:53:12   223.2   226.0   231.6   10.17   32.0   50.00   100.0   216.0   32.4   29.0         
 23.05.2016   10:43:12   223.2   226.0   231.6   10.28   32.0   50.00   100.0   216.0   32.4   29.0         
 23.05.2016   10:33:12   223.2   226.0   231.6   10.07   32.0   50.01   100.0   216.0 

ATS feed 2: Smart-UPS RT 10000 RM XL  with this data: 

Date  Time  Vmin  Vmax  Vout  Iout  %Wout  Freq  %Cap  Vbat  TupsC  %VAout  TIambC  %HumI 
  Smart-UPS RT 10000 RM XL Integrated EMU
Date  Time  Vmin  Vmax  Vout  Iout  %Wout  Freq  %Cap  Vbat  TupsC  %VAout  TIambC  %HumI 
 23.05.2016   11:01:11   226.0   228.9   230.4   0.00   0.0   49.98   100.0   213.8   33.0   0.0         
 23.05.2016   10:51:11   227.5   228.9   230.4   0.00   0.0   50.00   100.0   216.0   33.0   0.0         
 23.05.2016   10:41:11   226.0   228.9   230.4   0.00   0.0   49.98   100.0   216.0   33.0   0.0         
 23.05.2016   10:31:11   227.5   228.9   230.4   0.00   0.0   49.99   100.0   216.0   33.0   0.0         
 23.05.2016   10:21:11   227.5   228.9   230.4   0.00   0.0   49.99   100.0   216.0   33.0   0.0 

 I've spent all morning reading about this and I can`t seem to find the problem because both feeds seem to me to be in sync.

Anyone around to give a hand?

Thanks

Daniel

Tags (4)

Accepted Solutions
Terry_Kennedy_apc
Commander
Commander
0 Likes
0
620

Re: AP7724 and Phase Sync:Out of sync

This was originally posted on APC forums on 7/1/2017


I think that FAQ was written in response to a discussion I had with APC...

It can still be an issue in 3.9.2, so we now insist on our datacenter A and B power being on the same phase (but from different sources) to avoid it.

There isn't really an ideal solution for this other than the above. Theoretically, you'd want to switch over as fast as you can. But the units use mechanical relays, so there is unavoidably some time delay during the cutover. As an example, if the ATS waited 1 second between opening (disconnecting) one source relay and closing (connecting) the other, the problem would never happen. But your equipment will get power cycled because most things can't tolerate a 1 second interval without power. So the delay gets shortened to something most equipment will tolerate. The other configuration options work to prevent reverting back to the original source (preferred source none), not switching for minor disturbances (sensitivity low), etc.

BTW, the reason the release notes warn you to disconnect one source when upgrading firmware is that there are 2 pieces of firmware - one for the (slow but complicated) management card, and one for the (fast but simple) ATS microcontroller. If there is a new version of microcontroller firmware, when it is written to the micrcontroller it will reset when done (like a reboot after Windows Update). While the microcontroller is rebooting it has no control over the source relays and they can be in an indeterminate state. If they happen to end up with source A and B on different phases, that is the same as trying to do a short circuit between the phases. That causes the breaker(s) to trip and damages the ATS relays.

It might be better for the hardware to use DPDT relays, with one side being source A and the other being B, with the common being the load. I'm not sure why the units don't do that - perhaps it does not meet isolation requirements.

See Answer In Context

3 Replies 3
BillP
Administrator Administrator
Administrator
0 Likes
0
620

Re: AP7724 and Phase Sync:Out of sync

This reply was originally posted by Angela on APC forums on 5/23/2016


Hi Daniel,

The Rack ATS products, such as AP7724, report phase out of sync when your input power sources are as little as around 10-11º out of phase. The ATS can switch sources that are as much as 180º out of phase (with firmware v3.5.7 or higher).

I would say it is extremely rare to have two input sources that are exactly 0º out of phase and thus many users see this message. You'll notice that this a status item on the ATS but not an alarm. The information you showed from the UPS data log really cannot show anything related to this message. You just need to make sure your UPSs are fed from different sources themselves and not use two UPS's that are on the same input source to avoid any problems with the ATS operation. This phase sync item won't be able to detect or report that to you though - I just meant it as a side note.

Beyond that, there is no problem here if your UPSs are fed from different input power feeds themselves as you won't be able to sync two power sources to exactly 0º majority of the time.

I hope that helps. Let me know if I was not clear.

hachete_apc
Crewman
Crewman
0 Likes
0
620

Re: AP7724 and Phase Sync:Out of sync

This was originally posted on APC forums on 6/30/2017


Hi Angela,

We indeed feed from two different sources, but we're now running into this problem when swithching sources although we're following all the recommendations : http://www.apc.com/us/en/faqs/FA156201/

  1. Dropping the load when the ATS switches sources while both appear to have power
The tripping of the circuit breakers during the switching process has been determined to be the result of switching between two sources that are out of phase with each other producing a voltage potential that exceeds the rating of the internal relays responsible for switching from one power source to the other. ATS firmware v3.5.7 and higher aims to resolve this issue and will help prevent the ATS from tripping the upstream circuit breakers
Any idea of what the solution could be to this problem?
Thank you,
Daniel
Terry_Kennedy_apc
Commander
Commander
0 Likes
0
621

Re: AP7724 and Phase Sync:Out of sync

This was originally posted on APC forums on 7/1/2017


I think that FAQ was written in response to a discussion I had with APC...

It can still be an issue in 3.9.2, so we now insist on our datacenter A and B power being on the same phase (but from different sources) to avoid it.

There isn't really an ideal solution for this other than the above. Theoretically, you'd want to switch over as fast as you can. But the units use mechanical relays, so there is unavoidably some time delay during the cutover. As an example, if the ATS waited 1 second between opening (disconnecting) one source relay and closing (connecting) the other, the problem would never happen. But your equipment will get power cycled because most things can't tolerate a 1 second interval without power. So the delay gets shortened to something most equipment will tolerate. The other configuration options work to prevent reverting back to the original source (preferred source none), not switching for minor disturbances (sensitivity low), etc.

BTW, the reason the release notes warn you to disconnect one source when upgrading firmware is that there are 2 pieces of firmware - one for the (slow but complicated) management card, and one for the (fast but simple) ATS microcontroller. If there is a new version of microcontroller firmware, when it is written to the micrcontroller it will reset when done (like a reboot after Windows Update). While the microcontroller is rebooting it has no control over the source relays and they can be in an indeterminate state. If they happen to end up with source A and B on different phases, that is the same as trying to do a short circuit between the phases. That causes the breaker(s) to trip and damages the ATS relays.

It might be better for the hardware to use DPDT relays, with one side being source A and the other being B, with the common being the load. I'm not sure why the units don't do that - perhaps it does not meet isolation requirements.