Brand new Symmetra LX - On bypass due to fault

[Anonymous user]

Hi,

Hopefully someone can help us with this.  We have just had a brand new Symmetra LX 16000 RM installed.  A few weeks later, we were getting lots of alerts:

09/25/2021    05:46:18    Device    UPS: The main intelligence module is now in control.    0x0220
09/25/2021    05:46:10    Device    UPS: A main intelligence module fault no longer exists.    0x0204
09/25/2021    05:45:50    Device    UPS: A main intelligence module fault exists.    0x0203
09/25/2021    05:45:49    Device    UPS: A main intelligence module fault no longer exists.    0x0204
09/25/2021    05:45:24    Device    UPS: A main intelligence module fault exists.    0x0203
09/25/2021    05:45:23    Device    UPS: Redundant Intelligence Module In Control    0x021F

AND

09/26/2021    17:45:54    Device    UPS: The number of batteries decreased.    0x0242
09/25/2021    05:46:33    Device    UPS: The number of batteries increased.    0x0241
09/25/2021    05:46:18    Device    UPS: The number of batteries decreased.    0x0242

APC suggested that we remove and reinsert the MIM.  We duly put it into logical and maintenance bypass and I removed and reinserted the MIM, then after 30 seconds and with the MIM status green as expected, removed and reinserted the RIM (just in case that was the issue, didn't want to do this twice).  The UPS then stated "UPS in bypass due to a fault".  Neither display or web access give any further clues as to the fault, everything shows as normal including power modules, MIM, RIM, batteries, etc..

That was last Thursday evening, so far APC have asked for some information and otherwise done nothing (despite desparate pleas for an update).  We have been stuck in maintenance bypass with no backup or under/overvoltage protection for nearly a week!

I suspect it's got confused and that a reboot of the UPS electronics will fix it.  As far as I can see, I can cycle the System Enable switch whilst in maintenance bypass without dropping the load?

Alternatively, maybe a self-test will force it to discover that there is no fault after all.  But again, I don't want to do anything to drop the load.

Suggestions very welcome!

Regards,

Sid

First time Symmetra user, currently wishes he stayed with multiple SmartUPS units 😞

[APC_Matt]

Hi Sid

I'm working with the UK customer care team to resolve this with you.

Matt

[Anonymous user]

14 days have now passed since our new Symmetra got stuck on bypass due to a fault.  After 14 days with no working UPS, we have still not received any useful suggestions from APC.  In some instances, 48 hours have passed without any updates.  Even the installer (a major UK data centre provider) was unable to get any useful suggestions from APC technical support.

Faced with another weekend with no working UPS, I took the matter into my own hands and started experimenting.  For the sake of any other Symmetra LX customers, here is how I resolved the faults "On bypass due to a fault" and "Fault, Power Module Will Not Turn Off":

1. Due to the issues listed in the original post, APC instructed us to reseat the Main Intelligence module.  It since turns out that these issues are a known firmware fault (not due to be fixed until December 2021)!
2. Reseating the MIM result in the UPS in maintenance bypass and reporting "On bypass due to a fault".  
3. After FOURTEEN days of waiting for APC to suggest something useful, we turned off the System Enable switch with the intention of restart the UPS.  After 30 seconds, turned on the System Enable switch.
4. UPS started up and reported "Fault, Power Module Will Not Turn Off".  Googling this message brought no answers.  No individual power modules listed as being at fault.
5. Checked all diagnostic information including raw status information.  All power modules, batteries, XR modules and intelligence modules showing no faults.
6. Removed and re-seated the Main Intelligence Module.  When it took back control from the Redundant Intelligence module, the fault disappeared and the fault light was extinguished.
7. Released maintenance bypass and logical bypass and the UPS is back running again.

My happiness in having a working UPS is balanced by the frustration and disappointment that we had to resort to workout out the solution ourselves.