Setting up a Smart UPS 10000 for generator backup

[mlewis_apc]

I have set up an automated transfer switch so that if there is a power loss, a generator will kick in.

In testing, I could see 240VAC and around 63HZ yet the UPS would not use the power and instead stayed on batteries even though there was power.

The generator puts out 30AMPS and the load is only 15AMPS. There was plenty of time for the UPS to accept the generator power but it didn't.

What reason would it do that? The second light below the utility light was on and it stayed on batteries until it died.

Help... please!

[BillP]

Can you get me the full event.txt and data.txt logs from this time period? I need the data.txt to look at at least to compare to what you just posted.

[mlewis_apc]

I have put them both online. Do you still have the URL I gave you or do you want me to post it for a moment?

[BillP]

I have it. The issue I see is no matter if I go to /event.txt or /data.txt, I am only seeing the event.txt file and no data log. I also might as well ask - no changes recently in shutdown settings right? I looked at your old config.ini and I will go off of that assuming you did not change the low battery duration value.

[mlewis_apc]

I must have jumbled something up. Try again, I've reuploaded everything .

And no, haven't changed any settings. My assumption is batteries need to be changed, as we discussed before. Just weird that it held the load no problem, only losing one LED of the five, then the next time, didn't even hold the load for a moment.

[BillP]

Well, I'd definitely start with the management card logs and reviewing them from your testing time.

[mlewis_apc]

I looked and there wasn't anything much other than the UPS saying 'UPS: The battery power is too low to support the load; if power fails, the UPS will be shut down immediately.'. But, that was the error before anyhow.

[mlewis_apc]

I think I'll have to start from scratch again.

My concern is that testing is killing the batteries which aren't great to start with.

I'd like to be able to test in a way that puts the load on battery long enough to see if the UPS would take the genset power or not then back to utility.

[BillP]

Could I take a look at the log files? If you want to post them, can you tell me what time/dates to reference during the test?

I believe you are familiar with the log file download but here are instructions if you need it-> http://www.apc.com/support/index?page=content&country=US〈=en&id=FA156131

I am also interested in the [SmartUPS/Matrix] version of the config.ini to see what the thresholds for voltage/frequency are set to.

[mlewis_apc]

You can pick up the config.ini at /config.ini

You can pick up the log at /apc.log

Please let me know once done and I will delete them.

Please let me know if this is what you need.

The testing would have been on the 28th and 29th I believe. You can tell because the dns servers went missing when everything went down.

The 28th might actually be the test date because I see;

07/28/2013	23:36:17	System: Coldstart.
07/28/2013	23:25:53	UPS: The output power is turned off.

[BillP]

The instructions I sent show you how to FTP in to the NMC and you get config.ini, data.txt and event.txt. Then you can select "use advanced editor" in the reply box and attach the files there.

I need to look at the data log and event log at the same time. Event log alone (that you provided) doesn't answer all of my questions

[mlewis_apc]

	Hardware Factory Model Number: AP9619 Serial Number: ZA0443007046 Hardware Revision: A10 Manufacture Date: 10/21/2004 MAC Address: 00 C0 B7 71 0E A6 Management Uptime: 9 Days 16 Hours 7 Minutes
	Application Module Name: sumx Version: v3.5.8 Date: 11/05/2008 Time: 15:08:57
	APC OS (AOS) Name: aos Version: v3.6.1 Date: 08/13/2008 Time: 18:49:49

[mlewis_apc]

Same url, only /data.txt and /event.txt

[BillP]

I got event.txt and data.txt but I just cannot get to config.ini at the URL. You can remove them as I downloaded and saved them.

I am focusing on the 28th/29th but now I just need to verify the threshold settings via the config.ini for frequency and voltage to see if the problem was that even though it was not too far out of tolerance in your opinion from the generator, that the UPS setting was too restrictive.

[mlewis_apc]

Oops, thought you got that one already so had deleted it. It's back now

[mlewis_apc]

I have since been working on the generator which seems to have developed a problem. It doesn't appear to be putting out 240VAC anymore so I might have a voltage controller problem.

I did have a meter on it when I was testing and it was putting out 240VAC but it's possible that it showed that briefly while I was looking then might not have been.

Once you see the details, you might be able to tell me if my UPS settings are ok at least. They were not changed from the previous setup which was with a genset also but maybe resetting the card and powering off, etc, changed some settings.

[BillP]

Thanks! I also consulted my Smart UPS RT counterpart in escalation for some input. You can remove those files now.

Here are our observations:

• You only have 30% load on the UPS so you might be able to get away with a smaller sized generator but ideally, it'd be a 30kVA generator. (what size is it?) If that was the issue, it could be Total Harmonic Distortion on the input to the UPS but I know you've already mentioned potential generator issues but see point # 2.
• We did not see anything change between input on AC and generator as far as voltage/frequency.
• In the config.ini, your output frequency threshold is set to auto which basically says that the UPS will try to match the input frequency to output frequency. You could try adjusting it to 60Hz -/+ 3 Hz for testing as well but we don't believe it will make a huge difference but it is worth a shot.
• Other than that, not too much we can do overall because it is already a Double Online Conversion UPS and is designed to work with a varying range of input. No other issues with anything we found in the log files.

[mlewis_apc]

I am buying a new generator so I will update this thread once I have received and installed it and am ready to test again.

I hope you'll continue to help me once I have those things in place. I am extremely eager to get this working because I've lost two expensive storage devices thanks to power failures in the last couple of months.

Thank you!

[BillP]

Yes.. I am sure I'll be here

[mlewis_apc]

>You only have 30% load on the UPS so you might be able to get away with a smaller sized generator but ideally, it'd be a 30kVA generator.

That sounds very wrong. In the previous server room, the amperage being used was more like 25amps at a higher load on same UPS. I had a 16KW NG genset which kept the load up without a problem. At one point, we had the load on the genset for over 24hrs and there wasn't one single blip.

>In this case, (what size is it?) If that was the issue,

The new configuration uses a lot more virtualization so I was able to downsize a lot of the hardware and am now using 12amps to 14amps average at 30% load.

The genset I have now is a 7KW putting out 30amps at full load so I can't see it having a problem with this size load.

>it could be Total Harmonic Distortion on the input to the UPS but I know you've already mentioned potential generator issues but see point

Right, that's what I am thinking. I did make sure the genset was putting out 240VAC at 60hz when I was testing but a while into testing, I started to see the voltage being wrong. Further testing seemed to show new problems with the genset. I have a new one on order which should be here next week.

>Other than that, not too much we can do overall because it is already a Double Online Conversion UPS and is designed to work with a

>varying range of input. No other issues with anything we found in the log files.

So it might have and probably was the genset going bad then? Once I have the new machine connected, tested and ready to go, I'll post just before testing so that we have a fresh start. I can't afford any more outages because they are ruining some expensive storage devices. The batteries seem to hold the load long enough to get to genset so I'll deal with the batteries once the generator is in place.

I am very thankful for your help and input on this!!!

[BillP]

You're welcome. The generator size recommendation of 30kVA was based on 3 times the full capacity of your UPS which is typically our recommendation for this UPS type. You'd probably be fine with at least 1.5 times though.

We'll have to see on the generator but I'd have to say probably because we don't see anything else wrong at this time.

[mlewis_apc]

It should arrive on the 20th so if there's any luck, it will be installed that day or over the next two days then I'll update this thread.

[BillP]

Yes, everything I am being told seems like it is bad batteries. How old are they again? From going back to your previous posts, this UPS has (2) RBC44 and each RBC44 is (4) "trays." From what I read, previously, you only replaced (1) RBC44 or (2) trays? Is that right? I am asking because since you only replaced half, the batteries try to equalize and the older ones could've destroyed the RBC44 replacement you had done 1.5 years ago (according to the old post I found from June). Did I misunderstand? Your thoughts?

Regardless, the UPS should've indicated a bad battery but sometimes it happens randomly like this if there is an issue that a self test cannot detect or if the event happens before the self test.

What was the runtime reported now after all of this happen? I know the issue when you had 3 minutes reported..wondering if its back around there?

[mlewis_apc]

And that is what is so confusing. I don't want to spend the money just yet if I don't have to.

The unit tells me I have 5 minutes of run time which would be plenty for a genset fail over. I was on batteries \for only about two minutes I think while changing the breaker and only one LED down from full battery. Then when I tried it again, it turned itself off almost instantly. So it almost seems like it's not registering the right amount of battery power to begin with.

Input Voltage:	247.6 VAC
Output Voltage:	209.4 VAC
Runtime Remaining:                                                    5 minutes
Last Battery Transfer:	Unacceptable utility voltage rate of change

[mlewis_apc]

Runtime still shows 5 minutes.

[BillP · Accepted solution]

All of this still tells me bad batteries. In a perfect world, it would be obvious and I know it is annoying that the UPS is not actually lasting for the reported runtime but depending on the type of battery failure, it is possible the replace battery LED does not show or that beyond a certain amount of usage of the batteries per outage, that the runtime just drops off. The reported runtime while online is a really good estimate and when it switches to battery is when it is able to calculate it a lot better.

My recommendation here to address this is to replace both RBC44 in the UPS. If I look at the runtime chart, (UPS Runtime Charts) even at half load on SURT10000XLT you should be getting closer to 15 minutes, nevermind the 16-30% load that showes in your data log. The other thing that would be nice to know that I don't recall currently is how long it actually lasts on battery. If we go by what just happened, let's say that's about 2-3 minutes. It'd be a different issue if it lasted 20 minutes but only reported 5.

[mlewis_apc]

Alright, so I guess I'll have to budget for batteries and then make sure the UPS is reset to fresh batteries.

You didn't see anything in the settings which sounded too off so I'll use the same settings I have now but reset the batteries timer.

Thanks very much for everything on this. It is very much appreciated.

[BillP]

No problem! Glad to help. Settings looked fine as you mentioned. I'll be here if you need anything else

[mlewis_apc]

I have set up an automated transfer switch so that if there is a power loss, a generator will kick in.

In testing, I could see 240VAC and around 63HZ yet the UPS would not use the power and instead stayed on batteries even though there was power.

The generator puts out 30AMPS and the load is only 15AMPS. There was plenty of time for the UPS to accept the generator power but it didn't.

What reason would it do that? The second light below the utility light was on and it stayed on batteries until it died.

Help... please!

[BillP]

Hi mlewis

So just to be clear, you saw the on battery LED on, correct? See screenshot below.

That is the on battery LED with Online above it and bypass is below it so I just want to ensure you did not mean bypass because then we'd have to find out why it was in bypass.

Were you able to check the management card logs at all? I know that this UPS should accept +/- 5Hz if I am not mistaken...

Event and data log may help tremendously though to better understand why it was on battery.

[mlewis_apc]

Unfortunately, I'm going to have to get on this next week, I'm simply out of time. Yes, it's that LED.

Thanks. I'll get on this when I get back.

[mlewis_apc]

Ok, back from our trip and ready to tackle this. Badly need to get the UPS and the genset to work together so any help is very welcome!