I have read your post twice now and I'm confused, but I'll give it a shot.
"Whats going on" - The firmware for the BackUPS is simply not designed to support the advanced functionality of the Li batteries.
"Is there a fix to this" - Yes, you could purchase a device that is specifically designed to properly support the functionality of Li batteries.
"Is there another brand" - Yes, E**t*n, for example, makes a P5x series. I'm sure there are others.
"I'm not looking to spend thousands" - Well, then you won't be buying Li products for quite a while. The aforementioned goes between $1250 and $1500.
Long before Schneider Electric took over, APC had a "3-5-7" approach for UPS devices. Batteries will usually last from 3 to 5 years, so they wanted to make sure you purchased an extended warranty. If the battery went during initial warranty, you were covered with a free replacement. If it went during the extension, you were also covered. At 7 years they wanted you to use the Trade-UPS program to replace the device with a newer one, offering a slight discount. That program eliminated BackUPS devices earlier this year - only SmartUPS are eligible.
I've been reselling APC UPSs for over a decade and my clients have never had to purchase a replacement battery outright.
The stated goals of Li batteries are longevity and the elimination of replacement batteries over the life of the unit. The technology - at least for now - is expensive.
Bottom line: I don't think you are going to achieve what you want from an ad-hoc solution, especially using consumer-based products.