ION Setup 6 decimals problem

[JuanMQT]

Dear experts,

My client is the electricity regulator, they are downloading data from a ION7650, and they would really like to see those decimals as you can see in the screenshot

The value stored is 14929.305664, and you can see it in the ION Setup V3.0.15121, but in the ION Setup V3.2.23060 you see 14929.306

 

What should I do in order to download the data with 6 decimals ?

[Charles_Murison]

Hello @JuanMQT ,

The number stored in the meter use 32 bit floating point numbers. General rule of thumb is that this format has ~7 digits of precision. The more exact calculation is decimals between 2ⁿ and 2ⁿ⁺¹ have fixed interval of 2^n-23

In your example ~14929 is between 2¹³ and 2¹⁴ , therefor the smallest precision is 2^-10 , or 0.0009765625. While ION Setup may have shown more decimal points in the past, these extra digits do not have significance. Please note that inside the meter, the energy is calculated with 64 bit math, it is just when communicating these values outside the module you are limited to 32 bit format. Another example 14929.305664 stored in float is 14929.3056640625, the next smallest number is 14929.3046875. While you thought you had 6 decimal places in reality the IEEE 754 float32 format is limited to no more than 8 decimal digits of precision, which for this example value has only 3 significant digits to the right of the decimal place.

see also https://en.wikipedia.org/wiki/IEEE_754. 

Regards,

Charles

[Robert_Lee]

The primary issue was the older version of ION Setup was showing decimal places that were not readily present in the transmitted data from the device.

Part of the ION conversion in the older ION Setup was taking the FLOAT32 value and casting it into a double precision variable which gave the user a false sense of data precision (more digits of precision than the data being transmitted).

Ex. The actual 32 bit value transmitted by the ION meter was 0x46694539 which if converted to the standard single precision value would be only 14929.306.  Casting it as a double precision value would yield 14929.305664062500 although the original data is unchanged.

To illustrate the issue with FLOAT32 precision when using larger values, the next available number that can be represented using FLOAT32 would be 14929.307 but using double precision would only be 14929.306640625000.

But with that said the new version of ION Setup should be showing 6 decimal places consistently (although likely the data should be shown as 14929.306000 for improved readability).

[JuanMQT]

Hello experts,

Thank you for your clarification, I will take care of explaining to the client,

I have one more question, what about these Vll3_avg data? Why do they sometimes have 6 decimals and sometimes 3 decimals?

[Robert_Lee · Accepted solution]

The differing number of decimal places is an incorrect application within ION Setup of when to perform the significant digit reduction.  At present this is being in done when the number being passed back has more than 10 significant digits.  When you see 6 decimal places and see how many significant digits (ex. 135023.125 has only 9 total which is 6 prior to the decimal, 3 post), it will show the number as is.  If however, it has more (ex. 134626.36167 which has 11 = 6 + 5), then it will attempt to *reduce* the number of digits to what FLOAT32 could represent properly (which I think would be 134626.36).

The next update of ION Setup in May will likely show readings in all cases with 6 decimals but will also restrict to the FLOAT32 limitation (so 134626.360000 would be shown).  This will make the readings easier to read but still reflect the limitation of the data being transferred (and not give a false impression of reading precision that isn't really available when the numbers get larger and larger). 

[darkenwald]

Did you solve the problem with the May update?

[Robert_Lee]

Yes all updates post May will now do the following when viewing data recorders:

1. Show 6 decimal places.

2. Restrict accuracy for FLOAT32 to 7 significant digits (thus when numbers begin to be larger than 9.999999, it will begin to only place 0 (zeros) at the end of the number).

Ex. 123.4567 will become 123.456700

This is to make it so viewing/importing into Excel, it is way easier to read the numbers while still keeping stated FLOAT32 resolution.