Arithmetic modul calculates wrong

[Sven_Kermer]

Hello,

I'm using an ION7550RTU to capture kWh-pulses. I use an arithmetic module for multiplying the pulse counter with the pulse weight. In my example, the pulse counter has the value 758093 and the pulse weight is 1.2

The arithmetic module multiplies both values. The result not  909711.6 as expected, rather 909711.625. The customer and me are confused, why the arithmetic module calculates something wrong. Of course the difference is very low, but not explainable.

This issue is existing at an ION7550RTU with firmware V371.

What can I do, to get a right result?

By the way:

At the VIP of a PME 7.2 this issue don't exist, the arithmetic module calculates the right result

Best regards

Sven

[Hal_Etheridge]

Internally the ION devices use a float32 for doing (most) calculations.  A float32 has a precision of around 7-8 digits.

The VIP uses a float64 (or double) for its calculations.  A float64 has a precision of 15-16 digits.