VBA rounding problem

I have this obscure rounding issue in VBA.

a = 61048.4599674847 b = 154553063.208822 c = a + b debug.print c Result: 154614111.66879 

Here's the question, why did VBA round the variable c? I did not perform rounding functions. The value I was expecting was 154614111.6687894847. Even if I round or change the variable c to 15 decimal places, I still do not get the expected result.

Any explanation would be appreciated.

Edit:

The expected results were obtained using cDec. I read this in response to Jonathan Allen's Why does CLng produce different results?

Here is the test result:

 a = cDec(61048.4599674847) b = cDec(154553063.208822) c = a + b ?c 154614111.6687894847 
+6
source share
2 answers

The reason is a limited precession, which can be stored in a floating-point variable.
For a complete explanation, you read David Goldberg's What Every Computer Scientist Should Know About Floating-Point Arithmetic, published in the March 1991 issue of Computational Research.

Paper Link

In VBA, the default floating point type is Double , which is a 64-bit (8-byte) IEEE floating point number.

Another type is available: Decimal , which is a 96-bit (12-byte) signed integer with a variable power of 10
Simply put, this provides a floating point number prior to the 28-bit precession.

For use in your example:

 a = CDec(61048.4599674847) b = CDec(154553063.208822) c = a + b debug.print c Result: 154614111.6687894847 
+11
source

This is unclear, but not necessarily obvious.

I think you somehow answered it, but the main problem is one of the "sizes" of the values, how much data can be stored in a variable of this type.

If (and this is very rude), you count the number of digits in each of the numbers in the first example, you will see that you have 15, while the range of values ​​that the float can represent (the default type) is huge accuracy is limited to 15 digits (I'm sure someone will fix this, I will mark the wiki window ...)

Therefore, when you add two numbers together, they lose the least significant values ​​in order to stay within the acceptable accuracy of the stream.

By executing cDec, you convert to another type of variable (decimal), capable of greater accuracy

+3
source

Source: https://habr.com/ru/post/896767/


All Articles