Lua: subtracting decimal numbers does not return the correct precision

I am using Lua 5.1

print(10.08 - 10.07)

Instead of typing 0.01, the above prints 0.0099999999999998.

Any idea how to get 0.01 from this subtraction?

+3
source share
4 answers

You got 0.01 from the deduction. It is just in the form of a repeating decimal with a bit of precision lost.

Lua uses type C doubleto represent numbers. This is on almost every platform you will use, with a 64-bit binary floating-point value with approximately 23 decimal digits of precision. However, for an accuracy of 0.01, precision in the binary system is not accurate enough. The situation is similar to trying to write 1/3 in decimal form.

, , . .

. , , , (, ) . ​​ 100 .

, , , , - . .

:

  • Lua Users Wiki
  • IEEE, , , .
  • Wiki IEEE.
  • Wiki .
  • - .

: WECSSKAFPA. , , , . Knuth Volume 2 . , lhf ​​ , Lua wiki , .

+12

Lua string.format:

print(string.format('%.02f', 10.08 - 10.07))
+3

.

+1

Lua.

+1

Source: https://habr.com/ru/post/1761833/


All Articles