I work in JavaScript, but the problem is general. Take this rounding error:
>> 0.1 * 0.2
0.020000000000000004
qaru.site/questions/561 / ... it gives a nice explanation. Essentially, some decimal numbers cannot be represented exactly in binary form. This is intuitive, since 1/3 has a similar problem in base-10. Now it works:
>> (0.1 * (1000*0.2)) / 1000
0.02
My question is: how does it work?
source
share