All geek questions in one place

Calculate double value of nearest preferred decimal result

Let N (x) be the value of the decimal digit with the least significant digits such that x is the double value closest to the value of the digit.

Given the double values ​​of a and b, how can we calculate the double value of the nearest N (b) -N (a)?

eg:.

  • If a and b are double values, the closest .2 and .3,
    • the desired result is the double value, the closest .1,
      • 0.1000000000000000055511151231257827021181583404541015625,
    • and not the result of direct subtraction of a and b,
      • 0.09999999999999997779553950749686919152736663818359375.
+2
source share
3 answers

As a baseline: in Java, Double.toString() provides the N (x) function described in the question, returning its value as a digit. You can take the strings for a and b, subtract them by the elementary school method and convert the resulting string to double .

This demonstrates that a solution to the problem is entirely possible using existing library routines. This leaves the task of improving the solution. I suggest to study:

  • Is there a function D (x) that returns the number of significant digits after the decimal point for the digit described in N (x)? If so, can we multiply a and b by the force of ten, determined by D (a) and D (b), round as necessary to obtain the correct integer results (for situations when they are represented as double values), subtract them, and divide by the power of ten?
  • Can we establish criteria for which ba or some simple expression can be quickly rounded to something around a decimal digit, bypassing the code that is needed for more complex cases? For example, is it possible to prove that for numbers in a certain range (round(10000*b)-round(10000*a))/10000 always gives the desired result?
+2
source

You can convert to "integers", then multiplying the division by ten:

 (10*.3 - 10*.2)/10 == 0.1000000000000000055511151231257827021181583404541015625

Perhaps the corresponding ten power will be generated from the string representation of the number. @PatriciaShanahan suggests looking for duplicate 0 or 9.

Instead, use the BigDecimal library, such as javascript-bignum .

0
source

You can also query in Smalltalk Pharo 2.0, where your query translates:

 ^(b asMinimalDecimalFraction - a asMinimalDecimalFraction) asFloat

The code can be found as an attachment for release 4957 on code.google.com/p/pharo/issues - alas, a dead link, and the new bugtracker requires login ...

https://pharo.fogbugz.com/f/cases/5000/Let-asScaledDecimal-use-the-right-number-of-decimals

the source code is also on github, currently:

https://github.com/pharo-project/pharo-core/blob/6.0/Kernel.package/Float.class/instance/printing/asMinimalDecimalFraction.st

The algorithm is based on:

Robert G. Burger and R. Kent Dybwig
Print floating point numbers quickly and accurately
ACM SIGPLAN 1996 Conference on Programming and Development of Programming Languages
June 1996
http://www.cs.indiana.edu/~dyb/pubs/FP-Printing-PLDI96.pdf

0
source

Source: https://habr.com/ru/post/1502951/

More articles:


All Articles