Accuracy, scale, amount, split .. Truncation

I have the following code:

SELECT -701385.10 -- -701385.10 SELECT SUM(-701385.10) -- -701385.10 SELECT -701385.10/2889991754.89 -- -0.000242694498630 SELECT SUM(-701385.10)/2889991754.89 -- -0.000242 

In the last SELECT result is truncated to six decimal places. I read the article Precision, Scale and Length , and if my work does not work, I do not understand why truncation occurs. The expression type SUM(-701385.10) must be DECIMAL(38,2) - see SUM - therefore, the type obtained as a result of division must have:

Accuracy:

  • p1 - ​​s1 + s2 + max (6, s1 + p2 + 1)
  • 38 - 2 + 2 + max (6, 2 + 10 + 1)
  • 38 - max (6,13)
  • 38 - 13
  • 25

Scale:

  • max (6, s1 + p2 + 1)
  • max (6, 2 + 10 + 1)
  • max (6, 13)
  • 13

So why are truncated decimal places reduced?

+4
source share
1 answer

Your work is wrong

 Precision: p1 - s1 + s2 + max(6, s1 + p2 + 1) Scale: max(6, s1 + p2 + 1) 

gives

 Precision: 38 - 2 + 2 + max(6, 2 + 12 + 1) = 53 Scale: max(6, 2 + 12 + 1) = 15 

Which is greater than 38, so you get a truncation as described here

+4
source

Source: https://habr.com/ru/post/1383099/


All Articles