Types and rounds in SQL Server -
I have the following query:
numeric as DECLARE @ A (36, 14) = 480 DECLARE @ B as Numeric (36,14) = 1 Select @ B / @ A cast cast (decimal as of B) / Cast (decimal as @)
Why the first calculation returns 0.002083
and the other one 0.00208333333333333
?
is not If I only use numeric (36,14) enough enough for a good precision (just as a second query)?
numeric
, instead of numeric (36,14)
, I is again a good precision:
select cast (@ b numeric) / cast (numeric as @ A)
You can calculate the exact and scale using the documentation from SQL Server Books Online
.
I tried to make accurate calculation and scale for your case (operation = division, p = 36, s = 14)
and I got a very strange result ...
Accurate of result: [P1 - S1 + S2 + max (6, S1 + P2 + 1)] - & gt; 36-14 + 14 + max (6, 14 + 36 + 1) = 36 + 51 = Result of result: [Maximum (6, s1 + p2 + 1)] - & gt; Maximum (6, 14 + 36 + 1) = 51
In this situation, precision is greater than 38 and in this case (as mentioned in the document)
* The result is precision and maximum is 38 on the scale. When the result is more than exactly 38, the same scale is being reduced, resulting in the reduction of an integral part of the result.
(87-38 =) must be less than 49 , it (51-49 =) < I think the minimum level length is 6 (due to expression scale = [maximum (6, S1 + p2 + 1)] ) and it can not be reduced to 6 - that we have as a result ( 0.002083
).
Comments
Post a Comment