Types and rounds in SQL Server -


I have the following query:

  numeric as DECLARE @ A (36, 14) = 480 DECLARE @ B as Numeric (36,14) = 1 Select @ B / @ A cast cast (decimal as of B) / Cast (decimal as @)  

Why the first calculation returns 0.002083 and the other one 0.00208333333333333 ?

is not numeric (36,14) enough enough for a good precision (just as a second query)?

If I only use numeric , instead of numeric (36,14) , I is again a good precision:

  select cast (@ b numeric) / cast (numeric as @ A)  

You can calculate the exact and scale using the documentation from SQL Server Books Online .

I tried to make accurate calculation and scale for your case (operation = division, p = 36, s = 14) and I got a very strange result ...

  Accurate of result: [P1 - S1 + S2 + max (6, S1 + P2 + 1)] - & gt; 36-14 + 14 + max (6, 14 + 36 + 1) = 36 + 51 = Result of result: [Maximum (6, s1 + p2 + 1)] - & gt; Maximum (6, 14 + 36 + 1) = 51  

In this situation, precision is greater than 38 and in this case (as mentioned in the document)

* The result is precision and maximum is 38 on the scale. When the result is more than exactly 38, the same scale is being reduced, resulting in the reduction of an integral part of the result.

(87-38 =) must be less than 49 , it (51-49 =) < I think the minimum level length is 6 (due to expression scale = [maximum (6, S1 + p2 + 1)] ) and it can not be reduced to 6 - that we have as a result ( 0.002083 ).


Comments

Popular posts from this blog

apache - 504 Gateway Time-out The server didn't respond in time. How to fix it? -

c# - .net WebSocket: CloseOutputAsync vs CloseAsync -

c++ - How to properly scale qgroupbox title with stylesheet for high resolution display? -