mirrors
/
postgres
mirror of https://github.com/postgres/postgres
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Clarify the choice of rscale in numeric_sqrt(). 

Improve the comment explaining the choice of rscale in numeric_sqrt(),
and ensure that the code works consistently when other values of
NBASE/DEC_DIGITS are used.

Note that, in practice, we always expect DEC_DIGITS == 4, and this
does not change the computation in that case.

Joel Jacobson and Dean Rasheed

Discussion: https://postgr.es/m/06712c29-98e9-43b3-98da-f234d81c6e49%40app.fastmail.com
This commit is contained in:
Dean Rasheed 2023-02-02 09:41:22 +00:00
parent 9a84f2947b
commit 0736fc1ceb
1 changed files with 15 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

17
src/backend/utils/adt/numeric.c
View File

@ -3693,8 +3693,21 @@ numeric_sqrt(PG_FUNCTION_ARGS)
init_var(&result);
/* Assume the input was normalized, so arg.weight is accurate */
sweight = (arg.weight + 1) * DEC_DIGITS / 2 - 1;
/*
* Assume the input was normalized, so arg.weight is accurate. The result
* then has at least sweight = floor(arg.weight * DEC_DIGITS / 2 + 1)
* digits before the decimal point. When DEC_DIGITS is even, we can save
* a few cycles, since the division is exact and there is no need to round
* towards negative infinity.
*/
#if DEC_DIGITS == ((DEC_DIGITS / 2) * 2)
sweight = arg.weight * DEC_DIGITS / 2 + 1;
#else
if (arg.weight >= 0)
sweight = arg.weight * DEC_DIGITS / 2 + 1;
else
sweight = 1 - (1 - arg.weight * DEC_DIGITS) / 2;
#endif
rscale = NUMERIC_MIN_SIG_DIGITS - sweight;
rscale = Max(rscale, arg.dscale);