fix integer overflow in float printf needed-precision computation

if the requested precision is close to INT_MAX, adding
LDBL_MANT_DIG/3+8 overflows. in practice the resulting undefined
behavior manifests as a large negative result, which is then used to
compute the new end pointer (z) with a wildly out-of-bounds value
(more overflow, more undefined behavior). the end result is at least
incorrect output and character count (return value); worse things do
not seem to happen, but detailed analysis has not been done.

this patch fixes the overflow by performing the intermediate
computation as unsigned; after division by 9, the final result
necessarily fits in int.
This commit is contained in:
Rich Felker 2016-10-19 20:17:16 -04:00
parent aee6abb240
commit 70d2687d85

View File

@ -312,7 +312,7 @@ static int fmt_fp(FILE *f, long double y, int w, int p, int fl, int t)
}
while (e2<0) {
uint32_t carry=0, *b;
int sh=MIN(9,-e2), need=1+(p+LDBL_MANT_DIG/3+8)/9;
int sh=MIN(9,-e2), need=1+(p+LDBL_MANT_DIG/3U+8)/9;
for (d=a; d<z; d++) {
uint32_t rm = *d & (1<<sh)-1;
*d = (*d>>sh) + carry;