inftrees.c was subtracting an offset from a pointer to an array,
in order to provide a pointer that allowed indexing starting at
the offset. This is not compliant with the C standard, for which
the behavior of a pointer decremented before its allocated memory
is undefined. Per the recommendation of a security audit of the
zlib code by Trail of Bits and TrustInSoft, in support of the
Mozilla Foundation, this tiny optimization was removed, in order
to avoid the possibility of undefined behavior.
inftrees.c compared the number of used table entries to the maximum
allowed value using >= instead of >. This patch fixes those to use
>. The bug was discovered by Ignat Kolesnichenko of Yandex LC
where they have run petabytes of data through zlib. Triggering the
bug is apparently very rare, seeing as how it has been out there in
the wild for almost three years before being discovered. The bug
is instantiated only if the exact maximum number of decoding table
entries, ENOUGH_DISTS or ENOUGH_LENS is used by the block being
decoded, resulting in the false positive of overflowing the table.
Due to earlier changes in the error checking in inflate_table(), the
code to fill in a table for an incomplete code handled cases that can
never actually occur. This simplifies that code to handle the only
possible case, which is a single empty table entry for a code with
a single symbol with a length of one bit.
Also added "-motley" to ZLIB_VERSION in zlib.h, so that versions
in-between 1.2.5.1 and 1.2.5.2 that are pulled down from github
can be identified as such if bugs are reported on them.