stbi__process_frame_header had two bugs when dealing with progressive
JPEGs:
1. when malloc failed allocating raw_data, previous components'
raw_coeff didn't get freed
2. no out-of-memory check in raw_coeff allocation
Fix both and share a bit more cleanup code in general.
Adds some helpers that check whether a product of multiple
factors (that need to be non-negative: this is enforced)
summed with another non-negative value overflows when
performed as int. Since stb_image mostly works in ints,
this seems like the safest route. Limits size of images
to 2GB but several of the decoders already enforce this
limit (or even lower ones).
Also adds wrappers for malloc that combine a mul-add-with-
overflow-check with the actual malloc, and return NULL
on failure. Then use them when allocating something that
is the product of multiple factors.
For image formats, also add a top-level "is this too big?"
check that gives a more useful error message; otherwise,
the failed mallocs result in an "out of memory" error.
The idea is that the top-level checks should be the primary
way to catch these bugs (and produce a useful error message).
But a misleading error message is still vastly preferable to
a buffer overflow exploit.
Fixes issues #310, #313, #314, #318. (Verified with the
provided test images)
Along the way, this fixes a previously unnoticed bug in
ldr_to_hdr / hdr_to_ldr (missing NULL check); these functions
are called with the result of an image decoder, so NULLs can
definitely happen.
Another bug noticed along the way is that handling of
interlaced 16-bit PNGs was incorrect. Fixing this (along
with the previous modifications) fixes issue #311.
Yet another bug noticed during this change is that reduce_png
did not check the right pointer during its out of memory
check. Fix that too.
I claimed that if the most significant bit of a 16bit pixel is set,
it should be opaque (as is suggested by some sources on the internet),
but implemented the opposite.
If implemented "correctly", lots of 16bit TGAs become invisible.. so I
guess 16bit TGAs aren't really supposed to have an alpha-channel, or at
least most 16bit TGAs (despite having set an "alpha-bit" in the "image
descriptor byte") in the wild don't seem to work like that.
So just assume 16bit non-greyscale TGAs are always STBI_rgb without
an alpha channel.
* Calculate correct stb format (incl. proper 16bit support) also when
using a colormap (palette)
* Create colormap with tga_comp, to correctly support 16bit RGB
(instead of using tga_palette_bits/8 and just copying the data)
* For TGAs with colormap, the TGA bits per pixel field specifies the
size of an index to the colormap - the "real" color depth
of the image is saved in the color map specification's bits per pixel
field. I think only 8 and 16bit indices make sense (16 should be
supported, otherwise the colormap length could be u8 instead of u16),
so I added support for both.
* Helper functions stbi__tga_get_comp() to calculate stb pixelformat and
stbi__tga_read_rgb16() to read one 16bit pixel and convert it to
24/32bit RGB(A) - for less duplicate code
