This patch contains:
* checks for malloc return value + treat callers;
* modified malloc() + ZeroMemory() to calloc();
* misc fixes of micro errors seen during the code audit:
** some invalid checks in gcc.c, also there were some possible
integer overflow. This is interesting because at the end the data are parsed
and freed directly, so it's a vulnerability in some kind of dead code (at least
useless);
** fixed usage of GetComputerNameExA with just one call, when 2 were used
in misc places. According to MSDN GetComputerNameA() is supposed to return
an error when called with NULL;
** there were a bug in the command line parsing of shadow;
** in freerdp_dynamic_channel_collection_add() the size of array was multiplied
by 4 instead of 2 on resize
* top level GDI functions return 0 on error and != 0 otherwise but the
low level functions (16bpp.c, 8bpp.c 32bpp.c) which are called did it
exactly the other way around. Those were adapted.
* change gdi_InvalidateRegion to BOOL and check calls where appropriate
* integrate comments from pull request
Change the return type of Stream_Ensure*Capacity from void to BOOL to be
able to detect realloc problems easily. Otherwise the only way to detect
this was to check if the capacity after the call was >= the required
size.
In case Stream_Ensure*Capacity fails the old memory is still available
and need to freed outside.
This commit also adds checks to most calls of Stream_Ensure*Capacity to
check if the call was successful.
refreshRectSupport and suppressOutputSupport of the General
Capability Set (MS-RDPBCGR 2.2.7.1.1) are server-only flags
that indicate whether the Refresh Rect or Suppress Output
PDUs are supported by the server.
Therefore in rdp_read_general_capability_set() we must only
change the respective settings if we are not in server mode.
(freerdp_color_convert_drawing_order_color_to_gdi_color): Declare new
function.
* libfreerdp/codec/color.c:
(freerdp_color_convert_drawing_order_color_to_gdi_color): Implement.
(freerdp_image_convert_8bpp): Properly use the ARGB32/ABGR32/RGB32/BGR32
macros when converting 8bpp data to 32bpp.
(freerdp_image_convert_32bpp): Fix CLRCONV_ALPHA and CLRCONV_INVERT
processing for 32bpp destination.
(freerdp_mono_image_convert): Use ARGB32/ABGR32 when converting to 32bpp
and CLRCONV_ALPHA is set.
* libfreerdp/core/orders.c: Color data from drawing orders is
interpreted in big endian mode.
* libfreerdp/core/update.c (update_read_palette): Likewise.
* libfreerdp/gdi/16bpp.c (gdi_get_color_16bpp): GDI colors are stored as
RGB now.
* libfreerdp/gdi/32bpp.c (gdi_get_color_32bpp): Likewise.
* libfreerdp/gdi/gdi.c:
Use freerdp_color_convert_drawing_order_color_to_gdi_color() to convert
from drawing order color representation to GDI color representation
troughout.
* libfreerdp/gdi/graphics.c (gdi_Glyph_BeginDraw): Likewise.
(gdi_Glyph_EndDraw): Likewise.
This patch introduce misc checks when receiving pointer updates. We check
that the cursor are in the bounds defined by the spec. We also check that
the announced mask sizes are what they should be.
