Commit Graph

53 Commits

Author SHA1 Message Date
Marc-André Lureau
0f9668e0c1 Remove qemu-common.h include from most units
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20220323155743.1585078-33-marcandre.lureau@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-04-06 14:31:55 +02:00
Markus Armbruster
b21e238037 Use g_new() & friends where that makes obvious sense
g_new(T, n) is neater than g_malloc(sizeof(T) * n).  It's also safer,
for two reasons.  One, it catches multiplication overflowing size_t.
Two, it returns T * rather than void *, which lets the compiler catch
more type errors.

This commit only touches allocations with size arguments of the form
sizeof(T).

Patch created mechanically with:

    $ spatch --in-place --sp-file scripts/coccinelle/use-g_new-etc.cocci \
	     --macro-file scripts/cocci-macro-file.h FILES...

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Cédric Le Goater <clg@kaod.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <20220315144156.1595462-4-armbru@redhat.com>
Reviewed-by: Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
2022-03-21 15:44:44 +01:00
Liao Pingfang
d560a06c60 vnc: Remove the superfluous break
Remove the superfluous break, as there is a "return" before.

Signed-off-by: Liao Pingfang <liao.pingfang@zte.com.cn>a
Signed-off-by: Yi Wang <wang.yi59@zte.com.cn>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Message-Id: <1594631086-36509-1-git-send-email-wang.yi59@zte.com.cn>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
2020-09-01 08:36:53 +02:00
Li Qiang
6bf21f3d83 vnc: fix memory leak when vnc disconnect
Currently when qemu receives a vnc connect, it creates a 'VncState' to
represent this connection. In 'vnc_worker_thread_loop' it creates a
local 'VncState'. The connection 'VcnState' and local 'VncState' exchange
data in 'vnc_async_encoding_start' and 'vnc_async_encoding_end'.
In 'zrle_compress_data' it calls 'deflateInit2' to allocate the libz library
opaque data. The 'VncState' used in 'zrle_compress_data' is the local
'VncState'. In 'vnc_zrle_clear' it calls 'deflateEnd' to free the libz
library opaque data. The 'VncState' used in 'vnc_zrle_clear' is the connection
'VncState'. In currently implementation there will be a memory leak when the
vnc disconnect. Following is the asan output backtrack:

Direct leak of 29760 byte(s) in 5 object(s) allocated from:
    0 0xffffa67ef3c3 in __interceptor_calloc (/lib64/libasan.so.4+0xd33c3)
    1 0xffffa65071cb in g_malloc0 (/lib64/libglib-2.0.so.0+0x571cb)
    2 0xffffa5e968f7 in deflateInit2_ (/lib64/libz.so.1+0x78f7)
    3 0xaaaacec58613 in zrle_compress_data ui/vnc-enc-zrle.c:87
    4 0xaaaacec58613 in zrle_send_framebuffer_update ui/vnc-enc-zrle.c:344
    5 0xaaaacec34e77 in vnc_send_framebuffer_update ui/vnc.c:919
    6 0xaaaacec5e023 in vnc_worker_thread_loop ui/vnc-jobs.c:271
    7 0xaaaacec5e5e7 in vnc_worker_thread ui/vnc-jobs.c:340
    8 0xaaaacee4d3c3 in qemu_thread_start util/qemu-thread-posix.c:502
    9 0xffffa544e8bb in start_thread (/lib64/libpthread.so.0+0x78bb)
    10 0xffffa53965cb in thread_start (/lib64/libc.so.6+0xd55cb)

This is because the opaque allocated in 'deflateInit2' is not freed in
'deflateEnd'. The reason is that the 'deflateEnd' calls 'deflateStateCheck'
and in the latter will check whether 's->strm != strm'(libz's data structure).
This check will be true so in 'deflateEnd' it just return 'Z_STREAM_ERROR' and
not free the data allocated in 'deflateInit2'.

The reason this happens is that the 'VncState' contains the whole 'VncZrle',
so when calling 'deflateInit2', the 's->strm' will be the local address.
So 's->strm != strm' will be true.

To fix this issue, we need to make 'zrle' of 'VncState' to be a pointer.
Then the connection 'VncState' and local 'VncState' exchange mechanism will
work as expection. The 'tight' of 'VncState' has the same issue, let's also turn
it to a pointer.

Reported-by: Ying Fang <fangying1@huawei.com>
Signed-off-by: Li Qiang <liq3ea@163.com>
Message-id: 20190831153922.121308-1-liq3ea@163.com
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
2019-09-17 13:45:10 +02:00
Markus Armbruster
a8d2532645 Include qemu-common.h exactly where needed
No header includes qemu-common.h after this commit, as prescribed by
qemu-common.h's file comment.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190523143508.25387-5-armbru@redhat.com>
[Rebased with conflicts resolved automatically, except for
include/hw/arm/xlnx-zynqmp.h hw/arm/nrf51_soc.c hw/arm/msf2-soc.c
block/qcow2-refcount.c block/qcow2-cluster.c block/qcow2-cache.c
target/arm/cpu.h target/lm32/cpu.h target/m68k/cpu.h target/mips/cpu.h
target/moxie/cpu.h target/nios2/cpu.h target/openrisc/cpu.h
target/riscv/cpu.h target/tilegx/cpu.h target/tricore/cpu.h
target/unicore32/cpu.h target/xtensa/cpu.h; bsd-user/main.c and
net/tap-bsd.c fixed up]
2019-06-12 13:20:20 +02:00
Li Qiang
18e845f7df vnc: fix unalignment access in tight_pack24
When adding '-fsanitize=undefined' in compiling configuration
and connect VM with vnc, it reports following error:

ui/vnc-enc-tight.c:910:13: runtime error: load of
misaligned address 0x621000466513 for type 'uint32_t',
which requires 4 byte alignment

This patch fix this issue.

Signed-off-by: Li Qiang <liq3ea@163.com>
Message-id: 20190318010442.14897-1-liq3ea@163.com
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
2019-03-18 10:49:18 +01:00
Philippe Mathieu-Daudé
949ed4c227 ui/vnc: Remove useless parenthesis around DIV_ROUND_UP macro
Patch created mechanically by rerunning:

  $  spatch --sp-file scripts/coccinelle/round.cocci \
            --macro-file scripts/cocci-macro-file.h \
            --dir . --in-place

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20180704153919.12432-7-f4bug@amsat.org>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
2018-08-24 08:40:11 +02:00
Marc-André Lureau
659c90eed8 vnc: use DIV_ROUND_UP
I used the clang-tidy qemu-round check to generate the fix:
https://github.com/elmarco/clang-tools-extra

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Richard Henderson <rth@twiddle.net>
2017-08-31 12:29:07 +02:00
Marc-André Lureau
01b2ffcedd qapi: merge QInt and QFloat in QNum
We would like to use a same QObject type to represent numbers, whether
they are int, uint, or floats. Getters will allow some compatibility
between the various types if the number fits other representations.

Add a few more tests while at it.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20170607163635.17635-7-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[parse_stats_intervals() simplified a bit, comment in
test_visitor_in_int_overflow() tidied up, suppress bogus warnings]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2017-06-20 14:31:31 +02:00
Alex Bennée
d9d2663c33 ui/vnc-enc-tight: remove switch and have single return
When enabling the sanitizer build it will complain about control
reaching a non-void function. Normally the compiler should detect that
there is only one possible exit given a static VNC_SERVER_FB_BYTES.

As we always expect a static VNC_SERVER_FB_BYTES I've added a compile
time assert and just called the sub-function directly.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
2016-09-28 12:55:09 +02:00
Peter Lieven
66668d197f vnc-tight: fix regression with libxenstore
commit 095497ff added thread local storage for the color counting
palette. Unfortunately, a VncPalette is about 7kB on a x86_64 system.
This memory is reserved from the stack of every thread and it
exhausted the stack space of a libxenstore thread.

Fix this by allocating memory only for the VNC encoding thread.

Fixes: 095497ffc6
Reported-by: Juergen Gross <jgross@suse.com>
Tested-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Peter Lieven <pl@kamp.de>
Message-id: 1468575911-20656-1-git-send-email-pl@kamp.de
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
2016-07-15 12:11:55 +02:00
Herongguang (Stephen)
3f7e51bca3 vnc-enc-tight: fix off-by-one bug
In tight_encode_indexed_rect32, buf(or src)’s size is count. In for loop,
the logic is supposed to be that i is an index into src, i should be
incremented when incrementing src.

This is broken when src is incremented but i is not before while loop,
resulting in off-by-one bug in while loop.

Signed-off-by: He Rongguang <herongguang.he@huawei.com>
Message-id: 5784B8EB.7010008@huawei.com
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
2016-07-15 12:11:55 +02:00
Peter Lieven
095497ffc6 vnc-enc-tight: use thread local storage for palette
currently the color counting palette is allocated from heap, used and destroyed
for each single subrect. Use a static palette per thread for this purpose and
avoid the malloc and free for each update.

Signed-off-by: Peter Lieven <pl@kamp.de>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Message-id: 1467280846-9674-1-git-send-email-pl@kamp.de
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
2016-07-12 08:34:13 +02:00
Peter Maydell
e16f4c8770 ui: Clean up includes
Clean up includes so that osdep.h is included first and headers
which it implies are not included manually.

This commit was created with scripts/clean-includes.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Message-id: 1454089805-5470-2-git-send-email-peter.maydell@linaro.org
2016-02-04 17:01:04 +00:00
Gonglei
525965b85d vnc-enc-tight: fix Arguments in wrong order
Arguments in wrong order (SWAPPED_ARGUMENTS)
The positions of arguments in the call to
tight_fill_palette do not match the ordering of the parameters:
 &fg is passed to bg
 &bg is passed to fg

Cc: Gerd Hoffmann <kraxel@redhat.com>
Signed-off-by: Gonglei <arei.gonglei@huawei.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
2014-12-10 10:08:12 +01:00
Benjamin Herrenschmidt
77bfcf28f1 console: Remove unused QEMU_BIG_ENDIAN_FLAG
If we need to, we should use the pixman formats instead but for
now this is unused except in commented out code so take it out
to avoid further confusion about surface endianness.

Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
2014-09-05 15:38:04 +02:00
Gonglei
b52991537c vnc-enc-tight: Fix divide-by-zero in tight_detect_smooth_image{16,24,32}
Spotted by Coverity:

(1) Event assignment:  Assigning: "pixels" = "0".
(2) Event cond_true:  Condition "y < h", taking true branch
(3) Event cond_false:  Condition "x < w", taking false branch
(4) Event loop_end:  Reached end of loop
(5) Event divide_by_zero:  In expression "(stats[0] + stats[1]) * 100U / pixels",
division by expression "pixels" which may be zero has undefined behavior.

290     DEFINE_DETECT_FUNCTION(16)
291     DEFINE_DETECT_FUNCTION(32)

Signed-off-by: Gonglei <arei.gonglei@huawei.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
2014-06-02 16:30:52 +02:00
Markus Armbruster
2e7bcdb99a vnc: Fix tight_detect_smooth_image() for lossless case
VncTight member uint8_t quality is either (uint8_t)-1 for lossless or
less than 10 for lossy.

tight_detect_smooth_image() first promotes it to int, then compares
with -1.  Always unequal, so we always execute the lossy code.  Reads
beyond tight_conf[] and returns crap when quality is actually
lossless.

Compare to (uint8_t)-1 instead, like we do elsewhere.

Spotted by Coverity.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
2014-03-10 12:35:04 +01:00
Gerd Hoffmann
d39fa6d86d vnc: stop using DisplayState
Rework DisplayStateListener callbacks to not use the DisplayState
any more.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
2013-03-18 10:21:58 +01:00
Paolo Bonzini
1de7afc984 misc: move include files to include/qemu/
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2012-12-19 08:32:39 +01:00
Paolo Bonzini
7b1b5d1913 qapi: move include files to include/qobject/
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2012-12-19 08:31:31 +01:00
Gerd Hoffmann
bc210eb163 pixman: fix vnc tight png/jpeg support
This patch adds an x argument to qemu_pixman_linebuf_fill so it can
also be used to convert a partial scanline.  Then fix tight + png/jpeg
encoding by passing in the x+y offset, so the data is read from the
correct screen location instead of the upper left corner.

Cc: 1087974@bugs.launchpad.net
Cc: qemu-stable@nongnu.org
Reported-by: Tim Hardeneck <thardeck@suse.de>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Signed-off-by: Blue Swirl <blauwirbel@gmail.com>
2012-12-14 20:56:19 +00:00
Gerd Hoffmann
94362682d3 pixman/vnc: remove dead code.
Switching the vnc server framebuffer to use 32bpp unconditionally
turns the code bits which handle 8 and 16 bpp into dead code.
Remove them.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
2012-11-01 14:00:05 +01:00
Gerd Hoffmann
47683d669f pixman/vnc: remove rgb_prepare_row* functions
Let pixman do it instead.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
2012-11-01 14:00:04 +01:00
Gerd Hoffmann
9f64916da2 pixman/vnc: use pixman images in vnc.
The vnc code uses *three* DisplaySurfaces:

First is the surface of the actual QemuConsole, usually the guest
screen, but could also be a text console (monitor/serial reachable via
Ctrl-Alt-<nr> keys).  This is left as-is.

Second is the current server's view of the screen content.  The vnc code
uses this to figure which parts of the guest screen did _really_ change
to reduce the amount of updates sent to the vnc clients.  It is also
used as data source when sending out the updates to the clients.  This
surface gets replaced by a pixman image.  The format changes too,
instead of using the guest screen format we'll use fixed 32bit rgb
framebuffer and convert the pixels on the fly when comparing and
updating the server framebuffer.

Third surface carries the format expected by the vnc client.  That isn't
used to store image data.  This surface is switched to PixelFormat and a
boolean for bigendian byte order.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
2012-11-01 14:00:04 +01:00
Anthony Liguori
7267c0947d Use glib memory allocation and free functions
qemu_malloc/qemu_free no longer exist after this commit.

Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2011-08-20 23:01:08 -05:00
Stefan Weil
2fb0c09f4f vnc: Fix compilation with --enable-vnc-png
Commit f26e428da5 fixed compilation
with --enable-vnc-png, but broke it with --enable-vnc-png.

The breakage is caused by pngconfig.h which checks whether
setjmp.h was already included and fails because qemu-common.h
includes setjmp.h.

The check is disabled by defining PNG_SKIP_SETJMP_CHECK.

Cc: Blue Swirl <blauwirbel@gmail.com>
Signed-off-by: Stefan Weil <weil@mail.berlios.de>
Reviewed-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2011-06-27 10:21:34 -05:00
Roy Tam
f26e428da5 Fix MinGW compilation when --enable-vnc-jpeg is specified
Fix conflicting types for 'INT32' in basetsd.h and jmorecfg.h by
including qemu-common.h first.

Signed-off-by: Roy Tam <roytam@gmail.com>
Acked-by: Stefan Weil <weil@mail.berlios.de>
Signed-off-by: Blue Swirl <blauwirbel@gmail.com>
2011-06-26 20:19:38 +00:00
Michael Tokarev
2caa9e9d2e vnc: tight: Fix crash after 2GB of output
fix 2Gb integer overflow in in VNC tight and zlib encodings

As found by Roland Dreier <roland@purestorage.com> (excellent
catch!), when amount of VNC compressed data produced by zlib
and sent to client exceeds 2Gb, integer overflow occurs because
currently, we calculate amount of data produced at each step by
comparing saved total_out with new total_out, and total_out is
something which grows without bounds.  Compare it with previous
avail_out instead of total_out, and leave total_out alone.

The same code is used in vnc-enc-tight.c and vnc-enc-zlib.c,
so fix both cases.

There, there's no actual need to save previous_out value, since
capacity-offset (which is how that value is calculated) stays
the same so it can be recalculated again after call to deflate(),
but whole thing becomes less readable this way.

Reported-by: Roland Dreier <roland@purestorage.com>
Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>
Signed-off-by: Corentin Chary <corentin.chary@gmail.com>
Signed-off-by: Aurelien Jarno <aurelien@aurel32.net>
2011-04-10 00:14:51 +02:00
Peter Maydell
cf76a1ce8b ui/vnc-enc-tight.c: Fix compile failure if CONFIG_VNC_JPEG not defined
Add some missing #ifdefs to fix compilation failures in the !CONFIG_VNC_JPEG
case introduced by commit ce702e93.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2011-02-24 11:25:38 -06:00
Corentin Chary
80e0c8c39b vnc: add a non-adaptive option
This option allow to disable adaptive behaviors in some encodings.

Signed-off-by: Corentin Chary <corentincj@iksaif.net>
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2011-02-23 16:28:29 -06:00
Corentin Chary
8cb4a6b755 vnc: tight: tweak adaptive tight settings
The force_jpeg threshold was too low.

Signed-off-by: Corentin Chary <corentincj@iksaif.net>
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2011-02-23 16:28:29 -06:00
Corentin Chary
368d25881c vnc: fix uint8_t comparisons with negative values
Signed-off-by: Corentin Chary <corentincj@iksaif.net>
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2011-02-23 16:28:28 -06:00
Corentin Chary
ce702e93b0 vnc: tight: use the update frequency to choose between lossy and lossless
Use the new update frequency infrastructure to use jpeg for regions with
high update frequency.

Signed-off-by: Corentin Chary <corentincj@iksaif.net>
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2011-02-23 16:28:28 -06:00
Stefan Weil
b0cd712cc3 Fix spelling in comments
multifuction -> multifunction
successfull -> successful.

Signed-off-by: Stefan Weil <weil@mail.berlios.de>
2010-10-05 13:53:56 -05:00
Izumi Tsutsui
ba5e7f8216 vnc: use bswapNN() rather than bswap_NN()
bswap_NN() variants are not always available in CONFIG_MACHINE_BSWAP_H case
and bswapNN() are public APIs in "bswap.h".

Signed-off-by: Izumi Tsutsui <tsutsui@ceres.dti.ne.jp>
Signed-off-by: Blue Swirl <blauwirbel@gmail.com>
2010-09-04 10:11:32 +00:00
Serge Ziryukin
49e3fcc249 vnc: tight: remove unused variable
Signed-off-by: Serge Ziryukin <ftrvxmtrx@gmail.com>
Signed-off-by: Blue Swirl <blauwirbel@gmail.com>
2010-09-04 09:55:10 +00:00
Jes Sorensen
2116eff93c size_t is unsigned, change to ssize_t to handle errors from tight_compress_data()
Signed-off-by: Jes Sorensen <Jes.Sorensen@redhat.com>
Signed-off-by: Blue Swirl <blauwirbel@gmail.com>
2010-09-04 09:45:53 +00:00
Blue Swirl
ad7ee4ad6c Initialize a variable in all cases
Commit d167f9bc06 missed this one:
/src/qemu/ui/vnc-enc-tight.c:1483: warning: 'ret' may be used uninitialized in this function

Signed-off-by: Blue Swirl <blauwirbel@gmail.com>
2010-07-31 19:43:37 +00:00
Blue Swirl
aee474ebc6 Fix uint8_t comparison with negative value
Commit 7bccf57383 missed this one:
/src/qemu/ui/vnc-enc-tight.c: In function 'send_sub_rect':
/src/qemu/ui/vnc-enc-tight.c:1527: warning: comparison is always true due to limited range of data type

Signed-off-by: Blue Swirl <blauwirbel@gmail.com>
2010-07-31 19:40:17 +00:00
Blue Swirl
249cdb420a Fix mingw32 build
Fix mingw32 build errors like
/src/qemu/ui/vnc-enc-tight.c: In function 'tight_detect_smooth_image24':
/src/qemu/ui/vnc-enc-tight.c:119: error: 'uint' undeclared (first use in this function)

Replace 'uint' with proper 'unsigned int'.

Signed-off-by: Blue Swirl <blauwirbel@gmail.com>
2010-07-27 17:26:08 +00:00
Blue Swirl
d167f9bc06 Initialize a variable in all cases
Fix a warning with some GCCs:
/src/qemu/ui/vnc-enc-tight.c: In function `send_sub_rect_nojpeg':
/src/qemu/ui/vnc-enc-tight.c:1458: warning: `ret' might be used uninitialized in this function

Signed-off-by: Blue Swirl <blauwirbel@gmail.com>
2010-07-27 15:34:10 +00:00
Blue Swirl
7bccf57383 Fix uint8_t comparisons with negative values
Fix the following warnings:
/src/qemu/hw/ide/core.c: In function `ide_drive_pio_post_load':
/src/qemu/hw/ide/core.c:2767: warning: comparison is always false due to limited range of data type

/src/qemu/ui/vnc-enc-tight.c: In function `tight_detect_smooth_image':
/src/qemu/ui/vnc-enc-tight.c:284: warning: comparison is always true due to limited range of data type
/src/qemu/ui/vnc-enc-tight.c:297: warning: comparison is always true due to limited range of data type
/src/qemu/ui/vnc-enc-tight.c: In function `tight_encode_indexed_rect16':
/src/qemu/ui/vnc-enc-tight.c:456: warning: comparison is always false due to limited range of data type
/src/qemu/ui/vnc-enc-tight.c: In function `tight_encode_indexed_rect32':
/src/qemu/ui/vnc-enc-tight.c:457: warning: comparison is always false due to limited range of data type

Signed-off-by: Blue Swirl <blauwirbel@gmail.com>
2010-07-27 15:32:39 +00:00
Corentin Chary
03817eb874 vnc: tight: split send_sub_rect
Split send_sub_rect in send_sub_rect_jpeg and send_sub_rect_nojpeg to
remove all these #ifdef CONFIG_JPEG.

Signed-off-by: Corentin Chary <corentincj@iksaif.net>
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2010-07-26 17:36:14 -05:00
Corentin Chary
4043a0137b vnc: tight: fix rgb_prepare_row
rgb_prepare_row bpp depends on the server display surface, not
the client.

Signed-off-by: Corentin Chary <corentincj@iksaif.net>
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2010-07-26 17:36:14 -05:00
Corentin Chary
b5469b1104 vnc: fix tight png memory leak
The tight.png buffer was never released.

Signed-off-by: Corentin Chary <corentincj@iksaif.net>
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2010-07-26 17:36:14 -05:00
Corentin Chary
d1af0e056a vnc: encapsulate encoding members
This will allow to implement the threaded VNC server in a
more cleaner way.

Signed-off-by: Corentin Chary <corentincj@iksaif.net>
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2010-07-26 17:36:14 -05:00
Corentin Chary
5136a05269 vnc: tight: stop using qdict for palette stuff
Profiling with callgrind seems to show that a lot of time is spent
in the palette code (mostly due to memory allocation and qdict to int
conversion).

This patch adds a VncPalette implementation. The palette is stored
in a hash table, like qdict, but which does way less memory allocations,
and doesn't suffer from the QObject overhead.

Signed-off-by: Corentin Chary <corentincj@iksaif.net>
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2010-07-26 17:36:14 -05:00
Corentin Chary
3941bf6fb1 vnc: tight: specific zlib level and filters for each compression level
Disable png filters for lower compression levels. This should lower
the CPU consumption and reduce encoding time.

This isn't in tight_conf because:
* tight_conf structure must not change, because it's shared with other
  tight implementations (libvncserver, etc..).
* it'd exceed the 80 col limit.
* PNG_ macros are only defined if CONFIG_VNC_PNG is defined

Signed-off-by: Corentin Chary <corentincj@iksaif.net>
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2010-07-26 17:36:14 -05:00
Corentin Chary
efe556adb7 vnc: tight add PNG encoding
Introduce a new encoding: VNC_ENCODING_TIGHT_PNG [1] (-269) with a new
tight filter VNC_TIGHT_PNG (0x0A). When the client tells it supports the Tight PNG
encoding, the server will use tight, but will always send encoding pixels using
PNG instead of zlib. If the client also told it support JPEG, then the server can
send JPEG, because PNG will only be used in the cases zlib was used in normal tight.

This encoding was introduced to speed up HTML5 based VNC clients like noVNC [2], but
can also be used on devices like iPhone where PNG can be rendered in hardware.

[1] http://wiki.qemu.org/VNC_Tight_PNG
[2] http://github.com/kanaka/noVNC/

Signed-off-by: Corentin Chary <corentincj@iksaif.net>
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2010-07-26 17:36:14 -05:00