Commit Graph

2074 Commits

Author SHA1 Message Date
Wen Congyang
06c3916b35 Backup: don't do copy-on-read in before_write_notifier
We will copy data in before_write_notifier to do backup.
It is a nested I/O request, so we cannot do copy-on-read.

The steps to reproduce it:
1. -drive copy-on-read=on,...  // qemu option
2. drive_backup -f disk0 /path_to_backup.img // monitor command

Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
Tested-by: Jeff Cody <jcody@redhat.com>
Message-id: 1441682913-14320-3-git-send-email-wency@cn.fujitsu.com
Signed-off-by: Jeff Cody <jcody@redhat.com>
2015-09-25 08:37:07 -04:00
Wen Congyang
9568b511c9 block: Introduce a new API bdrv_co_no_copy_on_readv()
In some cases, we need to disable copy-on-read, and just
read the data.

Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
Message-id: 1441682913-14320-2-git-send-email-wency@cn.fujitsu.com
Signed-off-by: Jeff Cody <jcody@redhat.com>
2015-09-25 08:37:07 -04:00
Liu Yuan
4da65c8092 sheepdog: add reopen support
With reopen supported, block-commit (and offline commit) is now supported for
image files whose base image uses the Sheepdog protocol driver.

Cc: qemu-devel@nongnu.org
Cc: Jeff Cody <jcody@redhat.com>
Cc: Kevin Wolf <kwolf@redhat.com>
Cc: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Liu Yuan <liuyuan@cmss.chinamobile.com>
Message-id: 1440730438-24676-1-git-send-email-namei.unix@gmail.com
Signed-off-by: Jeff Cody <jcody@redhat.com>
2015-09-25 08:37:07 -04:00
Peter Lieven
18a8056e0b block/nfs: cache allocated filesize for read-only files
If the file is readonly its not expected to grow so
save the blocking call to nfs_fstat_async and use
the value saved at connection time. Also important
the monitor (and thus the main loop) will not hang
if block device info is queried and the NFS share
is unresponsive.

Signed-off-by: Peter Lieven <pl@kamp.de>
Reviewed-by: Jeff Cody <jcody@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 1440671441-7978-1-git-send-email-pl@kamp.de
Signed-off-by: Jeff Cody <jcody@redhat.com>
2015-09-25 08:37:07 -04:00
Peter Lieven
055c6f912c block/nfs: fix calculation of allocated file size
st.st_blocks is always counted in 512 byte units. Do not
use st.st_blksize as multiplicator which may be larger.

Cc: qemu-stable@nongnu.org
Signed-off-by: Peter Lieven <pl@kamp.de>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Jeff Cody <jcody@redhat.com>
Message-id: 1440067607-14547-1-git-send-email-pl@kamp.de
Signed-off-by: Jeff Cody <jcody@redhat.com>
2015-09-25 08:37:07 -04:00
Peter Maydell
007e620a75 Block layer patches (v2)
-----BEGIN PGP SIGNATURE-----
 Version: GnuPG v2.0.22 (GNU/Linux)
 
 iQIcBAABAgAGBQJV9uA2AAoJEH8JsnLIjy/WPa0P/0tjyUtcp+rTd2yAzC+BQlOA
 dxjq3c3P2HbJKnKa74PwgIBqt7w20TRa8OtMXuJ9XB75iuVRs51dXUDYHUCNvYbW
 dse33PRAUoSYfiaJ3UrsstM5PJH9sDvPHCBekP9CrUa+S9AdcX/7GUiXaiIwB+sj
 X2aur6muwFMK6hIHnUTYypx11/pYYvxVOm5xDMHQWtzbtXHeyVyxJvZkLZzT/DJ2
 1sP3P65Ku0gZQA3rMOnKV6iYhAxrApgAJzhDzPdjKiD7nfxiatIauTvxXhMM2h6Y
 bHHAXAHbf8/kBPbklltwuihXX6/OdMM02S7dU42cPp5TFSPYDLLfRoF34pVy8Ycj
 9BK8H9NNUg/TbHxWv8JLKcuTvk0wv7TDa+zah/Rt7o6jTSn50sxOWnMbj1KbP+IK
 9zkg0hwvUhqDCbkqd1iFYe/5DfVA7eUu5MwhE0Dkncqflmmytw5BZAYFWuPOP4u8
 rH66kg8JFIhLp+H0R3lqSBTezLh8GwMQRTNfrbemiDkA8Pd3GXhNHg3tGPTXK+FS
 4YwUTL2AaJgDRXzz3CpaYh2Pku5t7LsXKRCG3BR7corkhmTBNiHn6V07D6d1kxHa
 cnzsG2gvJqDzELzG3tfsTGkfCtNJrqD0Uj+bB+f7V3K7TiN4RcC2b0Nejn54Jp94
 YZMLP101bpYIPTkVDnRe
 =R3AS
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches (v2)

# gpg: Signature made Mon 14 Sep 2015 15:56:54 BST using RSA key ID C88F2FD6
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>"

* remotes/kevin/tags/for-upstream: (23 commits)
  qcow2: Make qcow2_alloc_bytes() more explicit
  vmdk: Fix next_cluster_sector for compressed write
  iotests: Add test for checking large image files
  qcow2: Make size_to_clusters() return uint64_t
  qemu-iotests: More qcow2 reopen tests
  qemu-iotests: Reopen qcow2 with lazy-refcounts change
  qcow2: Support updating driver-specific options in reopen
  qcow2: Make qcow2_update_options() suitable for transactions
  qcow2: Fix memory leak in qcow2_update_options() error path
  qcow2: Leave s unchanged on qcow2_update_options() failure
  qcow2: Move rest of option handling to qcow2_update_options()
  qcow2: Move qcow2_update_options() call up
  qcow2: Factor out qcow2_update_options()
  qcow2: Improve error message
  qemu-io: Add command 'reopen'
  qemu-io: Remove duplicate 'open' error message
  block: Allow specifying driver-specific options to reopen
  qcow2: Rename BDRVQcowState to BDRVQcow2State
  block: Drop bdrv_find_whitelisted_format()
  block: Drop drv parameter from bdrv_fill_options()
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2015-09-14 18:51:09 +01:00
Peter Maydell
a2aa09e181 * Support for jemalloc
* qemu_mutex_lock_iothread "No such process" fix
 * cutils: qemu_strto* wrappers
 * iohandler.c simplification
 * Many other fixes and misc patches.
 
 And some MTTCG work (with Emilio's fixes squashed):
 * Signal-free TCG kick
 * Removing spinlock in favor of QemuMutex
 * User-mode emulation multi-threading fixes/docs
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v2
 
 iQEcBAABCAAGBQJV8Tk7AAoJEL/70l94x66Ds3QH/3bi0RRR2NtKIXAQrGo5tfuD
 NPMu1K5Hy+/26AC6mEVNRh4kh7dPH5E4NnDGbxet1+osvmpjxAjc2JrxEybhHD0j
 fkpzqynuBN6cA2Gu5GUNoKzxxTmi2RrEYigWDZqCftRXBeO2Hsr1etxJh9UoZw5H
 dgpU3j/n0Q8s08jUJ1o789knZI/ckwL4oXK4u2KhSC7ZTCWhJT7Qr7c0JmiKReaF
 JEYAsKkQhICVKRVmC8NxML8U58O8maBjQ62UN6nQpVaQd0Yo/6cstFTZsRrHMHL3
 7A2Tyg862cMvp+1DOX3Bk02yXA+nxnzLF8kUe0rYo6llqDBDStzqyn1j9R0qeqA=
 =nB06
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into staging

* Support for jemalloc
* qemu_mutex_lock_iothread "No such process" fix
* cutils: qemu_strto* wrappers
* iohandler.c simplification
* Many other fixes and misc patches.

And some MTTCG work (with Emilio's fixes squashed):
* Signal-free TCG kick
* Removing spinlock in favor of QemuMutex
* User-mode emulation multi-threading fixes/docs

# gpg: Signature made Thu 10 Sep 2015 09:03:07 BST using RSA key ID 78C7AE83
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>"
# gpg:                 aka "Paolo Bonzini <pbonzini@redhat.com>"

* remotes/bonzini/tags/for-upstream: (44 commits)
  cutils: work around platform differences in strto{l,ul,ll,ull}
  cpu-exec: fix lock hierarchy for user-mode emulation
  exec: make mmap_lock/mmap_unlock globally available
  tcg: comment on which functions have to be called with mmap_lock held
  tcg: add memory barriers in page_find_alloc accesses
  remove unused spinlock.
  replace spinlock by QemuMutex.
  cpus: remove tcg_halt_cond and tcg_cpu_thread globals
  cpus: protect work list with work_mutex
  scripts/dump-guest-memory.py: fix after RAMBlock change
  configure: Add support for jemalloc
  add macro file for coccinelle
  configure: factor out adding disas configure
  vhost-scsi: fix wrong vhost-scsi firmware path
  checkpatch: remove tests that are not relevant outside the kernel
  checkpatch: adapt some tests to QEMU
  CODING_STYLE: update mixed declaration rules
  qmp: Add example usage of strto*l() qemu wrapper
  cutils: Add qemu_strtoull() wrapper
  cutils: Add qemu_strtoll() wrapper
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2015-09-14 16:13:16 +01:00
Max Reitz
2ac01520be qcow2: Make qcow2_alloc_bytes() more explicit
In case of -EAGAIN returned by update_refcount(), we should discard the
cluster offset we were trying to allocate and request a new one, because
in theory that old offset might now be taken by a refcount block.

In practice, this was not the case due to update_refcount() generally
returning strictly monotonic increasing cluster offsets. However, this
behavior is not set in stone, and it is also not obvious when looking at
qcow2_alloc_bytes() alone, so we should not rely on it.

Reported-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2015-09-14 16:51:37 +02:00
Radoslav Gerganov
3efffc3292 vmdk: Fix next_cluster_sector for compressed write
When the VMDK is streamOptimized (or compressed), the
next_cluster_sector must not be incremented by a fixed number of
sectors. Instead of this, it must be rounded up to the next consecutive
sector. Fixing this results in much smaller compressed images.

Signed-off-by: Radoslav Gerganov <rgerganov@vmware.com>
Reviewed-by: Fam Zheng <famz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2015-09-14 16:51:37 +02:00
Max Reitz
b6d36def6d qcow2: Make size_to_clusters() return uint64_t
Sadly, some images may have more clusters than what can be represented
using a plain int. We should be prepared for that case (in
qcow2_check_refcounts() we actually were trying to catch that case, but
since size_to_clusters() truncated the returned value, that check never
did anything useful).

Cc: qemu-stable <qemu-stable@nongnu.org>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2015-09-14 16:51:37 +02:00
Kevin Wolf
5b0959a7d4 qcow2: Support updating driver-specific options in reopen
For updating the cache sizes, disabling lazy refcounts and updating the
clean_cache_timer there is a bit more to do than just changing the
variables, but otherwise we're all set for changing options during
bdrv_reopen().

Just implement the missing pieces and hook the functions up in
bdrv_reopen().

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
2015-09-14 16:51:37 +02:00
Kevin Wolf
ee55b17304 qcow2: Make qcow2_update_options() suitable for transactions
Before we can allow updating options at runtime with bdrv_reopen(), we
need to split the function into prepare/commit/abort parts.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
2015-09-14 16:51:37 +02:00
Kevin Wolf
c1344ded70 qcow2: Fix memory leak in qcow2_update_options() error path
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
2015-09-14 16:51:36 +02:00
Kevin Wolf
007dbc396c qcow2: Leave s unchanged on qcow2_update_options() failure
On return, either all new options should be applied to BDRVQcowState (on
success), or all of the old settings should be preserved (on failure).

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
2015-09-14 16:51:36 +02:00
Kevin Wolf
94edf3fbe8 qcow2: Move rest of option handling to qcow2_update_options()
With this commit, the handling of driver-specific options in
qcow2_open() is completely separated out into qcow2_update_options().

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
2015-09-14 16:51:36 +02:00
Kevin Wolf
90efa0eaef qcow2: Move qcow2_update_options() call up
qcow2_update_options() only updates some variables in BDRVQcowState and
doesn't really depend on other parts of it being initialised yet, so it
can be moved so that it immediately follows the other half of option
handling code in qcow2_open().

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
2015-09-14 16:51:36 +02:00
Kevin Wolf
4c75d1a157 qcow2: Factor out qcow2_update_options()
Eventually we want to be able to change options at runtime. As a first
step towards that goal, separate some option handling code from the
general initialisation code in qcow2_open().

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
2015-09-14 16:51:36 +02:00
Kevin Wolf
f113ae839e qcow2: Improve error message
Eric says that "any" sounds better than "either", and my non-native
feeling says the same, so let's change it.

Suggested-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
2015-09-14 16:51:36 +02:00
Kevin Wolf
4d2cb09251 block: Allow specifying driver-specific options to reopen
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
2015-09-14 16:51:36 +02:00
Kevin Wolf
ff99129ab8 qcow2: Rename BDRVQcowState to BDRVQcow2State
BDRVQcowState is already used by qcow1, and gdb is always confused which
one to use. Rename the qcow2 one so they can be distinguished.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
2015-09-14 16:51:36 +02:00
Max Reitz
6ebf9aa2ef block: Drop drv parameter from bdrv_open()
Now that this parameter is effectively unused, we can drop it and just
pass NULL on to bdrv_open_inherit().

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2015-09-14 16:51:36 +02:00
Max Reitz
e6641719fe block: Always pass NULL as drv for bdrv_open()
Change all callers of bdrv_open() to pass the driver name in the options
QDict instead of passing its BlockDriver pointer.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2015-09-14 16:51:36 +02:00
Daniel P. Berrange
b6af097528 maint: remove / fix many doubled words
Many source files have doubled words (eg "the the", "to to",
and so on). Most of these can simply be removed, but a couple
were actual mis-spellings (eg "to to" instead of "to do").
There was even one triple word score "to to to" :-)

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>
2015-09-11 10:21:38 +03:00
Daniel P. Berrange
a8f15a2775 maint: remove double semicolons in many files
A number of source files have statements accidentally
terminated by a double semicolon - eg 'foo = bar;;'.
This is harmless but a mistake none the less.

The tcg/ia64/tcg-target.c file is whitelisted because
it has valid use of ';;' in a comment containing assembly
code.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>
2015-09-11 10:21:38 +03:00
Peter Lieven
6d1f252d8c block/iscsi: validate block size returned from target
It has been reported that at least tgtd returns a block size of 0
for LUN 0. To avoid running into divide by zero later on and protect
against other problematic block sizes validate the block size right
at connection time.

Cc: qemu-stable@nongnu.org
Reported-by: Andrey Korolyov <andrey@xdel.ru>
Signed-off-by: Peter Lieven <pl@kamp.de>
Message-Id: <1439552016-8557-1-git-send-email-pl@kamp.de>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2015-09-07 18:14:03 +02:00
Wen Congyang
834cb2ada5 quorum: validate vote threshold against num_children even if read-pattern is fifo
We need to use threshold to check if too many write operation fails.
If threshold is larger than num children, we always get write error
event even if all write operations success.

Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
Message-id: 55962F72.3060003@cn.fujitsu.com
Reviewed-by: Alberto Garcia <berto@igalia.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2015-09-04 21:00:32 +02:00
Alberto Garcia
909c260c71 qcow2: reorder fields in Qcow2CachedTable to reduce padding
Changing the current ordering saves 8 bytes per cache entry in x86_64.

Signed-off-by: Alberto Garcia <berto@igalia.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-id: 0bd55291211df3dfb514d0e7d2031dd5c4f9f807.1438690126.git.berto@igalia.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2015-09-04 21:00:32 +02:00
Alberto Garcia
279621c046 qcow2: add option to clean unused cache entries after some time
This adds a new 'cache-clean-interval' option that cleans all qcow2
cache entries that haven't been used in a certain interval, given in
seconds.

This allows setting a large L2 cache size so it can handle scenarios
with lots of I/O and at the same time use little memory during periods
of inactivity.

This feature currently relies on MADV_DONTNEED to free that memory, so
it is not useful in systems that don't follow that behavior.

Signed-off-by: Alberto Garcia <berto@igalia.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: a70d12da60433df9360ada648b3f34b8f6f354ce.1438690126.git.berto@igalia.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2015-09-04 21:00:32 +02:00
Alberto Garcia
355ee2d0e8 qcow2: mark the memory as no longer needed after qcow2_cache_empty()
After having emptied the cache, the data in the cache tables is no
longer useful, so we can tell the kernel that we are done with it. In
Linux this frees the resources associated with it.

The effect of this can be seen in the HMP commit operation: it moves
data from the top to the base image (and fills both caches), then it
empties the top image. At this point the data in that cache is no
longer needed so it's just wasting memory.

Signed-off-by: Alberto Garcia <berto@igalia.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 08538b098e1faf6c92496477cf9b47a20e5aacea.1438690126.git.berto@igalia.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2015-09-04 21:00:32 +02:00
Max Reitz
bdd03cdf5d block/raw-posix: Use raw_normalize_devicepath()
The filename given to qemu_open() in block/raw-posix.c should generally
have been processed by raw_normalize_devicepath(); unless we are only
probing (in which case the caller often checks whether the file is a
block device or not, and this property will be changed by
raw_normalize_devicepath() on NetBSD) or it is about a deprecated device
(i.e. floppy).

Signed-off-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2015-09-04 20:59:48 +02:00
Wen Congyang
e12f378409 block: more check for replaced node
We use mirror+replace to fix quorum's broken child. bs/s->common.bs
is quorum, and to_replace is the broken child. The new child is target_bs.
Without this patch, the replace node can be any node, and it can be
top BDS with BB, or another quorum's child. We just check if the broken
child is part of the quorum BDS in this patch.

Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
Message-id: 55A86486.1000404@cn.fujitsu.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2015-09-02 14:56:39 +01:00
Kevin Wolf
e424aff5f3 mirror: Fix coroutine reentrance
This fixes a regression introduced by commit dcfb3beb ("mirror: Do zero
write on target if sectors not allocated"), which was reported to cause
aborts with the message "Co-routine re-entered recursively".

The cause for this bug is the following code in mirror_iteration_done():

    if (s->common.busy) {
        qemu_coroutine_enter(s->common.co, NULL);
    }

This has always been ugly because - unlike most places that reenter - it
doesn't have a specific yield that it pairs with, but is more
uncontrolled.  What we really mean here is "reenter the coroutine if
it's in one of the four explicit yields in mirror.c".

This used to be equivalent with s->common.busy because neither
mirror_run() nor mirror_iteration() call any function that could yield.
However since commit dcfb3beb this doesn't hold true any more:
bdrv_get_block_status_above() can yield.

So what happens is that bdrv_get_block_status_above() wants to take a
lock that is already held, so it adds itself to the queue of waiting
coroutines and yields. Instead of being woken up by the unlock function,
however, it gets woken up by mirror_iteration_done(), which is obviously
wrong.

In most cases the code actually happens to cope fairly well with such
cases, but in this specific case, the unlock must already have scheduled
the coroutine for wakeup when mirror_iteration_done() reentered it. And
then the coroutine happened to process the scheduled restarts and tried
to reenter itself recursively.

This patch fixes the problem by pairing the reenter in
mirror_iteration_done() with specific yields instead of abusing
s->common.busy.

Cc: qemu-stable@nongnu.org
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Jeff Cody <jcody@redhat.com>
Message-id: 1439455310-11263-1-git-send-email-kwolf@redhat.com
Signed-off-by: Jeff Cody <jcody@redhat.com>
2015-08-14 09:51:31 -04:00
Stefan Hajnoczi
cae98cb87d block/mirror: limit qiov to IOV_MAX elements
If mirror has more free buffers than IOV_MAX, preadv(2)/pwritev(2)
EINVAL failures may be encountered.

It is possible to trigger this by setting granularity to a low value
like 8192.

This patch stops appending chunks once IOV_MAX is reached.

The spurious EINVAL failure can be reproduced with a qcow2 image file
and the following QMP invocation:

  qmp.command('drive-mirror', device='virtio0', target='/tmp/r7.s1',
              granularity=8192, sync='full', mode='absolute-paths',
              format='raw')

While the guest is running dd if=/dev/zero of=/var/tmp/foo oflag=direct
bs=4k.

Cc: Jeff Cody <jcody@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Message-id: 1435761950-26714-1-git-send-email-stefanha@redhat.com
Signed-off-by: Jeff Cody <jcody@redhat.com>
2015-08-06 04:41:09 -04:00
Sascha Silbe
e94867ed5f block: don't register quorum driver if SHA256 support is unavailable
Commit 488981a4 [block: convert quorum blockdrv to use crypto APIs]
broke qemu-iotest 041 on hosts with GnuTLS < 2.10.0. It converted a
compile-time check to a run-time check at device open time. The result
is that we now advertise a feature (the quorum block driver) that will
never work (on those hosts). There's no way (short of parsing
human-readable error messages) for qemu-iotests or any other API
consumer to recognise that the quorum block driver isn't _actually_
available and shouldn't be used or tested.

Move the run-time check to bdrv_quorum_init() to avoid registering the
quorum block driver if we know it cannot work. This way API consumers
can recognise it's unavailable.

Fixes: 488981a4af
Signed-off-by: Sascha Silbe <silbe@linux.vnet.ibm.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Daniel P. Berrange <berrange@redhat.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Message-id: 1438699705-21761-1-git-send-email-silbe@linux.vnet.ibm.com
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2015-08-05 15:19:32 +01:00
Peter Maydell
9f8c5b69c2 -----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
 
 iQIcBAABAgAGBQJVtwOFAAoJEL2+eyfA3jBXeN8QAJAuQL5Wvwe1epGhiOIAVadX
 Zx0Yx8DnYbHH184tsEOyuliHUxSN6X6bBlO1rTaI/tpEOeZEBY9EBUBDdcty//Y4
 ZHCKDppYSP773wJDLIvHn7gw83Z+zqgRw1WXcyd4piBKiilJ5c2wvpDrTNqwFh0I
 1mqTFoVOARK0OVlxuXdhaH+hBI823MaPaSRckOWYgZjRZN2HlCAN8sDigdQtqBGp
 Udugw4TijlMi5/JmLUmvDHLaVZz2EEKlH7fHjInU98Z2p5UE/dkTI2QIj2smTSEk
 XpUkmukHgLHyIxDnE2pAhdQG12RTd8HrnkmohauHXsvVzQTvW6tOBfR57sqVSy1g
 WPxEiAgvs5kWnF7LLmR4AO7xd+r9/EFrNsMpdinKjA68RAnxEfFcCriNMPtPtv2h
 g/Nw60uqrSiv98/JL7ApcqwcevJOteBY50D//hkpyevlyPkh4hg0vISXfaAaDFrp
 YC0byq32UtkWZJkgAAj+ZkGJOdeRGl+Ma8+2FwxHAaZepCMGJC2xRAQ28CEHUEfP
 WAM/VRirgmxV4kG2JTnotlWqj6z0iYdjiL+AV+Mq0cywkboOQoyaOoIGC30kovGZ
 XMcEDLTr1wke/ODU7JegOX+SPLU+Nf4zXz6QJMaIRlAGO+EpQiJoHjH0PWvjAoqV
 xcWr+wXm9H0ejX5Rx1is
 =9QFT
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/cody/tags/jtc-for-upstream-pull-request' into staging

# gpg: Signature made Tue Jul 28 05:22:29 2015 BST using RSA key ID C0DE3057
# gpg: Good signature from "Jeffrey Cody <jcody@redhat.com>"
# gpg:                 aka "Jeffrey Cody <jeff@codyprime.org>"
# gpg:                 aka "Jeffrey Cody <codyprime@gmail.com>"
# gpg: WARNING: This key is not certified with sufficiently trusted signatures!
# gpg:          It is not certain that the signature belongs to the owner.
# Primary key fingerprint: 9957 4B4D 3474 90E7 9D98  D624 BDBE 7B27 C0DE 3057

* remotes/cody/tags/jtc-for-upstream-pull-request:
  block/ssh: Avoid segfault if inet_connect doesn't set errno.
  sheepdog: serialize requests to overwrapping area

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2015-07-28 13:22:57 +01:00
Richard W.M. Jones
325e390421 block/ssh: Avoid segfault if inet_connect doesn't set errno.
On some (but not all) systems:

  $ qemu-img create -f qcow2 overlay -b ssh://xen/
  Segmentation fault

It turns out this happens when inet_connect returns -1 in the
following code, but errno == 0.

  s->sock = inet_connect(s->hostport, errp);
  if (s->sock < 0) {
      ret = -errno;
      goto err;
  }

In the test case above, no host called "xen" exists, so getaddrinfo fails.

On Fedora 22, getaddrinfo happens to set errno = ENOENT (although it
is *not* documented to do that), so it doesn't segfault.

On RHEL 7, errno is not set by the failing getaddrinfo, so ret =
-errno = 0, so the caller doesn't know there was an error and
continues with a half-initialized BDRVSSHState struct, and everything
goes south from there, eventually resulting in a segfault.

Fix this by setting ret to -EIO (same as block/nbd.c and
block/sheepdog.c).  The real error is saved in the Error** errp
struct, so it is printed correctly:

  $ ./qemu-img create -f qcow2 overlay -b ssh://xen/
  qemu-img: overlay: address resolution failed for xen:22: No address associated with hostname

Signed-off-by: Richard W.M. Jones <rjones@redhat.com>
Reported-by: Jun Li
BZ: https://bugzilla.redhat.com/show_bug.cgi?id=1147343
Signed-off-by: Jeff Cody <jcody@redhat.com>
2015-07-28 00:19:05 -04:00
Hitoshi Mitake
6a55c82cec sheepdog: serialize requests to overwrapping area
Current sheepdog driver only serializes create requests in oid
unit. This mechanism isn't enough for handling requests to
overwrapping area spanning multiple oids, so it can result bugs like
below:
https://bugs.launchpad.net/sheepdog-project/+bug/1456421

This patch adds a new serialization mechanism for the problem. The
difference from the old one is:
1. serialize entire aiocb if their targetting areas overwrap
2. serialize all requests (read, write, and discard), not only creates

This patch also removes the old mechanism because the new one can be
an alternative.

Cc: Kevin Wolf <kwolf@redhat.com>
Cc: Stefan Hajnoczi <stefanha@redhat.com>
Cc: Teruaki Ishizaki <ishizaki.teruaki@lab.ntt.co.jp>
Cc: Vasiliy Tolstov <v.tolstov@selfip.ru>
Signed-off-by: Hitoshi Mitake <mitake.hitoshi@lab.ntt.co.jp>
Tested-by: Vasiliy Tolstov <v.tolstov@selfip.ru>
Signed-off-by: Jeff Cody <jcody@redhat.com>
2015-07-28 00:16:57 -04:00
Jeff Cody
b15deac795 block: vpc - prevent overflow if max_table_entries >= 0x40000000
When we allocate the pagetable based on max_table_entries, we multiply
the max table entry value by 4 to accomodate a table of 32-bit integers.
However, max_table_entries is a uint32_t, and the VPC driver accepts
ranges for that entry over 0x40000000.  So during this allocation:

s->pagetable = qemu_try_blockalign(bs->file, s->max_table_entries * 4);

The size arg overflows, allocating significantly less memory than
expected.

Since qemu_try_blockalign() size argument is size_t, cast the
multiplication correctly to prevent overflow.

The value of "max_table_entries * 4" is used elsewhere in the code as
well, so store the correct value for use in all those cases.

We also check the Max Tables Entries value, to make sure that it is <
SIZE_MAX / 4, so we know the pagetable size will fit in size_t.

Cc: qemu-stable@nongnu.org
Reported-by: Richard W.M. Jones <rjones@redhat.com>
Signed-off-by: Jeff Cody <jcody@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2015-07-27 17:19:06 +02:00
Fam Zheng
9990069758 mirror: Speed up bitmap initial scanning
Limiting to sectors_per_chunk for each bdrv_is_allocated_above is slow,
because the underlying protocol driver would issue much more queries
than necessary. We should coalesce the query.

Signed-off-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: <1436413678-7114-4-git-send-email-famz@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2015-07-22 11:14:21 +01:00
Richard W.M. Jones
796a060bc0 block/curl: Don't lose original error when a connection fails.
Currently if qemu is connected to a curl source (eg. web server), and
the web server fails / times out / dies, you always see a bogus EIO
"Input/output error".

For example, choose a large file located on any local webserver which
you control:

  $ qemu-img convert -p http://example.com/large.iso /tmp/test

Once it starts copying the file, stop the webserver and you will see
qemu-img fail with:

  qemu-img: error while reading sector 61440: Input/output error

This patch does two things: Firstly print the actual error from curl
so it doesn't get lost.  Secondly, change EIO to EPROTO.  EPROTO is a
POSIX.1 compatible errno which more accurately reflects that there was
a protocol error, rather than some kind of hardware failure.

After this patch is applied, the error changes to:

  $ qemu-img convert -p http://example.com/large.iso /tmp/test
  qemu-img: curl: transfer closed with 469989 bytes remaining to read
  qemu-img: error while reading sector 16384: Protocol error

Signed-off-by: Richard W.M. Jones <rjones@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Jeff Cody <jcody@redhat.com>
2015-07-14 21:50:13 -04:00
Wen Congyang
48ac0a4df8 mirror: correct buf_size
If bus_size is less than 0, the command fails.
If buf_size is 0, use DEFAULT_MIRROR_BUF_SIZE.
If buf_size % granularity is not 0, mirror_free_init() will
do dangerous things.

Signed-off-by: Wen Congyang <wency@cn.fujitsu.com>
Reviewed-by: Fam Zheng <famz@redhat.com>
Message-id: 5555A588.3080907@cn.fujitsu.com
Signed-off-by: Jeff Cody <jcody@redhat.com>
2015-07-14 21:50:13 -04:00
Stefan Hajnoczi
17d9716d7b block: keep bitmap if incremental backup job is cancelled
Reclaim the dirty bitmap if an incremental backup block job is
cancelled.  The ret variable may be 0 when the job is cancelled so it's
not enough to check ret < 0.

Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 1434380534-7680-1-git-send-email-stefanha@redhat.com
Signed-off-by: Jeff Cody <jcody@redhat.com>
2015-07-14 21:50:13 -04:00
Fam Zheng
4c0cbd6fec block/mirror: Sleep periodically during bitmap scanning
Before, we only yield after initializing dirty bitmap, where the QMP
command would return. That may take very long, and guest IO will be
blocked.

Add sleep points like the later mirror iterations.

Signed-off-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Wen Congyang <wency@cn.fujitsu.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 1431486673-19280-1-git-send-email-famz@redhat.com
Signed-off-by: Jeff Cody <jcody@redhat.com>
2015-07-14 21:50:13 -04:00
Josh Durgin
e34d8f297d rbd: fix ceph settings precedence
Apply the ceph settings from a config file before any ceph settings
from the command line. Since the ceph config file location may be
specified on the command line, parse it once to read the config file,
and do a second pass to apply the rest of the command line ceph
options.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2015-07-14 17:15:23 +02:00
Josh Durgin
99a3c89d5d rbd: make qemu's cache setting override any ceph setting
To be safe, when cache=none is used ceph settings should not be able
to override it to turn on caching. This was previously possible with
rbd_cache=true in the rbd device configuration or a ceph configuration
file. Similarly, rbd settings could have turned off caching when qemu
requested it, although this would just be a performance problem.

Fix this by changing rbd's cache setting to match qemu after all other
ceph settings have been applied.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Jeff Cody <jcody@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2015-07-14 17:15:23 +02:00
Josh Durgin
3dbf00e058 rbd: remove unused constants and fields
RBDAIOCB.status was only used for cancel, which was removed in
7691e24dbe.

RBDAIOCB.sector_num was never used.

RADOSCB.done and rcbid were never used.

RBD_FD* are obsolete since the pipe was removed in
e04fb07fd1.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Jeff Cody <jcody@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2015-07-14 17:15:23 +02:00
Peter Maydell
acf7b7fdf3 Bugfixes and Daniel Berrange's crypto library.
-----BEGIN PGP SIGNATURE-----
 Version: GnuPG v2
 
 iQEcBAABCAAGBQJVnQWdAAoJEL/70l94x66D6OgIAKJlzQfmy5w7Q9WD4vCMhD76
 JrpLSsn7Gx/Bws0Nu9nLQlqun5z4hiUxyG2kP/WqD9+tV3cpSMSyrG6ImVdqKnQ5
 +Z8WJZuREkQv0aqDUjQVST+eIDZuh2LWJXAjhgsCXUHY77eWb/7WmKT79xJOa+5C
 5xB1qxudqX5IsTvpiKKPbmUGYkAcvRX1dUSaFwRIMO0UyKn59B9WfM9a5slIbLW7
 XfI8+wEJshTVLuQkkTfdidWQc5M5DwlmO7ESUNR/BRPCPFeyjcDqgQY5pBM5XVo9
 C+S0R3zIt3Ew0fhCtLRyjlIT0bGfwjbU5HRiHcyldBKhNUZZjSUoOWJnYRHXUDY=
 =H8wA
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into staging

Bugfixes and Daniel Berrange's crypto library.

# gpg: Signature made Wed Jul  8 12:12:29 2015 BST using RSA key ID 78C7AE83
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>"
# gpg:                 aka "Paolo Bonzini <pbonzini@redhat.com>"
# gpg: WARNING: This key is not certified with sufficiently trusted signatures!
# gpg:          It is not certain that the signature belongs to the owner.
# Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4  E2F7 7E15 100C CD36 69B1
#      Subkey fingerprint: F133 3857 4B66 2389 866C  7682 BFFB D25F 78C7 AE83

* remotes/bonzini/tags/for-upstream:
  ossaudio: fix memory leak
  ui: convert VNC to use generic cipher API
  block: convert qcow/qcow2 to use generic cipher API
  ui: convert VNC websockets to use crypto APIs
  block: convert quorum blockdrv to use crypto APIs
  crypto: add a nettle cipher implementation
  crypto: add a gcrypt cipher implementation
  crypto: introduce generic cipher API & built-in implementation
  crypto: move built-in D3DES implementation into crypto/
  crypto: move built-in AES implementation into crypto/
  crypto: introduce new module for computing hash digests
  vl: move rom_load_all after machine init done

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2015-07-08 20:46:35 +01:00
Daniel P. Berrange
f6fa64f6d2 block: convert qcow/qcow2 to use generic cipher API
Switch the qcow/qcow2 block driver over to use the generic cipher
API, this allows it to use the pluggable AES implementations,
instead of being hardcoded to use QEMU's built-in impl.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Message-Id: <1435770638-25715-10-git-send-email-berrange@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2015-07-08 13:11:01 +02:00
Daniel P. Berrange
488981a4af block: convert quorum blockdrv to use crypto APIs
Get rid of direct use of gnutls APIs in quorum blockdrv in
favour of using the crypto APIs. This avoids the need to
do conditional compilation of the quorum driver. It can
simply report an error at file open file instead if the
required hash algorithm isn't supported by QEMU.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Message-Id: <1435770638-25715-8-git-send-email-berrange@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2015-07-08 13:11:01 +02:00
Ting Wang
970311646a blockjob: add block_job_release function
There is job resource leak in function mirror_start_job,
although bdrv_create_dirty_bitmap is unlikely failed.
Add block_job_release for each release when needed.

Signed-off-by: Ting Wang <kathy.wangting@huawei.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-id: 1435311455-56048-1-git-send-email-kathy.wangting@huawei.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2015-07-07 14:27:14 +01:00