Commit Graph

3743 Commits

Author SHA1 Message Date
Peter Maydell
c8b7e627b4 nbd patches for 2018-05-04
- Vladimir Sementsov-Ogievskiy: 0/2 fix coverity bugs
 - Eric Blake: nbd/client: Fix error messages during NBD_INFO_BLOCK_SIZE
 - Eric Blake: nbd/client: Relax handling of large NBD_CMD_BLOCK_STATUS reply
 -----BEGIN PGP SIGNATURE-----
 Comment: Public key at http://people.redhat.com/eblake/eblake.gpg
 
 iQEcBAABCAAGBQJa7F9jAAoJEKeha0olJ0NqwZIH/3LbaF7Q0CcuB6d+nQo3jYm2
 fGIb8pV4pZLgC4D4qrelXP2Ttn0RNMLNdq2UR6F66/MAFj2/sF4gRE/p82exZRK3
 be2hDQpzKOTgrF7SQN9ccI1df7nrMhvXgo1Y4rhSFZKtMTBPZirDFdAP/2xklSzI
 jMHE/Iq9Ng16303FR5KEiVtmAWxAaagapcvEKIrD0SpoiAX9jk6dAT5EwOHLi4Lf
 2CCe/5iBFC96zLE2xJ8n+esZ1chJJp/2gubmYON/lwLx5fXqYVowywDVNzv+uc8A
 sg2VMjb/UySOLJ6IxdxgGdln0w46RB7u55nRnyH6LcU2IdBeladhkI7Oh/reOTI=
 =wnw9
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2018-05-04' into staging

nbd patches for 2018-05-04

- Vladimir Sementsov-Ogievskiy: 0/2 fix coverity bugs
- Eric Blake: nbd/client: Fix error messages during NBD_INFO_BLOCK_SIZE
- Eric Blake: nbd/client: Relax handling of large NBD_CMD_BLOCK_STATUS reply

# gpg: Signature made Fri 04 May 2018 14:25:55 BST
# gpg:                using RSA key A7A16B4A2527436A
# gpg: Good signature from "Eric Blake <eblake@redhat.com>"
# gpg:                 aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>"
# gpg:                 aka "[jpeg image of size 6874]"
# Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2  F3AA A7A1 6B4A 2527 436A

* remotes/ericb/tags/pull-nbd-2018-05-04:
  nbd/client: Relax handling of large NBD_CMD_BLOCK_STATUS reply
  nbd/client: Fix error messages during NBD_INFO_BLOCK_SIZE
  migration/block-dirty-bitmap: fix memory leak in dirty_bitmap_load_bits
  nbd/client: fix nbd_negotiate_simple_meta_context

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-05-04 14:42:46 +01:00
Eric Blake
acfd8f7a5f nbd/client: Relax handling of large NBD_CMD_BLOCK_STATUS reply
The NBD spec is proposing a relaxation of NBD_CMD_BLOCK_STATUS
where a server may have the final extent per context give a
length beyond the original request, if it can easily prove that
subsequent bytes have the same status, on the grounds that a
client can take advantage of this information for fewer block
status requests.  Since qemu 2.12 as a client always sends
NBD_CMD_FLAG_REQ_ONE, and rejects a server that sends extra
length, the upstream NBD spec will probably limit this behavior
to clients that don't request REQ_ONE semantics; but it doesn't
hurt to relax qemu to always be permissive of this server
behavior, even if it continues to use REQ_ONE.

CC: qemu-stable@nongnu.org
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180503222626.1303410-1-eblake@redhat.com>
Reviewed-by:  Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2018-05-04 08:23:39 -05:00
Marc-André Lureau
f5a74a5a50 qobject: Modify qobject_ref() to return obj
For convenience and clarity, make it possible to call qobject_ref() at
the time when the reference is associated with a variable, or
argument, by making qobject_ref() return the same pointer as given.
Use that to simplify the callers.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180419150145.24795-5-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Useless change to qobject_ref_impl() dropped, commit message improved
slightly]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-05-04 08:27:53 +02:00
Marc-André Lureau
cb3e7f08ae qobject: Replace qobject_incref/QINCREF qobject_decref/QDECREF
Now that we can safely call QOBJECT() on QObject * as well as its
subtypes, we can have macros qobject_ref() / qobject_unref() that work
everywhere instead of having to use QINCREF() / QDECREF() for QObject
and qobject_incref() / qobject_decref() for its subtypes.

The replacement is mechanical, except I broke a long line, and added a
cast in monitor_qmp_cleanup_req_queue_locked().  Unlike
qobject_decref(), qobject_unref() doesn't accept void *.

Note that the new macros evaluate their argument exactly once, thus no
need to shout them.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180419150145.24795-4-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Rebased, semantic conflict resolved, commit message improved]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2018-05-04 08:27:53 +02:00
Vladimir Sementsov-Ogievskiy
605bc8be42 qcow2: try load bitmaps only once
Checking reopen by existence of some bitmaps is wrong, as it may be
some other bitmaps, or on the other hand, user may remove bitmaps. This
criteria is bad. To simplify things and make behavior more predictable
let's just add a flag to remember, that we've already tried to load
bitmaps on open and do not want do it again.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-id: 20180411122606.367301-2-vsementsov@virtuozzo.com
[mreitz: Changed comment wording according to Eric Blake's suggestion]
Signed-off-by: Max Reitz <mreitz@redhat.com>
2018-04-16 13:35:32 +02:00
Kevin Wolf
2fe4bba19b commit/stream: Reset delay_ns
Streaming and the commit block job only want to apply throttling when
they actually copied data instead of skipping it, so they made the
calculation of delay_ns conditional. However, delay_ns isn't reset when
skipping some sectors, so instead of not waiting, the old delay is
applied again.

Properly reset delay_ns where needed.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2018-04-10 16:33:47 +02:00
Jeff Cody
bfb15b4bec block/rbd: remove processed options from qdict
Commit 4bfb274 added some QAPIfication of option parsing in
qemu_rbd_open().  We need to remove all the options we processed,
otherwise in bdrv_open_inherit() we will think the remaining options are
invalid.

(This needs to go in 2.12 to avoid a regression that prevents rbd
from being opened.)

Suggested-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Jeff Cody <jcody@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
2018-04-04 12:05:13 -04:00
Peter Maydell
fd69ad866b Block layer patches
-----BEGIN PGP SIGNATURE-----
 
 iQIcBAABAgAGBQJaw6JlAAoJEH8JsnLIjy/W8JcP/06bxQ+8056Vz/+oRAm3VEZN
 iFJ1w+A1cFVpaqeE1fl+nfluyT+bU4J8YIyeHQlgckFyzQz13v0cSSSpBJYfJAO9
 XSO/g96kILwN+h16NNdKX2K7uewVPZgyDbNf+FzM2UHuSSXdSp0ZbeW1yXvB6cpx
 hk2exA0Vfxm2dq2Xzz00EzGCWdZAnWWesj9pimASQ1W0426t7PJRCrRaUVawaJfT
 J+IB4zsF4BJ49CvqmeFqyPfDV9q1GF3c9EkXzFgw7qYY2C4F4bAKv82AxWiKEiRI
 cm1bHEdVyDFuBxzMuGOhZS3g2YkJDnEbkfJ0zj5cKmmYO2Pc0q8DeAMZTvy3aobL
 R5UawZTgPv2O71hOlRFl/YZwMjhb80BXDq9mx7TrrhQ5/wCAUOs4e1XYHn+kzL3W
 ohp5r74Qj1s84pGi8sPaXWm8myxalkTicT1cKFHztb+3+DwVPQIOXhMkMqrJyAK8
 3GwZsGmtzI5DCIyF3whLbElN/OHRM9zr/T+uwhRQw8bs/XYPU27f367Y+JJGAaZq
 xTD7vlF49XSOoSDTYcvJfODlloJODyL+CxFS/vpJKfMYgDWYQZiwvYeIEywmwIYE
 2bEdgcN/9BOdYK4NRuJzDY+Vra7iVnDvXhsZ8UClp/SPLspJ54zwCny9f5rSIDJF
 NSiBcahA5femreYqAfWz
 =quyk
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches

# gpg: Signature made Tue 03 Apr 2018 16:48:53 BST
# gpg:                using RSA key 7F09B272C88F2FD6
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>"
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74  56FE 7F09 B272 C88F 2FD6

* remotes/kevin/tags/for-upstream:
  iotests: Test abnormally large size in compressed cluster descriptor
  qemu-iotests: Use ppc64 qemu_arch on ppc64le host
  iotests: Test preallocated truncate of 2G image
  block/file-posix: Fix fully preallocated truncate
  iotests: fix 208 for luks format
  iotests: Update 186 after commit ac64273c66
  iotests: Update 051 and 186 after commit 1454509726
  block: handle invalid lseek returns gracefully
  gluster: Fix blockdev-add with server.N.type=unix

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-04-04 14:00:07 +01:00
Peter Maydell
e5efa1f5f2 -----BEGIN PGP SIGNATURE-----
iQIcBAABAgAGBQJaw6duAAoJEL2+eyfA3jBXoYUP/07v6FQ0E91eprL2Zg2uowA1
 u6e/neVhlzZgMpOevWfnjScBYHA5wjPjM2ZbKckbnFjH60s7gVBuYtbnwcChRMY1
 LXJts1LivuQOru5QIf0Fwi2W4vkPzV8rQsWp6nmnIDf/Jzs2iKEe6jy6A2enSLYM
 FKOrJc1ViNWFpW1y/7mV8gHIAChEC3wEL9L2JUQEvi4VQD7uCwJv0YS5Ci+xjV21
 /DPaVfZA4VMNNi+FTRezW+l5m0zOxenxUnDXqpP5yjgPyTpyqfHSCCVK54HLQ2L/
 8uMblucjRRFXl6zsX2Quzwqiq80D9KxidhOKk9OU1Yi+rbDz4vSTlE/fBRHKVVln
 Rd8DhTK7aPG1bsiQSHCqcd4+a9qsHOn0LY7tS1m78CaVx1O8JhMBe5Y+NJEMR8AM
 iiWss8QyXggDkagzyiGrMHguCI5yM0EWqYfWtjLBiKOkzTjn9xzSbAbn9JK9TBBa
 aZ6JwNE1krtsTydJ1K37KFNabX/yyZR1KlYUc90FwRC1b9pfPHCmaMQ2imx/pg/y
 f3p3uG1UQmBXMMgchstqc1Xf4BqEeOV5pBKlF2Dwodi1ewZoS+WtuvqjURs2aJUr
 2YLOGPBtPHboWl+kjmn+csJJ8zSWw9byRyHHKJN7zFEx3iXZ4I9HMauzap+goWJq
 2NCKYjzyWkuR7YLR4w3q
 =vLaJ
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/cody/tags/block-pull-request' into staging

# gpg: Signature made Tue 03 Apr 2018 17:10:22 BST
# gpg:                using RSA key BDBE7B27C0DE3057
# gpg: Good signature from "Jeffrey Cody <jcody@redhat.com>"
# gpg:                 aka "Jeffrey Cody <jeff@codyprime.org>"
# gpg:                 aka "Jeffrey Cody <codyprime@gmail.com>"
# Primary key fingerprint: 9957 4B4D 3474 90E7 9D98  D624 BDBE 7B27 C0DE 3057

* remotes/cody/tags/block-pull-request:
  gluster: Fix blockdev-add with server.N.type=unix
  blockjob: use qapi enum helpers
  blockjob: leak fix, remove from txn when failing early

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-04-04 12:33:23 +01:00
Max Reitz
82b45e0a0b block/file-posix: Fix fully preallocated truncate
Storing the lseek() result in an int results in it overflowing when the
file is at least 2 GB big.  Then, we have a 50 % chance of the result
being "negative" and thus thinking an error occurred when actually
everything went just fine.

So we should use the correct type for storing the result: off_t.

Reported-by: Daniel P. Berrange <berrange@redhat.com>
Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=1549231
Cc: qemu-stable@nongnu.org
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id: 20180228131315.30194-2-mreitz@redhat.com
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2018-04-03 17:39:37 +02:00
Kevin Wolf
9dae635afa gluster: Fix blockdev-add with server.N.type=unix
The legacy command line interface gets the socket path from an option
called 'socket'. QAPI in contract uses SocketAddress, where the
corresponding option is called 'path'.

Fix the gluster block driver to accept both 'socket' and 'path', with
'path' being the preferred syntax.

https://bugzilla.redhat.com/show_bug.cgi?id=1545155

Cc: qemu-stable@nongnu.org
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-id: 20180403110810.25624-1-kwolf@redhat.com
Signed-off-by: Jeff Cody <jcody@redhat.com>
2018-04-03 09:57:14 -04:00
Jeff Cody
a03083a017 block: handle invalid lseek returns gracefully
In commit 223a23c198, we implemented a
workaround in the gluster driver to handle invalid values returned for
SEEK_DATA or SEEK_HOLE.

In some instances, these same invalid values can be seen in the posix
file handler as well - for example, it has been reported on FUSE gluster
mounts.

Calling assert() for these invalid values is overly harsh; we can safely
return -EIO and allow this case to be treated as a "learned nothing"
case (e.g., D4 / H4, as commented in the code).

This patch does the same thing that 223a23c198 did for gluster.c,
except in file-posix.c

Signed-off-by: Jeff Cody <jcody@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-04-03 15:25:17 +02:00
Kevin Wolf
3e4d88eabf gluster: Fix blockdev-add with server.N.type=unix
The legacy command line interface gets the socket path from an option
called 'socket'. QAPI in contract uses SocketAddress, where the
corresponding option is called 'path'.

Fix the gluster block driver to accept both 'socket' and 'path', with
'path' being the preferred syntax.

https://bugzilla.redhat.com/show_bug.cgi?id=1545155

Cc: qemu-stable@nongnu.org
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2018-04-03 15:20:36 +02:00
Eric Blake
00d96a4612 nbd: Fix 32-bit compilation on BLOCK_STATUS
iotests 123 and 209 fail on 32-bit platforms.  The culprit:
sizeof(extent) is wrong; we want sizeof(*extent).  But since
the struct is 8 bytes, it happened to work on 64-bit platforms
where the pointer is also 8 bytes (nasty).

Fixes: 78a33ab58
Reported-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180327210517.1804242-1-eblake@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2018-04-02 08:45:21 -05:00
Peter Maydell
f55e88f2ab qapi patches for 2018-03-27, 2.12-rc1
- Marc-André Lureau: qmp-test: fix response leak
 - Eric Blake: tests: Silence false positive warning on generated test name
 - Laurent Vivier: 0/4 (partial) coccinelle: re-run scripts from scripst/coccinelle
 - Peter Xu: 0/8 Monitor: some oob related patches (fixes, new param, tests)
 - Satheesh Rajendran: hmp.c: Revert hmp_info_cpus output format change
 -----BEGIN PGP SIGNATURE-----
 Comment: Public key at http://people.redhat.com/eblake/eblake.gpg
 
 iQEcBAABCAAGBQJaumDMAAoJEKeha0olJ0NqY34IAJ1AJLZeqtb9HIPDrL/iydXQ
 qrMPxfz/fbcjMbomYvltvCYB3GMQw88YVtTuu2kS5aZsr6vLhpw2ZjGrb+a0llFM
 lF6oERg/6VmfQNaeqcHEzORIdG6/GEctxoWS07f+NgSrjdxNNHyU+i/LGvvA5K4m
 x7akPw3DZIF/oBLDlFgtl80KtOf8vQ8QitCIp2nsURQFLHvjVSoXHGUfhoiUUgNA
 89rF62Repq9xZXSsFr+Jpp4/bPGxA717sixXSxFUDuTVIZGnffbt4wVxLi4TKSx1
 1Ho6psPK3d82rQ9OeIc6CqulElLMPGn7aNwR1VR6ig0sB409K5PQJybJDqQD/C4=
 =ld/c
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/ericb/tags/pull-qapi-2018-03-27-v2' into staging

qapi patches for 2018-03-27, 2.12-rc1

- Marc-André Lureau: qmp-test: fix response leak
- Eric Blake: tests: Silence false positive warning on generated test name
- Laurent Vivier: 0/4 (partial) coccinelle: re-run scripts from scripst/coccinelle
- Peter Xu: 0/8 Monitor: some oob related patches (fixes, new param, tests)
- Satheesh Rajendran: hmp.c: Revert hmp_info_cpus output format change

# gpg: Signature made Tue 27 Mar 2018 16:18:36 BST
# gpg:                using RSA key A7A16B4A2527436A
# gpg: Good signature from "Eric Blake <eblake@redhat.com>"
# gpg:                 aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>"
# gpg:                 aka "[jpeg image of size 6874]"
# Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2  F3AA A7A1 6B4A 2527 436A

* remotes/ericb/tags/pull-qapi-2018-03-27-v2:
  hmp.c: Revert hmp_info_cpus output format change
  tests: qmp-test: add test for new "x-oob"
  tests: Add parameter to qtest_init_without_qmp_handshake
  monitor: new parameter "x-oob"
  qmp: cleanup qmp queues properly
  tests: add oob-test for qapi-schema
  tests: let qapi-schema tests detect oob
  qapi: restrict allow-oob value to be "true"
  qmp: fix qmp_capabilities error regression
  qdict: remove useless cast
  error: Remove NULL checks on error_propagate() calls
  error: Strip trailing '\n' from error string arguments (again again)
  tests: Silence false positive warning on generated test name
  qmp-test: fix response leak

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-03-27 19:20:57 +01:00
Peter Maydell
6cf38cbf29 -----BEGIN PGP SIGNATURE-----
iQEcBAABAgAGBQJaulgHAAoJEJykq7OBq3PI0owH/3isnzyO8efBzlv4HlWb1qV0
 F7m9urAKZm1QLCkJBlli/G5iNaNUfrXSum4qYK22XAiwV5WAoPAjwrOvtTRRTSfW
 vFGAucmf6NKkomuZRAiPQftqR+DL9VhBC7mcdRwfwWZ82AGZZnAQsDf8y9e3kal2
 YPwogmvQi+bn2Psv6q3zYtrTW9bC1VJWzYFP4xcSpcxGOY/WfLhPX9nrhjQz07mm
 h/y9mWqXA8PytmopMLj31xRmhTSndIwo+uvScLH95UXRuAOhjyHMvvD4HfI4nkR+
 WD94pE6UiUpIdCZ6fh8H/jaCpN8jPnXI4XJg9cQcRHZYkOrHwQ8G1xxoOintjW0=
 =55b0
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' into staging

# gpg: Signature made Tue 27 Mar 2018 15:41:11 BST
# gpg:                using RSA key 9CA4ABB381AB73C8
# gpg: Good signature from "Stefan Hajnoczi <stefanha@redhat.com>"
# gpg:                 aka "Stefan Hajnoczi <stefanha@gmail.com>"
# Primary key fingerprint: 8695 A8BF D3F9 7CDA AC35  775A 9CA4 ABB3 81AB 73C8

* remotes/stefanha/tags/block-pull-request:
  MAINTAINERS: add include/block/aio-wait.h
  coroutine: add test-aio coroutine queue chaining test case
  coroutine: avoid co_queue_wakeup recursion
  queue: add QSIMPLEQ_PREPEND()

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-03-27 17:11:33 +01:00
Laurent Vivier
625eaca9e5 qdict: remove useless cast
Re-run Coccinelle script scripts/coccinelle/qobject.cocci

Signed-off-by: Laurent Vivier <lvivier@redhat.com>
Message-Id: <20180323143202.28879-5-lvivier@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Acked-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Acked-by: Fam Zheng <famz@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-27 10:17:32 -05:00
Peter Maydell
bdc408e91b A fix for dirty bitmap migration through shared storage, and a VMDK
patch keeping us from creating too large extents.
 -----BEGIN PGP SIGNATURE-----
 
 iQEcBAABAgAGBQJauVVBAAoJEPQH2wBh1c9AGtYIAJ1ojl6+guKQHjtzv9W8ch50
 2tzH9/lFkhq/Tyay8MCcq1dWQr23tKqusxi6fDHVdc8XIx6fDuPFzWxUQwwLvS81
 9CA6Qs2NngkiAs89ZZ0Sc9aj+LzFbKvXDXPYd4FFeLVVzD0CA4qghH5THrrH6LRm
 4DoRUK1QejrOC0v2zCRQvEN6RlI6WFGCf9YiYKkdrvswnjtLgOobCt7TvC9Nade2
 smGyxtDENkV6bAZgQgxMAlf88GCyKslb4Fu6U+sfKMDejWmlYEREbBsQc+/Gp6Af
 R6ZeiXEJ+KUF34RktPTmhJlUbzRc4Mw0Ij7Abrfhtpre9hifIp62XSrM8sasgLo=
 =k+ly
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2018-03-26' into staging

A fix for dirty bitmap migration through shared storage, and a VMDK
patch keeping us from creating too large extents.

# gpg: Signature made Mon 26 Mar 2018 21:17:05 BST
# gpg:                using RSA key F407DB0061D5CF40
# gpg: Good signature from "Max Reitz <mreitz@redhat.com>"
# Primary key fingerprint: 91BE B60A 30DB 3E88 57D1  1829 F407 DB00 61D5 CF40

* remotes/maxreitz/tags/pull-block-2018-03-26:
  vmdk: return ERROR when cluster sector is larger than vmdk limitation
  iotests: enable shared migration cases in 169
  qcow2: fix bitmaps loading when bitmaps already exist
  qcow2-bitmap: add qcow2_reopen_bitmaps_rw_hint()

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-03-27 14:11:30 +01:00
Stefan Hajnoczi
c40a254570 coroutine: avoid co_queue_wakeup recursion
qemu_aio_coroutine_enter() is (indirectly) called recursively when
processing co_queue_wakeup.  This can lead to stack exhaustion.

This patch rewrites co_queue_wakeup in an iterative fashion (instead of
recursive) with bounded memory usage to prevent stack exhaustion.

qemu_co_queue_run_restart() is inlined into qemu_aio_coroutine_enter()
and the qemu_coroutine_enter() call is turned into a loop to avoid
recursion.

There is one change that is worth mentioning:  Previously, when
coroutine A queued coroutine B, qemu_co_queue_run_restart() entered
coroutine B from coroutine A.  If A was terminating then it would still
stay alive until B yielded.  After this patch B is entered by A's parent
so that a A can be deleted immediately if it is terminating.

It is safe to make this change since B could never interact with A if it
was terminating anyway.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Message-id: 20180322152834.12656-3-stefanha@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2018-03-27 13:05:28 +01:00
yuchenlin
a77672ea3d vmdk: return ERROR when cluster sector is larger than vmdk limitation
VMDK has a hard limitation of extent size, which is due to the size of grain
table entry is 32 bits. It means it can only point to a grain located at
offset = 2^32. To avoid writing the user data beyond limitation and record a useless offset
in grain table. We should return ERROR here.

Signed-off-by: yuchenlin <yuchenlin@synology.com>
Message-id: 20180322133337.28024-1-yuchenlin@synology.com
Reviewed-by: Fam Zheng <famz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2018-03-26 21:17:24 +02:00
Vladimir Sementsov-Ogievskiy
2d949dfcef qcow2: fix bitmaps loading when bitmaps already exist
On reopen with existing bitmaps, instead of loading bitmaps, lets
reopen them if needed. This also fixes bitmaps migration through
shared storage.
Consider the case. Persistent bitmaps are stored on bdrv_inactivate.
Then, on destination process_incoming_migration_bh() calls
bdrv_invalidate_cache_all() which leads to
qcow2_load_autoloading_dirty_bitmaps() which fails if bitmaps are
already loaded on destination start.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-id: 20180320170521.32152-3-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2018-03-26 21:17:24 +02:00
Vladimir Sementsov-Ogievskiy
b1336cc2ec qcow2-bitmap: add qcow2_reopen_bitmaps_rw_hint()
Add version of qcow2_reopen_bitmaps_rw, which do the same work but
also return a hint about was header updated or not. This will be
used in the following fix for bitmaps reloading after migration.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 20180320170521.32152-2-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2018-03-26 21:17:24 +02:00
Kevin Wolf
6f16f7c562 vhdx: Check for 4 GB maximum log size on creation
It's unclear what the real maximum is, but we use an uint32_t to store
the log size in vhdx_co_create(), so we should check that the given
value fits in 32 bits.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Jeff Cody <jcody@redhat.com>
2018-03-26 12:17:43 +02:00
Kevin Wolf
0fcc38e7d0 vhdx: Don't use error_setg_errno() with constant errno
error_setg_errno() is meant for cases where we got an errno from the OS
that can add useful extra information to an error message. It's
pointless if we pass a constant errno, these cases should use plain
error_setg().

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Jeff Cody <jcody@redhat.com>
2018-03-26 12:17:43 +02:00
Kevin Wolf
b412f49407 vhdx: Require power-of-two block size on create
Images with a non-power-of-two block size are invalid and cannot be
opened. Reject such block sizes when creating an image.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Jeff Cody <jcody@redhat.com>
2018-03-26 12:17:43 +02:00
Kevin Wolf
2332d82589 parallels: Check maximum cluster size on create
It's unclear what the real maximum cluster size is for the Parallels
format, but let's at least make sure that we don't get integer
overflows in our .bdrv_co_create implementation.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2018-03-26 12:17:43 +02:00
Kevin Wolf
120bc742c0 luks: Turn another invalid assertion into check
Commit e39e959e fixed an invalid assertion in the .bdrv_length
implementation, but left a similar assertion in place for
.bdrv_truncate. Instead of crashing when the user requests a too large
image size, fail gracefully.

A file size of exactly INT64_MAX caused failure before, but is actually
legal.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2018-03-26 12:17:40 +02:00
Kevin Wolf
95a14d51b2 vdi: Fix build with CONFIG_VDI_DEBUG
Use qemu_uuid_unparse() instead of uuid_unparse() to make vdi.c compile
again when CONFIG_VDI_DEBUG is set. In order to prevent future bitrot,
replace '#ifdef CONFIG_VDI_DEBUG' by 'if (VDI_DEBUG)' so that the
compiler always sees the code.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2018-03-26 12:16:12 +02:00
Kevin Wolf
61fa64871d vdi: Change 'static' create option to 'preallocation' in QMP
What static=on really does is what we call metadata preallocation for
other block drivers. While we can still change the QMP interface, make
it more consistent by using 'preallocation' for VDI, too.

This doesn't implement any new functionality, so the only supported
preallocation modes are 'off' and 'metadata' for now.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2018-03-26 12:16:12 +02:00
Alberto Garcia
abf754fe40 qcow2: Reset free_cluster_index when allocating a new refcount block
When we try to allocate new clusters we first look for available ones
starting from s->free_cluster_index and once we find them we increase
their reference counts. Before we get to call update_refcount() to do
this last step s->free_cluster_index is already pointing to the next
cluster after the ones we are trying to allocate.

During update_refcount() it may happen however that we also need to
allocate a new refcount block in order to store the refcounts of these
new clusters (and to complicate things further that may also require
us to grow the refcount table). After all this we don't know if the
clusters that we originally tried to allocate are still available, so
we return -EAGAIN to ask the caller to restart the search for free
clusters.

This is what can happen in a common scenario:

  1) We want to allocate a new cluster and we see that cluster N is
     free.

  2) We try to increase N's refcount but all refcount blocks are full,
     so we allocate a new one at N+1 (where s->free_cluster_index was
     pointing at).

  3) Once we're done we return -EAGAIN to look again for a free
     cluster, but now s->free_cluster_index points at N+2, so that's
     the one we allocate. Cluster N remains unallocated and we have a
     hole in the qcow2 file.

This can be reproduced easily:

     qemu-img create -f qcow2 -o cluster_size=512 hd.qcow2 1M
     qemu-io -c 'write 0 124k' hd.qcow2

After this the image has 132608 bytes (256 clusters), and the refcount
block is full. If we write 512 more bytes it should allocate two new
clusters: the data cluster itself and a new refcount block.

     qemu-io -c 'write 124k 512' hd.qcow2

However the image has now three new clusters (259 in total), and the
first one of them is empty (and unallocated):

     dd if=hd.qcow2 bs=512c skip=256 count=1 | hexdump -C

If we write larger amounts of data in the last step instead of the 512
bytes used in this example we can create larger holes in the qcow2
file.

What this patch does is reset s->free_cluster_index to its previous
value when alloc_refcount_block() returns -EAGAIN. This way the caller
will try to allocate again the original clusters if they are still
free.

The output of iotest 026 also needs to be updated because now that
images have no holes some tests fail at a different point and the
number of leaked clusters is different.

Signed-off-by: Alberto Garcia <berto@igalia.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-03-26 12:16:00 +02:00
Fabiano Rosas
8140e786f0 block/blkreplay: Remove protocol-related fields
The blkreplay driver is not a protocol so it should implement bdrv_open
instead of bdrv_file_open and not provide a protocol_name.

Attempts to invoke this driver using protocol syntax
(i.e. blkreplay:<filename:options:...>) will now fail gracefully:

  $ qemu-img info blkreplay:foo
  qemu-img: Could not open 'blkreplay:foo': Unknown protocol 'blkreplay'

Signed-off-by: Fabiano Rosas <farosas@linux.vnet.ibm.com>
Reviewed-by: Pavel Dovgalyuk <pavel.dovgaluk@ispras.ru>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-03-26 12:16:00 +02:00
Fabiano Rosas
a7328ba55f block/throttle: Remove protocol-related fields
The throttle driver is not a protocol so it should implement bdrv_open
instead of bdrv_file_open and not provide a protocol_name.

Attempts to invoke this driver using protocol syntax
(i.e. throttle:<filename:options:...>) will now fail gracefully:

  $ qemu-img info throttle:foo
  qemu-img: Could not open 'throttle:foo': Unknown protocol 'throttle'

Signed-off-by: Fabiano Rosas <farosas@linux.vnet.ibm.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-03-26 12:16:00 +02:00
Fabiano Rosas
65d2c3e2f6 block/quorum: Remove protocol-related fields
The quorum driver is not a protocol so it should implement bdrv_open
instead of bdrv_file_open and not provide a protocol_name.

Attempts to invoke this driver using protocol syntax
(i.e. quorum:<filename:options:...>) will now fail gracefully:

  $ qemu-img info quorum:foo
  qemu-img: Could not open 'quorum:foo': Unknown protocol 'quorum'

Signed-off-by: Fabiano Rosas <farosas@linux.vnet.ibm.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-03-26 12:16:00 +02:00
Fabiano Rosas
cb83d2efe1 block/replication: Remove protocol_name field
The protocol_name field is used when selecting a driver via protocol
syntax (i.e. <protocol_name>:<filename:options:...>). Drivers that are
only selected explicitly (e.g. driver=replication,mode=primary,...)
should not have a protocol_name.

This patch removes the protocol_name field from the brdv_replication
structure so that attempts to invoke this driver using protocol syntax
will fail gracefully:

  $ qemu-img info replication:foo
  qemu-img: Could not open 'replication:': Unknown protocol 'replication'

Buglink: https://bugs.launchpad.net/qemu/+bug/1726733
Signed-off-by: Fabiano Rosas <farosas@linux.vnet.ibm.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-03-26 12:16:00 +02:00
Vladimir Sementsov-Ogievskiy
7e5c776d15 qapi: add block latency histogram interface
Set (and clear) histograms through new command
block-latency-histogram-set and show new statistics in
query-blockstats results.

For now, the command is marked experimental with prefix 'x-',
to gain experience with the interface without being stuck
with design decisions.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20180309165212.97144-3-vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
[eblake: fix typos, mention x- prefix in commit message]
Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-19 14:58:38 -05:00
Vladimir Sementsov-Ogievskiy
b741ae7417 block/accounting: introduce latency histogram
Introduce latency histogram statics for block devices.
For each accounted operation type, the latency region [0, +inf) is
divided into subregions by several points. Then, calculate
hits for each subregion.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20180309165212.97144-2-vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-19 14:58:37 -05:00
Max Reitz
7dc847ebba qapi: Replace qobject_to_X(o) by qobject_to(X, o)
This patch was generated using the following Coccinelle script:

@@
expression Obj;
@@
(
- qobject_to_qnum(Obj)
+ qobject_to(QNum, Obj)
|
- qobject_to_qstring(Obj)
+ qobject_to(QString, Obj)
|
- qobject_to_qdict(Obj)
+ qobject_to(QDict, Obj)
|
- qobject_to_qlist(Obj)
+ qobject_to(QList, Obj)
|
- qobject_to_qbool(Obj)
+ qobject_to(QBool, Obj)
)

and a bit of manual fix-up for overly long lines and three places in
tests/check-qjson.c that Coccinelle did not find.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Message-Id: <20180224154033.29559-4-mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[eblake: swap order from qobject_to(o, X), rebase to master, also a fix
to latent false-positive compiler complaint about hw/i386/acpi-build.c]
Signed-off-by: Eric Blake <eblake@redhat.com>
2018-03-19 14:58:36 -05:00
Paolo Bonzini
b16a7cfac5 iscsi: fix iSER compilation
This fails in Fedora 28.

Reported-by: Andreas Schwab <schwab@suse.de>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-03-19 12:01:39 +01:00
Fam Zheng
4f8e3a1f22 vvfat: Fix inherit_options flags
Overriding flags violates the precedence rules of
bdrv_reopen_queue_child. Just like the read-only option, no-flush should
be put into the options. The same is done in bdrv_temp_snapshot_options.

Reported-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-03-19 12:01:39 +01:00
Liang Li
b76e4458b1 block/mirror: change the semantic of 'force' of block-job-cancel
When doing drive mirror to a low speed shared storage, if there was heavy
BLK IO write workload in VM after the 'ready' event, drive mirror block job
can't be canceled immediately, it would keep running until the heavy BLK IO
workload stopped in the VM.

Libvirt depends on the current block-job-cancel semantics, which is that
when used without a flag after the 'ready' event, the command blocks
until data is in sync.  However, these semantics are awkward in other
situations, for example, people may use drive mirror for realtime
backups while still wanting to use block live migration.  Libvirt cannot
start a block live migration while another drive mirror is in progress,
but the user would rather abandon the backup attempt as broken and
proceed with the live migration than be stuck waiting for the current
drive mirror backup to finish.

The drive-mirror command already includes a 'force' flag, which libvirt
does not use, although it documented the flag as only being useful to
quit a job which is paused.  However, since quitting a paused job has
the same effect as abandoning a backup in a non-paused job (namely, the
destination file is not in sync, and the command completes immediately),
we can just improve the documentation to make the force flag obviously
useful.

Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Jeff Cody <jcody@redhat.com>
Cc: Kevin Wolf <kwolf@redhat.com>
Cc: Max Reitz <mreitz@redhat.com>
Cc: Eric Blake <eblake@redhat.com>
Cc: John Snow <jsnow@redhat.com>
Reported-by: Huaitong Han <huanhuaitong@didichuxing.com>
Signed-off-by: Huaitong Han <huanhuaitong@didichuxing.com>
Signed-off-by: Liang Li <liliangleo@didichuxing.com>
Signed-off-by: Jeff Cody <jcody@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-03-19 12:01:39 +01:00
Kevin Wolf
1cfeaf386e vpc: Require aligned size in .bdrv_co_create
Perform the rounding to match a CHS geometry only in the legacy code
path in .bdrv_co_create_opts. QMP now requires that the user already
passes a CHS aligned image size, unless force-size=true is given.

CHS alignment is required to make the image compatible with Virtual PC,
but not for use with newer Microsoft hypervisors.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
2018-03-19 12:01:39 +01:00
Kevin Wolf
182c883550 vpc: Support .bdrv_co_create
This adds the .bdrv_co_create driver callback to vpc, which
enables image creation over QMP.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
2018-03-19 12:01:39 +01:00
Kevin Wolf
09b68dab5a vhdx: Support .bdrv_co_create
This adds the .bdrv_co_create driver callback to vhdx, which
enables image creation over QMP.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
2018-03-19 12:01:39 +01:00
Kevin Wolf
da23248f27 vdi: Make comments consistent with other drivers
This makes the .bdrv_co_create(_opts) implementation of vdi look more
like the other recently converted block drivers.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
2018-03-19 12:01:39 +01:00
Kevin Wolf
959355a476 qed: Support .bdrv_co_create
This adds the .bdrv_co_create driver callback to qed, which
enables image creation over QMP.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
2018-03-19 12:01:39 +01:00
Kevin Wolf
42a3e1ab36 qcow: Support .bdrv_co_create
This adds the .bdrv_co_create driver callback to qcow, which
enables image creation over QMP.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Jeff Cody <jcody@redhat.com>
2018-03-19 12:01:39 +01:00
Kevin Wolf
1511b49040 parallels: Support .bdrv_co_create
This adds the .bdrv_co_create driver callback to parallels, which
enables image creation over QMP.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Jeff Cody <jcody@redhat.com>
2018-03-19 12:01:39 +01:00
Max Reitz
e38105748f vdi: Implement .bdrv_co_create
Signed-off-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-03-19 12:01:39 +01:00
Max Reitz
ec73f06071 vdi: Move file creation to vdi_co_create_opts
Signed-off-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-03-19 12:01:39 +01:00
Max Reitz
49858b5098 vdi: Pull option parsing from vdi_co_create
In preparation of QAPI-fying VDI image creation, we have to create a
BlockdevCreateOptionsVdi type which is received by a (future)
vdi_co_create().

vdi_co_create_opts() now converts the QemuOpts object into such a
BlockdevCreateOptionsVdi object.  The protocol-layer file is still
created in vdi_co_do_create() (and BlockdevCreateOptionsVdi.file is set
to an empty string), but that will be addressed by a follow-up patch.

Note that cluster-size is not part of the QAPI schema because it is not
supported by default.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-03-19 12:01:39 +01:00