Convert the ptimer test cases to the transaction-based ptimer API,
by changing to ptimer_init(), dropping the now-unused QEMUBH
variables, and surrounding each set of changes to the ptimer
state in ptimer_transaction_begin/commit calls.
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20191008171740.9679-4-peter.maydell@linaro.org
Currently the ptimer design uses a QEMU bottom-half as its
mechanism for calling back into the device model using the
ptimer when the timer has expired. Unfortunately this design
is fatally flawed, because it means that there is a lag
between the ptimer updating its own state and the device
callback function updating device state, and guest accesses
to device registers between the two can return inconsistent
device state.
We want to replace the bottom-half design with one where
the guest device's callback is called either immediately
(when the ptimer triggers by timeout) or when the device
model code closes a transaction-begin/end section (when the
ptimer triggers because the device model changed the
ptimer's count value or other state). As the first step,
rename ptimer_init() to ptimer_init_with_bh(), to free up
the ptimer_init() name for the new API. We can then convert
all the ptimer users away from ptimer_init_with_bh() before
removing it entirely.
(Commit created with
git grep -l ptimer_init | xargs sed -i -e 's/ptimer_init/ptimer_init_with_bh/'
and three overlong lines folded by hand.)
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20191008171740.9679-2-peter.maydell@linaro.org
- block: Fix crash with qcow2 partial cluster COW with small cluster
sizes (misaligned write requests with BDRV_REQ_NO_FALLBACK)
- qcow2: Fix integer overflow potentially causing corruption with huge
requests
- vhdx: Detect truncated image files
- tools: Support help options for --object
- Various block-related replay improvements
- iotests/028: Fix for long $TEST_DIRs
-----BEGIN PGP SIGNATURE-----
iQIcBAABAgAGBQJdpJwuAAoJEH8JsnLIjy/W4EIP/ieDG6LwYIkxk6UPAnPm2ZtT
jEBj1jZNbPPMVPCzeVryS2qgoSB4ItnKlSpbW9Z+DX2EL8gr/sHZot4J6BXkxaJV
mxHa4KCLZUYHfLbVBX+ubb/fufu2ItGUtY2LcdxyWASk/6mWKYVivPunwaK4gEiD
CBl9RgSMSy1bpBGIky2NMylOCr5KlSeweH/XL3J3Jodpp/3nnDpZ96iy953R7Bes
OpwZEsDHQmYhDufPqIWVzyq2hEUn22kGe/Rn2KfOHO7SR6aqr+DqtUwlDE9mxxBf
hFhm6bqbJhHQOTMbTrzqiYwDirR4S5/FlynI9+YbngU9fnkbCIOheTKL5M8PlSow
H+ZUtmU1Avp0wG3RZVmtCT9upFV7hpC4/fiMr8bdXCyuWy/7d7WB1G4e9ELiX7uo
VCl6gVviDQbEgnoNS7v6JbP/xjhHuu7Fxh5K0xgT6wtwP53cBqbxORMkwv2u3zCI
QRuiKOHZW3wv8tdRP/5qhdtIxTy6w20v/lAO/s0Xqn8YlnyfrH71LCNWmG4MOfgP
ZXwCv9nxpzVsTPU2nLowl0avCwmDVY8Iv/0sN+eybo8xp47pCmPV9dKa0rJ+RhFe
N5blnnwsmJFPW+QD5gBZn7eH3jafHxN2URhG3cwNdWxQS6SiTcVXDsTfn5YB6PjN
Tb9P2aUJYw94BvUJF2c+
=gCHy
-----END PGP SIGNATURE-----
Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging
Block layer patches:
- block: Fix crash with qcow2 partial cluster COW with small cluster
sizes (misaligned write requests with BDRV_REQ_NO_FALLBACK)
- qcow2: Fix integer overflow potentially causing corruption with huge
requests
- vhdx: Detect truncated image files
- tools: Support help options for --object
- Various block-related replay improvements
- iotests/028: Fix for long $TEST_DIRs
# gpg: Signature made Mon 14 Oct 2019 17:02:54 BST
# gpg: using RSA key 7F09B272C88F2FD6
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>" [full]
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74 56FE 7F09 B272 C88F 2FD6
* remotes/kevin/tags/for-upstream:
iotests: Test large write request to qcow2 file
qcow2: Limit total allocation range to INT_MAX
qemu-nbd: Support help options for --object
qemu-img: Support help options for --object
qemu-io: Support help options for --object
vl: Split off user_creatable_print_help()
iotests/028: Fix for long $TEST_DIRs
block: Reject misaligned write requests with BDRV_REQ_NO_FALLBACK
replay: add BH oneshot event for block layer
replay: finish record/replay before closing the disks
replay: don't drain/flush bdrv queue while RR is working
replay: update docs for record/replay with block devices
replay: disable default snapshot for record/replay
block: implement bdrv_snapshot_goto for blkreplay
block/vhdx: add check for truncated image files
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Without HEAD^, the following happens when you attempt a large write
request to a qcow2 file such that the number of bytes covered by all
clusters involved in a single allocation will exceed INT_MAX:
(A) handle_alloc_space() decides to fill the whole area with zeroes and
fails because bdrv_co_pwrite_zeroes() fails (the request is too
large).
(B) If handle_alloc_space() does not do anything, but merge_cow()
decides that the requests can be merged, it will create a too long
IOV that later cannot be written.
(C) Otherwise, all parts will be written separately, so those requests
will work.
In either B or C, though, qcow2_alloc_cluster_link_l2() will have an
overflow: We use an int (i) to iterate over nb_clusters, and then
calculate the L2 entry based on "i << s->cluster_bits" -- which will
overflow if the range covers more than INT_MAX bytes. This then leads
to image corruption because the L2 entry will be wrong (it will be
recognized as a compressed cluster).
Even if that were not the case, the .cow_end area would be empty
(because handle_alloc() will cap avail_bytes and nb_bytes at INT_MAX, so
their difference (which is the .cow_end size) will be 0).
So this test checks that on such large requests, the image will not be
corrupted. Unfortunately, we cannot check whether COW will be handled
correctly, because that data is discarded when it is written to null-co
(but we have to use null-co, because writing 2 GB of data in a test is
not quite reasonable).
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
For long test image paths, the order of the "Formatting" line and the
"(qemu)" prompt after a drive_backup HMP command may be reversed. In
fact, the interaction between the prompt and the line may lead to the
"Formatting" to being greppable at all after "read"-ing it (if the
prompt injects an IFS character into the "Formatting" string).
So just wait until we get a prompt. At that point, the block job must
have been started, so "info block-jobs" will only return "No active
jobs" once it is done.
Reported-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
The BDRV_REQ_NO_FALLBACK flag means that an operation should only be
performed if it can be offloaded or otherwise performed efficiently.
However a misaligned write request requires a RMW so we should return
an error and let the caller decide how to proceed.
This hits an assertion since commit c8bb23cbdb if the required
alignment is larger than the cluster size:
qemu-img create -f qcow2 -o cluster_size=2k img.qcow2 4G
qemu-io -c "open -o driver=qcow2,file.align=4k blkdebug::img.qcow2" \
-c 'write 0 512'
qemu-io: block/io.c:1127: bdrv_driver_pwritev: Assertion `!(flags & BDRV_REQ_NO_FALLBACK)' failed.
Aborted
The reason is that when writing to an unallocated cluster we try to
skip the copy-on-write part and zeroize it using BDRV_REQ_NO_FALLBACK
instead, resulting in a write request that is too small (2KB cluster
size vs 4KB required alignment).
Signed-off-by: Alberto Garcia <berto@igalia.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Mostly cleanups and minor fixes
[Note I'm seeing a hang on the aarch64 hosted x86-64 tcg migration
test in xbzrle; but I'm seeing that on current head as well]
-----BEGIN PGP SIGNATURE-----
iQIzBAABCAAdFiEERfXHG0oMt/uXep+pBRYzHrxb/ecFAl2g1JcACgkQBRYzHrxb
/efl1RAAjYukmf+kCFCw4Ws6nJ4000O85mpj0117SJpgTck1ivTC968REpl5pD0C
aHDzamNW82fiqjRxwF6KJRWic217NrmR1Z/j++SDyIjOc1ERQdB+RdCc7T2NkBT5
2HiPaceNiu9wOpqX/bto/xAug9vAxq5/1jeq+vhKxd+IcvAZII0SwKWn9mWA2209
H4i3v8OCv9isT6MRNitfWT/giYkI5HwFzA9a13S+zXioEGnoAmqzrrAQs2/MkyDt
bIeLbZyonH9hKbdrwmIXCvNEHA32BOPQyrsRp9CPZwRKVP2AzRYU9K9UjKncmYJS
bPdLYFmqEQm8ILQI6lyJ+pW1r/cyAUQBQii6NA+9ZfimxCSB06ArU+JeM0csl7HV
b4cG/bENFmtOzaoc3SrE6t1APlTiS9nxW6iH8zW3ozMEQGGihru7/6VIlwKTOfeX
kXKF92FTiTBpJ1u3/t05TPnxo4c2bKWM+Gj1okDAUsP8HovQpvJa8r92n1cC0+l8
l3pkFnrejzTcrexWIiKXYnPnO7Ez/Dm+0aCzlQkX7DSFxDnwI2T/BYk21FNlcI/L
rCHnkSLjYMWPelTLo9ZNuFaKL9UMeMtLPaIU9NBSSmsQ32/d8EXpDQwe8uAq+9Z/
qBir/mKyDe7I/InumtWQS46SS1/E1VyxDG2dxRWK9lN8DDOXRlM=
=Jouv
-----END PGP SIGNATURE-----
Merge remote-tracking branch 'remotes/dgilbert/tags/pull-migration-20191011a' into staging
Migration pull 2019-10-11
Mostly cleanups and minor fixes
[Note I'm seeing a hang on the aarch64 hosted x86-64 tcg migration
test in xbzrle; but I'm seeing that on current head as well]
# gpg: Signature made Fri 11 Oct 2019 20:14:31 BST
# gpg: using RSA key 45F5C71B4A0CB7FB977A9FA90516331EBC5BFDE7
# gpg: Good signature from "Dr. David Alan Gilbert (RH2) <dgilbert@redhat.com>" [full]
# Primary key fingerprint: 45F5 C71B 4A0C B7FB 977A 9FA9 0516 331E BC5B FDE7
* remotes/dgilbert/tags/pull-migration-20191011a: (21 commits)
migration: Support gtree migration
migration/multifd: pages->used would be cleared when attach to multifd_send_state
migration/multifd: initialize packet->magic/version once at setup stage
migration/multifd: use pages->allocated instead of the static max
migration/multifd: fix a typo in comment of multifd_recv_unfill_packet()
migration/postcopy: check PostcopyState before setting to POSTCOPY_INCOMING_RUNNING
migration/postcopy: rename postcopy_ram_enable_notify to postcopy_ram_incoming_setup
migration/postcopy: postpone setting PostcopyState to END
migration/postcopy: mis->have_listen_thread check will never be touched
migration: report SaveStateEntry id and name on failure
migration: pass in_postcopy instead of check state again
migration/postcopy: fix typo in mark_postcopy_blocktime_begin's comment
migration/postcopy: map large zero page in postcopy_ram_incoming_setup()
migration/postcopy: allocate tmp_page in setup stage
migration: Don't try and recover return path in non-postcopy
rcu: Use automatic rc_read unlock in core memory/exec code
migration: Use automatic rcu_read unlock in rdma.c
migration: Use automatic rcu_read unlock in ram.c
migration: Fix missing rcu_read_unlock
rcu: Add automatically released rcu_read_lock variants
...
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
tests/test-bdrv-drain can hang in tests/iothread.c:iothread_run():
while (!atomic_read(&iothread->stopping)) {
aio_poll(iothread->ctx, true);
}
The iothread_join() function works as follows:
void iothread_join(IOThread *iothread)
{
iothread->stopping = true;
aio_notify(iothread->ctx);
qemu_thread_join(&iothread->thread);
If iothread_run() checks iothread->stopping before the iothread_join()
thread sets stopping to true, then aio_notify() may be optimized away
and iothread_run() hangs forever in aio_poll().
The correct way to change iothread->stopping is from a BH that executes
within iothread_run(). This ensures that iothread->stopping is checked
after we set it to true.
This was already fixed for ./iothread.c (note this is a different source
file!) by commit 2362a28ea1 ("iothread:
fix iothread_stop() race condition"), but not for tests/iothread.c.
Fixes: 0c330a734b
("aio: introduce aio_co_schedule and aio_co_wake")
Reported-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20191003100103.331-1-stefanha@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Introduce support for GTree migration. A custom save/restore
is implemented. Each item is made of a key and a data.
If the key is a pointer to an object, 2 VMSDs are passed into
the GTree VMStateField.
When putting the items, the tree is traversed in sorted order by
g_tree_foreach.
On the get() path, gtrees must be allocated using the proper
key compare, key destroy and value destroy. This must be handled
beforehand, for example in a pre_load method.
Tests are added to test save/dump of structs containing gtrees
including the virtio-iommu domain/mappings scenario.
Signed-off-by: Eric Auger <eric.auger@redhat.com>
Message-Id: <20191011121724.433-1-eric.auger@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
uintptr_t fixup for test on 32bit
Linux 5.3 has made 0.0.0.0/8 a working IPv4 subnet. As such, "42" is
now a valid host, and the connection to it will (hopefully) time out
over a long period rather than quickly return with EINVAL.
So let us use a negative integer for testing that NBD will not crash
when it receives integer hosts. This way, the connection will again
fail quickly and reliably.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id: 20191002174052.5773-1-mreitz@redhat.com
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Some distros are now defaulting to LUKS version 2 which QEMU cannot
process. For our I/O test that validates interoperability between the
kernel/cryptsetup and QEMU, we need to explicitly ask for version 1
of the LUKS format.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-id: 20190927101155.25896-1-berrange@redhat.com
Tested-by: Maxim Levitsky <mlevitsk@redhat.com>
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Drop write notifiers and use filter node instead.
= Changes =
1. Add filter-node-name argument for backup qmp api. We have to do it
in this commit, as 257 needs to be fixed.
2. There are no more write notifiers here, so is_write_notifier
parameter is dropped from block-copy paths.
3. To sync with in-flight requests at job finish we now have drained
removing of the filter, we don't need rw-lock.
4. Block-copy is now using BdrvChildren instead of BlockBackends
5. As backup-top owns these children, we also move block-copy state
into backup-top's ownership.
= Iotest changes =
56: op-blocker doesn't shoot now, as we set it on source, but then
check on filter, when trying to start second backup.
To keep the test we instead can catch another collision: both jobs will
get 'drive0' job-id, as job-id parameter is unspecified. To prevent
interleaving with file-posix locks (as they are dependent on config)
let's use another target for second backup.
Also, it's obvious now that we'd like to drop this op-blocker at all
and add a test-case for two backups from one node (to different
destinations) actually works. But not in these series.
141: Output changed: prepatch, "Node is in use" comes from bdrv_has_blk
check inside qmp_blockdev_del. But we've dropped block-copy blk
objects, so no more blk objects on source bs (job blk is on backup-top
filter bs). New message is from op-blocker, which is the next check in
qmp_blockdev_add.
257: The test wants to emulate guest write during backup. They should
go to filter node, not to original source node, of course. Therefore we
need to specify filter node name and use it.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-id: 20191001131409.14202-6-vsementsov@virtuozzo.com
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
125 should not use qemu-img to get the on-disk image size, because that
reports it in a human-readable format that is useless to us. Just use
stat instead (like we do to get the image file length).
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190925183231.11196-4-mreitz@redhat.com
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
And by that I mean all XFS versions, as far as I can tell. All details
are in the comment below.
We never noticed this problem because we only read the first number from
qemu-img info's "disk size" output -- and that is effectively useless,
because qemu-img prints a human-readable value (which generally includes
a decimal point). That will be fixed in the next patch.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190925183231.11196-3-mreitz@redhat.com
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
If we use growth_mode = metadata, it is very much possible that the file
uses more disk space after we have written something to the added area.
We did indeed want to test for this case, but unfortunately we evidently
just copied the code from the "Test creation preallocation" section and
forgot to replace "$create_mode" by "$growth_mode".
We never noticed because we only read the first number from qemu-img
info's "disk size" output -- and that is effectively useless, because
qemu-img prints a human-readable value (which generally includes a
decimal point). That will be fixed in the patch after the next one.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190925183231.11196-2-mreitz@redhat.com
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Anton Nefedov <anton.nefedov@virtuozzo.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20190923121737.83281-3-anton.nefedov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
SCSI devices are unused in test, drop them.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190920142056.12778-12-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
After previous commit Drive.device is actually unused. Drop it together
with .name property. While being here reuse .node in qmp commands
instead of writing 'drive0' twice.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190920142056.12778-11-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
After backup-top filter appearing it's not possible to see dirty
bitmaps in top node, so use node-name instead.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190920142056.12778-10-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
Upcoming asynchronous handling of sub-parts of qcow2 requests will
change number of leaked clusters and even make it racy. As a
preparation, ignore leaks on failure parts in 026.
It's not trivial to just grep or substitute qemu-img output for such
thing. Instead do better: 3 is a error code of qemu-img check, if only
leaks are found. Catch this case and print success output.
Suggested-by: Anton Nefedov <anton.nefedov@virtuozzo.com>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-id: 20190916175324.18478-2-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
This pull request also contains the two commits from the previous pull request
that was dropped due to a mingw compilation error. The compilation should now
be fixed.
-----BEGIN PGP SIGNATURE-----
iQEzBAABCAAdFiEEhpWov9P5fNqsNXdanKSrs4Grc8gFAl2coyIACgkQnKSrs4Gr
c8iE1AgAh1ACKXmuH+ml3jkKniLVpr4KEIWB5CmlMSQtmv2Hteyi+cI+T/WvhFGB
Xw8yW1yxVeV98lPWZJeb06s/Ry/ZXef+D63i6xcQzc/3a6pTJKeLdi8acbJuSRso
A3VjQaczjrP9GOPvOoy3XkXoLr5nuD4NI8TYMmhWgwb6eSETUHGYgQo1uoZkqkuF
4/K856dtN2wb3WqChdwBhg+1o8WRpxEZ74pVC0SBGtKcBDjiY7J/BEXfNJo5M4gJ
s1x3FVpSN19LYljgo4WCJt/hzMkOSy0uV4mSzCx6U/dEB+w+VUvuZ7wTt69Kweji
vTMsFDN67Xzq8xWqeiG5uGBM8NRIwQ==
=zUDj
-----END PGP SIGNATURE-----
Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' into staging
Pull request
This pull request also contains the two commits from the previous pull request
that was dropped due to a mingw compilation error. The compilation should now
be fixed.
# gpg: Signature made Tue 08 Oct 2019 15:54:26 BST
# gpg: using RSA key 8695A8BFD3F97CDAAC35775A9CA4ABB381AB73C8
# gpg: Good signature from "Stefan Hajnoczi <stefanha@redhat.com>" [full]
# gpg: aka "Stefan Hajnoczi <stefanha@gmail.com>" [full]
# Primary key fingerprint: 8695 A8BF D3F9 7CDA AC35 775A 9CA4 ABB3 81AB 73C8
* remotes/stefanha/tags/block-pull-request:
iotests/262: Switch source/dest VM launch order
block: Skip COR for inactive nodes
virtio-blk: schedule virtio_notify_config to run on main context
util/ioc.c: try to reassure Coverity about qemu_iovec_init_extended
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Launching the destination VM before the source VM gives us a regression
test for HEAD^:
The guest device causes a read from the disk image through
guess_disk_lchs(). This will not work if the first sector (containing
the partition table) is yet unallocated, we use COR, and the node is
inactive.
By launching the source VM before the destination, however, the COR
filter on the source will allocate that area in the image shared between
both VMs, thus the problem will not become apparent.
Switching the launch order causes the sector to still be unallocated
when guess_disk_lchs() runs on the inactive node in the destination VM,
and thus we get our test case.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20191001174827.11081-3-mreitz@redhat.com
Message-Id: <20191001174827.11081-3-mreitz@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
drop Python2 dependency in EDK2 build scripts.
-----BEGIN PGP SIGNATURE-----
iQIzBAABCAAdFiEEicHnj2Ae6GyGdJXLoqP9bt6twN4FAl2bPjoACgkQoqP9bt6t
wN6fOhAAzJm5IDx6w8HFqLyMhdgfBh0q/6xCNKCpbiVIpkmZeetUPb8WsKizCQQ+
uhsl6KcKErrGTorBLPUcSDaeCdaCCPOoLEZqxbCYQ1xNDjiK6pXStm+7Ztyp9KBG
e92qQEXZMjKel0toWMNMLlVVR3yY6zNkKTqBXmIYIBb+qVGFoZdwCo/9f/8KlGv1
iyN1Bl4hGRaIw/qQ9ptjctuWeE1dOm1UawXdvAi7yBUs4kuJ1nLk7MaBEueOKZHh
3KdcroJw55N5kTbHiUonRbmGHikjxicOlmpKGi7x415iec4o3pGiXR5AGcQuoJNX
lhopEG/+Eiq2IzmVUp6mYFDmvCKzt4sTnFGFkPWVuPDmJ8ym8x+D12ESrCn/tbwj
eaYBPhSi2BcEkaIblk4pj7Bma7RntVt5ga2q/2chqhSWI5X85rmW/avMgBj9y4Xo
hQVhyWKIqTuw2cFdigL9OQFzRxhPaltFQO6e/5iyjbwNZnFKwzN1OoZ2PAOK1BtZ
veoyDWsD/6YjhcOogABVlUI13rSj+M9soR++fjmMe/n+DTqpvfjPUOOmZ6PvWS9Q
zodyctKkKZau0ecq5ZoE3JfMLMSPO0XIKk46bO/cRphn5VoxC4bnEjqQ+rMbnEp+
1arIEW3m7e3Ngmvb6zUrHIL7dTtvzri9kq4vFkAskwrg8t2R0Kg=
=BE2l
-----END PGP SIGNATURE-----
Merge remote-tracking branch 'remotes/philmd-gitlab/tags/edk2-next-20191007' into staging
Improve scripts relying on the EDK2 submodule,
drop Python2 dependency in EDK2 build scripts.
# gpg: Signature made Mon 07 Oct 2019 14:31:38 BST
# gpg: using RSA key 89C1E78F601EE86C867495CBA2A3FD6EDEADC0DE
# gpg: Good signature from "Philippe Mathieu-Daudé (Phil) <philmd@redhat.com>" [marginal]
# gpg: WARNING: This key is not certified with sufficiently trusted signatures!
# gpg: It is not certain that the signature belongs to the owner.
# Primary key fingerprint: 89C1 E78F 601E E86C 8674 95CB A2A3 FD6E DEAD C0DE
* remotes/philmd-gitlab/tags/edk2-next-20191007:
edk2 build scripts: work around TianoCore#1607 without forcing Python 2
edk2 build scripts: honor external BaseTools flags with uefi-test-tools
roms: Add a 'make help' target alias
roms/Makefile.edk2: don't pull in submodules when building from tarball
make-release: pull in edk2 submodules so we can build it from tarballs
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
It turns out that forcing python2 for running the edk2 "build" utility is
neither necessary nor sufficient.
Forcing python2 is not sufficient for two reasons:
- QEMU is moving away from python2, with python2 nearing EOL,
- according to my most recent testing, the lacking dependency information
in the makefiles that are generated by edk2's "build" utility can cause
parallel build failures even when "build" is executed by python2.
And forcing python2 is not necessary because we can still return to the
original idea of filtering out jobserver-related options from MAKEFLAGS.
So do that.
While at it, cut short edk2's auto-detection of the python3.* minor
version, by setting PYTHON_COMMAND to "python3" (which we expect to be
available wherever we intend to build edk2).
With this patch, the guest UEFI binaries that are used as part of the BIOS
tables test, and the OVMF and ArmVirtQemu platform firmwares, will be
built strictly in a single job, regardless of an outermost "-jN" make
option. Alas, there appears to be no reliable way to build edk2 in an
(outer make, inner make) environment, with a jobserver enabled.
Cc: Eduardo Habkost <ehabkost@redhat.com>
Cc: John Snow <jsnow@redhat.com>
Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Reported-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20190920083808.21399-3-lersek@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Unify the recipe for "build-edk2-tools" in
"tests/uefi-test-tools/Makefile" with the recipe for "edk2-basetools" in
"roms/Makefile".
Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20190920083808.21399-2-lersek@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
-fsanitize=undefined is not the same thing as --enable-sanitizers. After
commit 47c823e ("tests/docker: add sanitizers back to clang build", 2019-09-11)
test-clang is almost duplicating the asan (test-debug) test, so
partly revert commit 47c823e5b while leaving ubsan enabled.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Serial test is currently hard-coded to /dev/null.
On Windows, serial chardev expect a COM: device, which may not be
availble.
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
There are just too many leaks in device-introspect-test (especially for
the plethora of arm and aarch64 boards) to make LeakSanitizer useful;
disable it for now.
Whoever is interested in debugging leaks can also use valgrind like this:
QTEST_QEMU_BINARY=aarch64-softmmu/qemu-system-aarch64 \
QTEST_QEMU_IMG=qemu-img \
valgrind --trace-children=yes --leak-check=full \
tests/device-introspect-test -p /aarch64/device/introspect/concrete/defaults/none
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Some scripts check the Python version number and have two code paths to
accomodate both Python 2 and 3. Remove the code specific to Python 2 and
assert the minimum version of 3.6 instead (check skips Python tests in
this case, so the assertion would only ever trigger if a Python script
is executed manually).
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Running iotests is not required to build QEMU, so we can have stricter
version requirements for Python here and can make use of new features
and drop compatibility code earlier.
This makes qemu-iotests skip all Python tests if a Python version before
3.6 is used for the build.
Suggested-by: Eduardo Habkost <ehabkost@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Yury Kotov <yury-kotov@yandex-team.ru>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <20190909131335.16848-4-yury-kotov@yandex-team.ru>
[Reorganize check_migration_status for rebase. - Paolo]
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
-----BEGIN PGP SIGNATURE-----
iQJGBAABCAAwFiEENUvIs9frKmtoZ05fOHC0AOuRhlMFAl2PebUSHGFybWJydUBy
ZWRoYXQuY29tAAoJEDhwtADrkYZT/VEP/Rru2BgOV/6OSIunW9Ii6qUsVNYtYM0L
4XrOaVJBhNnakAdPPphXmhk6iRCJESZ+4z+s4Iw+yD4aSRYEWowMFi1blaIk+3lH
BNBllepKrvX9ZZyMHCooKrrO2PEPOPRin5izCDn5O93onQjwzXfWtuxZRn6WaLzB
MOtixr340ysa+KNktpAWWPH/NJFq+LLvyQVUdN5xR1i5YBEVE9s0uZg9uq+zpFEg
xSw45BDQSCjNNywd5mqJ0x+y7PCeGAINS9el43ernn654qVXpNgYepE3PpkCjCH7
snFKFJA8+h1VjXPd8/amJimD11+CFIUlhvtlVwccCWme8PRE6Hveerf7FvexnjpN
yGfDOV2ezgrnFspAjI3hrPfsF1Vfxe+eDOUg+y9xGtRgD5LsWmkUBEukKZocVYQG
H52BT6qt3IlUbWVPuHlWEUOsWKk7e3IeAZgnoaafRxwmLHaVWX0SPZCGm1Bfvxe2
LxXnS/pPWubyTrmQA9xCDhILrvkquCyOVBufFnS6D3h45psQKXr2Rh+vVjnYslRR
/B1tDAFZvu9Q6+Y9//AnnLhTZlEPf+qg7Ajv+rH8PTl7SbB/tNVI+vrJJCp5AJab
OifkqahV4H12STf4N2R3g4j/HqPj2PB1/GdLpmOc94ZY5h8GMxviEpwZkKTnw5xD
zhwDdMVqL6c1
=CNvl
-----END PGP SIGNATURE-----
Merge remote-tracking branch 'remotes/armbru/tags/pull-qapi-2019-09-28' into staging
QAPI patches for 2019-09-28
# gpg: Signature made Sat 28 Sep 2019 16:18:13 BST
# gpg: using RSA key 354BC8B3D7EB2A6B68674E5F3870B400EB918653
# gpg: issuer "armbru@redhat.com"
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>" [full]
# gpg: aka "Markus Armbruster <armbru@pond.sub.org>" [full]
# Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867 4E5F 3870 B400 EB91 8653
* remotes/armbru/tags/pull-qapi-2019-09-28: (27 commits)
qapi: Improve source file read error handling
qapi: Improve reporting of redefinition
qapi: Improve reporting of missing documentation comment
qapi: Eliminate check_keys(), rename check_known_keys()
qapi: Improve reporting of invalid 'if' further
qapi: Avoid redundant definition references in error messages
qapi: Improve reporting of missing / unknown definition keys
qapi: Improve reporting of invalid flags
qapi: Improve reporting of invalid 'if' errors
qapi: Move context-free checking to the proper place
qapi: Move context-sensitive checking to the proper place
qapi: Inline check_name() into check_union()
qapi: Plumb info to the QAPISchemaMember
qapi: Make check_type()'s array case a bit more obvious
qapi: Move check for reserved names out of add_name()
qapi: Report invalid '*' prefix like any other invalid name
qapi: Use check_name_str() where it suffices
qapi: Improve reporting of invalid name errors
qapi: Reorder check_FOO() parameters for consistency
qapi: Improve reporting of member name clashes
...
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Everybody who used something like "-machine accel=kvm:tcg" in the past
might be tempted to specify a similar list with the -accel parameter,
too, for example "-accel kvm:tcg". However, this is not how this
options is thought to be used, since each "-accel" should only take care
of one specific accelerator.
In the long run, we really should rework the "-accel" code completely,
so that it does not set "-machine accel=..." anymore internally, but
is completely independent from "-machine". For the short run, let's
make sure that users cannot use "-accel xyz:tcg", so that we avoid
that we have to deal with such cases in the wild later.
Message-Id: <20190930123505.11607-1-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
While at it, simplify using $(land).
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20190926111955.17276-3-marcandre.lureau@redhat.com>
Reviewed-by: Laurent Vivier <lvivier@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Fixes: dad5ddcea3 ("check: Only test usb-ehci when it is compiled in")
Signed-off-by: Thomas Huth <thuth@redhat.com>
Fixes commit
e5758de4e8 ("tests/libqtest: Make
qtest_qmp_device_add/del independent from global_qtest")
and commit
dd21074972 ("tests/libqtest: Use
libqtest-single.h in tests that require global_qtest").
Cc: Thomas Huth <thuth@redhat.com>
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20190926111955.17276-2-marcandre.lureau@redhat.com>
Reviewed-by: Laurent Vivier <lvivier@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
* Fix the CBAR register implementation for Cortex-A53,
Cortex-A57, Cortex-A72
* Fix direct booting of Linux kernels on emulated CPUs
which have an AArch32 EL3 (incorrect NSACR settings
meant they could not access the FPU)
* semihosting cleanup: do more work at translate time
and less work at runtime
-----BEGIN PGP SIGNATURE-----
iQJNBAABCAA3FiEE4aXFk81BneKOgxXPPCUl7RQ2DN4FAl2OHYsZHHBldGVyLm1h
eWRlbGxAbGluYXJvLm9yZwAKCRA8JSXtFDYM3oHlD/4iD57WzVkf2EagPg61EbqV
KJU0bloj6lpfhI410zv6RLfSxRhuJKj1voBPl0wh/uWz4kIHBjcYZgRQGZz5+Fem
XE4j7bLfgXlbYkjl6CFo3oqZJM+iVmMofKVbpj7nEnO6cB9nW2O4Uk88vPTqCRUp
uip/ZveoQ3WvzyM8ERWiIiGZrvCRPnfTFvWGNEDd+ESx3ACmNbeAHilMURESkXR8
3iRt83bzL+H7xRpVEmLvUAbjJlf+4dzyftJSwTDquLsu+g4I45BDe1ki7ip9U06B
EvgNZ0TKchNI2kn6I4R0XAYAdZyKRONWqYTPE3xEtweihLwOKYsKfQViSHkhYxuE
upqMfsSzpT2ivqMb5myFU8JbG6jZZGTguAZ40MQT073gckgFoFfWjAtzR0fWa/Cy
VJ79fWIfOXrRsc76UDBeDuJ3CFEliFMSzDJWwglxlp9JX6ckfHH0Vwfmj9NPcuRw
AeAkI7Xh+emNKftJzNtC+6Ba7jMhMLLDBoe1r3NQYK1BFg/JRtkGCja3UAswotXH
hEYMicbMnkhOGEKxjKL0jbl33XKKAVq3pens2tT0QIz3Xqzh9iIcceCnv4MsddK9
MPU8yfQYcj6eNxVBLofhuRGURMK4BpQzj2Rxg03G3dRpFuNEwneUrx64q8lEv4Y5
EWSFxOoBPEpooiMCoboZ/A==
=/0m2
-----END PGP SIGNATURE-----
Merge remote-tracking branch 'remotes/pmaydell/tags/pull-target-arm-20190927' into staging
target-arm queue:
* Fix the CBAR register implementation for Cortex-A53,
Cortex-A57, Cortex-A72
* Fix direct booting of Linux kernels on emulated CPUs
which have an AArch32 EL3 (incorrect NSACR settings
meant they could not access the FPU)
* semihosting cleanup: do more work at translate time
and less work at runtime
# gpg: Signature made Fri 27 Sep 2019 15:32:43 BST
# gpg: using RSA key E1A5C593CD419DE28E8315CF3C2525ED14360CDE
# gpg: issuer "peter.maydell@linaro.org"
# gpg: Good signature from "Peter Maydell <peter.maydell@linaro.org>" [ultimate]
# gpg: aka "Peter Maydell <pmaydell@gmail.com>" [ultimate]
# gpg: aka "Peter Maydell <pmaydell@chiark.greenend.org.uk>" [ultimate]
# Primary key fingerprint: E1A5 C593 CD41 9DE2 8E83 15CF 3C25 25ED 1436 0CDE
* remotes/pmaydell/tags/pull-target-arm-20190927:
hw/arm/boot: Use the IEC binary prefix definitions
hw/arm/boot.c: Set NSACR.{CP11,CP10} for NS kernel boots
tests/tcg: add linux-user semihosting smoke test for ARM
target/arm: remove run-time semihosting checks for linux-user
target/arm: remove run time semihosting checks
target/arm: handle A-profile semihosting at translate time
target/arm: handle M-profile semihosting at translate time
tests/tcg: clean-up some comments after the de-tangling
target/arm: fix CBAR register for AArch64 CPUs
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
# Conflicts:
# tests/tcg/arm/Makefile.target
qapi-gen.py crashes when it can't open the main schema file, and when
it can't read from any schema file. Lazy.
Change QAPISchema.__init__() to take a file name instead of a file
object. Move the open code from _include() to __init__(), so it's
used for the main schema file, too.
Move the read into the try for good measure, and rephrase the error
message.
Reporting open or read failure for the main schema file needs a
QAPISourceInfo representing "no source". Make QAPISourceInfo cope
with fname=None.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190927134639.4284-27-armbru@redhat.com>
Point to the previous definition, unless it's a built-in.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190927134639.4284-26-armbru@redhat.com>
Have check_exprs() check this later, so the error message gains an "in
definition line". Tweak the error message.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190927134639.4284-25-armbru@redhat.com>
check_if()'s errors don't point to the offending part of the
expression. For instance:
tests/qapi-schema/alternate-branch-if-invalid.json:2: 'if' condition ' ' makes no sense
Other check_FOO() do, with the help of a @source argument. Make
check_if() do that, too. The example above improves to:
tests/qapi-schema/alternate-branch-if-invalid.json:2: 'if' condition ' ' of 'data' member 'branch' makes no sense
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190927134639.4284-23-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Many error messages refer to the offending definition even though
they're preceded by an "in definition" line. Rephrase them.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190927134639.4284-22-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Have check_exprs() call check_keys() later, so its error messages gain
an "in definition" line.
Both check_keys() and check_name_is_str() check the definition's name
is a string. Since check_keys() now runs after check_name_is_str()
rather than before, its check is dead. Bury it. Checking values in
check_keys() is unclean anyway.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190927134639.4284-21-armbru@redhat.com>
Split check_flags() off check_keys() and have check_exprs() call it
later, so its error messages gain an "in definition" line. Tweak the
error messages.
Checking values in a function named check_keys() is unclean anyway.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190927134639.4284-20-armbru@redhat.com>
Move check_if() from check_keys() to check_exprs() and call it later,
so its error messages gain an "in definition" line.
Checking values in a function named check_keys() is unclean anyway.
The original sin was commit 0545f6b887 "qapi: Better error messages
for bad expressions", which checks the value of key 'name'. More
sinning in commit 2cbf09925a "qapi: More rigorous checking for type
safety bypass", commit c818408e44 "qapi: Implement boxed types for
commands/events", and commit 967c885108 "qapi: add 'if' to top-level
expressions". This commit does penance for the latter. The next
commits will do penance for the others.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190927134639.4284-19-armbru@redhat.com>
QAPISchemaCommand.check() and QAPISchemaEvent().check() check 'data'
is present when 'boxed': true. That's context-free. Move to
check_command() and check_event().
Tweak the error message while there.
check_exprs() & friends now check exactly what qapi-code-gen.txt calls
the second layer of syntax.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190927134639.4284-18-armbru@redhat.com>
When we introduced the QAPISchema intermediate representation (commit
ac88219a6c), we took a shortcut: we left check_exprs() & friends
alone instead of moving semantic checks into the
QAPISchemaFOO.check(). The .check() assert check_exprs() did its job.
Time to finish the conversion job. Move exactly the context-sensitive
checks to the .check(). They replace assertions there. Context-free
checks stay put.
Fixes the misleading optional tag error demonstrated by test
flat-union-optional-discriminator.
A few other error message improve.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190927134639.4284-17-armbru@redhat.com>
