Commit Graph

7931 Commits

Author SHA1 Message Date
Peter Maydell
0021c4765a * SGX implementation for x86
* Miscellaneous bugfixes
 * Fix dependencies from ROMs to qtests
 -----BEGIN PGP SIGNATURE-----
 
 iQFIBAABCAAyFiEE8TM4V0tmI4mGbHaCv/vSX3jHroMFAmFVu/sUHHBib256aW5p
 QHJlZGhhdC5jb20ACgkQv/vSX3jHroNFUgf+OexjKqJw4qzbDdQrxWqw3upoFblk
 y4OrmrhCyCKDwPghnjHUEVGHnNKqKpCLoIvtvFZ7xX/qezpMtZxVUliOVNQGmioR
 MZU/DbdlvVL/t8yKjfz1ljshk55hnSJ7rAv8LBA+B3uNzyJ+LZU9+Kbvmei5oyex
 nenCtXnoVNBJMvTBE/KfJbp0UasEb1OTvPBa0Y7mHyDub28FDPKr9WZbloCLUtE+
 uXwbZ34VRDsxbLnXh+BJ+ljOQLdsJErAkiPKTnW1/3W8Ti7PzOzvLpbSIVdBv/9A
 U1qOEm48BjCrG/tFJvTUm0ZM7AHmqYfvmwpenDpL0FhReohMdUa3pycQ9g==
 =Hicy
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/bonzini-gitlab/tags/for-upstream' into staging

* SGX implementation for x86
* Miscellaneous bugfixes
* Fix dependencies from ROMs to qtests

# gpg: Signature made Thu 30 Sep 2021 14:30:35 BST
# gpg:                using RSA key F13338574B662389866C7682BFFBD25F78C7AE83
# gpg:                issuer "pbonzini@redhat.com"
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>" [full]
# gpg:                 aka "Paolo Bonzini <pbonzini@redhat.com>" [full]
# Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4  E2F7 7E15 100C CD36 69B1
#      Subkey fingerprint: F133 3857 4B66 2389 866C  7682 BFFB D25F 78C7 AE83

* remotes/bonzini-gitlab/tags/for-upstream: (33 commits)
  meson_options.txt: Switch the default value for the vnc option to 'auto'
  build-sys: add HAVE_IPPROTO_MPTCP
  memory: Add tracepoint for dirty sync
  memory: Name all the memory listeners
  target/i386: Fix memory leak in sev_read_file_base64()
  tests: qtest: bios-tables-test depends on the unpacked edk2 ROMs
  meson: unpack edk2 firmware even if --disable-blobs
  target/i386: Add the query-sgx-capabilities QMP command
  target/i386: Add HMP and QMP interfaces for SGX
  docs/system: Add SGX documentation to the system manual
  sgx-epc: Add the fill_device_info() callback support
  i440fx: Add support for SGX EPC
  q35: Add support for SGX EPC
  i386: acpi: Add SGX EPC entry to ACPI tables
  i386/pc: Add e820 entry for SGX EPC section(s)
  hw/i386/pc: Account for SGX EPC sections when calculating device memory
  hw/i386/fw_cfg: Set SGX bits in feature control fw_cfg accordingly
  Adjust min CPUID level to 0x12 when SGX is enabled
  i386: Propagate SGX CPUID sub-leafs to KVM
  i386: kvm: Add support for exposing PROVISIONKEY to guest
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-09-30 17:38:31 +01:00
Paolo Bonzini
809954efc2 tests: qtest: bios-tables-test depends on the unpacked edk2 ROMs
Skip the test if bzip2 is not available, and run it after they are
uncompressed.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20210923105529.3845741-2-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-09-30 15:30:24 +02:00
Yang Zhong
0205c4fa1e target/i386: Add the query-sgx-capabilities QMP command
Libvirt can use query-sgx-capabilities to get the host
sgx capabilities to decide how to allocate SGX EPC size to VM.

Signed-off-by: Yang Zhong <yang.zhong@intel.com>
Message-Id: <20210910102258.46648-3-yang.zhong@intel.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-09-30 15:30:24 +02:00
Yang Zhong
57d874c4c7 target/i386: Add HMP and QMP interfaces for SGX
The QMP and HMP interfaces can be used by monitor or QMP tools to retrieve
the SGX information from VM side when SGX is enabled on Intel platform.

Signed-off-by: Yang Zhong <yang.zhong@intel.com>
Message-Id: <20210910102258.46648-2-yang.zhong@intel.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-09-30 15:30:24 +02:00
Vladimir Sementsov-Ogievskiy
0c8022876f block: use int64_t instead of int in driver discard handlers
We are generally moving to int64_t for both offset and bytes parameters
on all io paths.

Main motivation is realization of 64-bit write_zeroes operation for
fast zeroing large disk chunks, up to the whole disk.

We chose signed type, to be consistent with off_t (which is signed) and
with possibility for signed return type (where negative value means
error).

So, convert driver discard handlers bytes parameter to int64_t.

The only caller of all updated function is bdrv_co_pdiscard in
block/io.c. It is already prepared to work with 64bit requests, but
pass at most max(bs->bl.max_pdiscard, INT_MAX) to the driver.

Let's look at all updated functions:

blkdebug: all calculations are still OK, thanks to
  bdrv_check_qiov_request().
  both rule_check and bdrv_co_pdiscard are 64bit

blklogwrites: pass to blk_loc_writes_co_log which is 64bit

blkreplay, copy-on-read, filter-compress: pass to bdrv_co_pdiscard, OK

copy-before-write: pass to bdrv_co_pdiscard which is 64bit and to
  cbw_do_copy_before_write which is 64bit

file-posix: one handler calls raw_account_discard() is 64bit and both
  handlers calls raw_do_pdiscard(). Update raw_do_pdiscard, which pass
  to RawPosixAIOData::aio_nbytes, which is 64bit (and calls
  raw_account_discard())

gluster: somehow, third argument of glfs_discard_async is size_t.
  Let's set max_pdiscard accordingly.

iscsi: iscsi_allocmap_set_invalid is 64bit,
  !is_byte_request_lun_aligned is 64bit.
  list.num is uint32_t. Let's clarify max_pdiscard and
  pdiscard_alignment.

mirror_top: pass to bdrv_mirror_top_do_write() which is
  64bit

nbd: protocol limitation. max_pdiscard is alredy set strict enough,
  keep it as is for now.

nvme: buf.nlb is uint32_t and we do shift. So, add corresponding limits
  to nvme_refresh_limits().

preallocate: pass to bdrv_co_pdiscard() which is 64bit.

rbd: pass to qemu_rbd_start_co() which is 64bit.

qcow2: calculations are still OK, thanks to bdrv_check_qiov_request(),
  qcow2_cluster_discard() is 64bit.

raw-format: raw_adjust_offset() is 64bit, bdrv_co_pdiscard too.

throttle: pass to bdrv_co_pdiscard() which is 64bit and to
  throttle_group_co_io_limits_intercept() which is 64bit as well.

test-block-iothread: bytes argument is unused

Great! Now all drivers are prepared to handle 64bit discard requests,
or else have explicit max_pdiscard limits.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20210903102807.27127-11-vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
2021-09-29 13:46:32 -05:00
Vladimir Sementsov-Ogievskiy
e75abedab7 block: use int64_t instead of uint64_t in driver write handlers
We are generally moving to int64_t for both offset and bytes parameters
on all io paths.

Main motivation is realization of 64-bit write_zeroes operation for
fast zeroing large disk chunks, up to the whole disk.

We chose signed type, to be consistent with off_t (which is signed) and
with possibility for signed return type (where negative value means
error).

So, convert driver write handlers parameters which are already 64bit to
signed type.

While being here, convert also flags parameter to be BdrvRequestFlags.

Now let's consider all callers. Simple

  git grep '\->bdrv_\(aio\|co\)_pwritev\(_part\)\?'

shows that's there three callers of driver function:

 bdrv_driver_pwritev() and bdrv_driver_pwritev_compressed() in
 block/io.c, both pass int64_t, checked by bdrv_check_qiov_request() to
 be non-negative.

 qcow2_save_vmstate() does bdrv_check_qiov_request().

Still, the functions may be called directly, not only by drv->...
Let's check:

git grep '\.bdrv_\(aio\|co\)_pwritev\(_part\)\?\s*=' | \
awk '{print $4}' | sed 's/,//' | sed 's/&//' | sort | uniq | \
while read func; do git grep "$func(" | \
grep -v "$func(BlockDriverState"; done

shows several callers:

qcow2:
  qcow2_co_truncate() write at most up to @offset, which is checked in
    generic qcow2_co_truncate() by bdrv_check_request().
  qcow2_co_pwritev_compressed_task() pass the request (or part of the
    request) that already went through normal write path, so it should
    be OK

qcow:
  qcow_co_pwritev_compressed() pass int64_t, it's updated by this patch

quorum:
  quorum_co_pwrite_zeroes() pass int64_t and int - OK

throttle:
  throttle_co_pwritev_compressed() pass int64_t, it's updated by this
  patch

vmdk:
  vmdk_co_pwritev_compressed() pass int64_t, it's updated by this
  patch

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20210903102807.27127-5-vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
2021-09-29 13:46:31 -05:00
Vladimir Sementsov-Ogievskiy
f7ef38dd13 block: use int64_t instead of uint64_t in driver read handlers
We are generally moving to int64_t for both offset and bytes parameters
on all io paths.

Main motivation is realization of 64-bit write_zeroes operation for
fast zeroing large disk chunks, up to the whole disk.

We chose signed type, to be consistent with off_t (which is signed) and
with possibility for signed return type (where negative value means
error).

So, convert driver read handlers parameters which are already 64bit to
signed type.

While being here, convert also flags parameter to be BdrvRequestFlags.

Now let's consider all callers. Simple

  git grep '\->bdrv_\(aio\|co\)_preadv\(_part\)\?'

shows that's there three callers of driver function:

 bdrv_driver_preadv() in block/io.c, passes int64_t, checked by
   bdrv_check_qiov_request() to be non-negative.

 qcow2_load_vmstate() does bdrv_check_qiov_request().

 do_perform_cow_read() has uint64_t argument. And a lot of things in
 qcow2 driver are uint64_t, so converting it is big job. But we must
 not work with requests that don't satisfy bdrv_check_qiov_request(),
 so let's just assert it here.

Still, the functions may be called directly, not only by drv->...
Let's check:

git grep '\.bdrv_\(aio\|co\)_preadv\(_part\)\?\s*=' | \
awk '{print $4}' | sed 's/,//' | sed 's/&//' | sort | uniq | \
while read func; do git grep "$func(" | \
grep -v "$func(BlockDriverState"; done

The only one such caller:

    QEMUIOVector qiov = QEMU_IOVEC_INIT_BUF(qiov, &data, 1);
    ...
    ret = bdrv_replace_test_co_preadv(bs, 0, 1, &qiov, 0);

in tests/unit/test-bdrv-drain.c, and it's OK obviously.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20210903102807.27127-4-vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[eblake: fix typos]
Signed-off-by: Eric Blake <eblake@redhat.com>
2021-09-29 13:46:31 -05:00
Cédric Le Goater
4c5fc0c5fc tests/acceptance: Test powernv machines
Fetch the OpenPOWER images to boot the powernv8 and powernv9 machines
with a simple PCI layout.

Cc: Cleber Rosa <crosa@redhat.com>
Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Cc: Wainer dos Santos Moschetta <wainersm@redhat.com>
Signed-off-by: Cédric Le Goater <clg@kaod.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210817093036.1288791-1-clg@kaod.org>
2021-09-27 19:21:37 +02:00
Cleber Rosa
3d2ec56550 tests/acceptance/ppc_prep_40p.py: clean up unused import
Just a removal of an unused imported symbol.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210924185506.2542588-16-crosa@redhat.com>
2021-09-27 19:06:47 +02:00
Cleber Rosa
089f25877f tests/acceptance/ppc_prep_40p.py: NetBSD 7.1.2 location update
The NetBSD-7.1.2-prep.iso is no longer available on the CDN, but it's
still available in the archive.

Let's update its location so that users without the file on cache can
still fetch it and run the test.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210924185506.2542588-15-crosa@redhat.com>
2021-09-27 19:06:47 +02:00
Cleber Rosa
6f1f86cfa7 tests/acceptance/boot_xen.py: removed unused import
Just a clean up for an unused import.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210924185506.2542588-13-crosa@redhat.com>
2021-09-27 19:06:47 +02:00
Cleber Rosa
0f981d8792 acceptance/tests/vnc.py: use explicit syntax for enabling passwords
This matches the command line on 82a17d1d67, where the "on" or "off"
should be explicitly given.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210924185506.2542588-9-crosa@redhat.com>
2021-09-27 19:06:47 +02:00
Cleber Rosa
181e1ab2ad Acceptance Tests: improve check-acceptance description
The "check-acceptance" make rule won't necessarily run *all* available
tests, because it employs a filter based on the currently configured
targets.  This change in the description of the rule makes that
behavior extra clear.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210924185506.2542588-3-crosa@redhat.com>
2021-09-27 19:06:47 +02:00
David Gibson
c5f0a81650 qemu: Split machine_ppc.py acceptance tests
machine_ppc.py contains tests for 3 different ppc based machine types.  It
is listed in MAINTAINERS along with the PPC TCG cpu code.  That's not
really accurate though, since it's really more about testing those machines
than the CPUs.

Therefore, split it up into separate files for the separate machine types,
and list those along with their machine types in MAINTAINERS.

Suggested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210927044808.73391-2-david@gibson.dropbear.id.au>
2021-09-27 19:06:47 +02:00
Willian Rampazzo
94c714620b tests/Makefile: add AVOCADO_TESTS option to make check-acceptance
Add the possibility of running all the tests from a single file, or
multiple files, running a single test within a file or multiple tests
within multiple files using `make check-acceptance` and the
AVOCADO_TESTS environment variable.

Suggested-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210923161141.232208-4-willianr@redhat.com>
2021-09-27 19:06:47 +02:00
Willian Rampazzo
23022794de tests/Makefile: allow control over tags during check-acceptance
Although it is possible to run a specific test using the avocado
command-line, a user may want to use a specific tag while running the
``make check-acceptance`` during the development or debugging.

This allows using the AVOCADO_TAGS environment variable where the user
takes total control of which tests should run based on the tags defined.

This also makes the check-acceptance command flexible to restrict tests
based on tags while running on CI.

e.g.:

AVOCADO_TAGS="foo bar baz" make check-acceptance

Signed-off-by: Willian Rampazzo <willianr@redhat.com>
Tested-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210923161141.232208-2-willianr@redhat.com>
2021-09-27 19:06:47 +02:00
Willian Rampazzo
e519df437a avocado_qemu: fix inheritance order on LinuxTest class
Class hierarchy on Python is defined from right to left. Although the
current code is not harmful, let's fix it to avoid problems in the future.

Signed-off-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210920204932.94132-7-willianr@redhat.com>
2021-09-27 19:06:42 +02:00
Willian Rampazzo
6c58af2e17 avocado_qemu: explicitly return None to avoid R1710
The linter is complaining the `pick_default_qemu_bin` is not explicitly
returning None. Fix it to explicitly return None and avoid R1710
inconsistent-return-statements.

Signed-off-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210920204932.94132-6-willianr@redhat.com>
2021-09-27 19:06:36 +02:00
Willian Rampazzo
22e82e0982 avocado_qemu: tweak ssh connect method
The current implementation will crash if the connection fails as the
`time` module is not imported. Fix the import problem. While here,
tweaks the connection to wait progressively when the connection fails.

Signed-off-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
[PMD: Reworded description]
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210920204932.94132-5-willianr@redhat.com>
2021-09-27 19:06:30 +02:00
Willian Rampazzo
e410bac070 avocado_qemu: fix import module based on isort
Signed-off-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210920204932.94132-4-willianr@redhat.com>
2021-09-27 19:06:25 +02:00
Willian Rampazzo
20bf915418 avocado_qemu: standardize super() call following PEP3135
PEP3135 states when calling super(), there is no need to use arguments.
This changes the calls on avocado_qemu to standardize according to
PEP3135 and avoid warnings from linters.

Signed-off-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210920204932.94132-3-willianr@redhat.com>
2021-09-27 19:06:15 +02:00
Cleber Rosa
1f1fcf0c57 Acceptance Tests: add standard clean up at test tearDown()
The avocado.Test class, used as the basis of the avocado_qemu.Test
class, performs a clean of temporary directories up as part of its own
tearDown() implementation.

But the avocado_qemu.Test class is currently missing the same clean
up, as it implemented its own tearDown() method without resorting to
the upper class behavior.

This brings avocado_qemu.Test behavior in sync with the standard
avocado.Test behavior and prevents temporary directories from
cluttering the test results directory (unless instructed to do so with
Avocado's "--keep-tmp" option).

Reported-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Cleber Rosa <crosa@redhat.com>
[willianr: respin to new Python super format]
Signed-off-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210920204932.94132-2-willianr@redhat.com>
2021-09-27 19:06:15 +02:00
Pavel Dovgalyuk
82184f4018 tests/acceptance: Linux boot test for record/replay
This patch adds a test for record/replay, which boots Linux
image from the disk and interacts with the network.
The idea and code of this test is borrowed from boot_linux.py
This test includes only x86_64 platform. Other platforms and
machines will be added later after testing and improving
record/replay to completely support them.

Each test consists of the following phases:
 - downloading the disk image
 - recording the execution
 - replaying the execution

Replay does not validates the output, but waits until QEMU
finishes the execution. This is reasonable, because
QEMU usually hangs when replay goes wrong.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <162737554047.1735673.13133593401566029378.stgit@pasha-ThinkPad-X280>
2021-09-27 18:55:41 +02:00
Pavel Dovgalyuk
7f7c382a4d tests/acceptance: add replay kernel test for alpha
This patch adds record/replay test which boots Linux
kernel on alpha platform. The test uses kernel binaries
taken from boot_linux_console test.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <162737553482.1735673.10021851966976933952.stgit@pasha-ThinkPad-X280>
2021-09-27 18:51:40 +02:00
Pavel Dovgalyuk
ce9771f5a4 tests/acceptance: add replay kernel test for nios2
This patch adds record/replay test which boots Linux
kernel on nios2 platform. The test uses kernel binaries
taken from boot_linux_console test.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <162737552919.1735673.12493523185952280539.stgit@pasha-ThinkPad-X280>
2021-09-27 18:51:40 +02:00
Pavel Dovgalyuk
27f551135e tests/acceptance: add replay kernel test for openrisc
This patch adds record/replay test which boots Linux
kernel on openrisc platform. The test uses kernel binaries
taken from boot_linux_console test.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <162737552350.1735673.14603125561530143423.stgit@pasha-ThinkPad-X280>
2021-09-27 18:51:40 +02:00
Pavel Dovgalyuk
f7ade77931 tests/acceptance: add replay kernel test for s390x
This patch adds record/replay test which boots Linux
kernel on s390x platform. The test uses kernel binaries
taken from boot_linux_console test.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Acked-by: Thomas Huth <thuth@redhat.com>
[PMD: Drop default '-smp 1' as suggested by Thomas]
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <162737551785.1735673.6775108576116333386.stgit@pasha-ThinkPad-X280>
2021-09-27 18:51:36 +02:00
Markus Armbruster
f333681c6e tests/qapi-schema: Make test-qapi.py -u work when files are absent
test-qapi.py -u updates the expected files.  Since it fails when they
are absent, users have to create them manually before they can use
test-qapi.py to fill in the contents, say for a new test.  Silly.
Improve -u to create them.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210922125619.670673-3-armbru@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
2021-09-27 08:23:25 +02:00
Markus Armbruster
436911c2a1 tests/qapi-schema: Use Python OSError instead of outmoded IOError
https://docs.python.org/3.6/library/exceptions.html has

    Changed in version 3.3: EnvironmentError, IOError, WindowsError,
    socket.error, select.error and mmap.error have been merged into
    OSError, and the constructor may return a subclass.

and

    The following exceptions are kept for compatibility with previous
    versions; starting from Python 3.3, they are aliases of OSError.

    exception EnvironmentError

    exception IOError

    exception WindowsError

        Only available on Windows.

Switch to the preferred name.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210922125619.670673-2-armbru@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
[Details added to commit message]
2021-09-27 08:23:25 +02:00
Markus Armbruster
f90ae4d104 test-clone-visitor: Correct an accidental rename
Commit b359f4b203 "tests: Rename UserDefNativeListUnion to
UserDefListUnion" renamed test_clone_native_list() to
test_clone_list_union().  The function has nothing to do with unions.
Rename it to test_clone_list().

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210917143134.412106-24-armbru@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2021-09-27 08:23:25 +02:00
Markus Armbruster
8b3b3a16df tests/qapi-schema: Rename flat-union-* test cases to union-*
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210917143134.412106-23-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com
2021-09-27 08:23:25 +02:00
Markus Armbruster
4e99f4b12c qapi: Drop simple unions
Simple unions predate flat unions.  Having both complicates the QAPI
schema language and the QAPI generator.  We haven't been using simple
unions in new code for a long time, because they are less flexible and
somewhat awkward on the wire.

The previous commits eliminated simple union from the tree.  Now drop
them from the QAPI schema language entirely, and update mentions of
"flat union" to just "union".

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210917143134.412106-22-armbru@redhat.com>
2021-09-27 08:23:25 +02:00
Markus Armbruster
76432d988b tests/qapi-schema: Purge simple unions from tests
Drop tests that are specifically about simple unions:

* SugaredUnion in doc-good: flat unions are covered by @Object.

* union-branch-case and union-clash-branches: branch naming for flat
  unions is enforced for the tag enum instead, which is covered by
  enum-member-case and enum-clash-member.

* union-empty: empty flat unions are covered by flat-union-empty.

Rewrite the remainder to use flat unions: args-union, bad-base,
flat-union-base-union, union-branch-invalid-dict, union-unknown.

Except drop union-optional-branch. because converting this one is not
worth the trouble; we don't explicitly check names beginning with '*'
in other places, either.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210917143134.412106-21-armbru@redhat.com>
2021-09-27 08:23:25 +02:00
Markus Armbruster
bb5821dd81 tests/qapi-schema: Drop simple union __org.qemu_x-Union1
Replace simple union __org.qemu_x-Union1 with flat union
__org.qemu_x-Union2, except drop it from __org.qemu_x-command, because
there it's only used to pull it into QMP.  Now drop the unused
-Union1, and rename -Union2 to -Union.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210917143134.412106-20-armbru@redhat.com>
2021-09-27 08:23:25 +02:00
Markus Armbruster
7a22dc17ac test-clone-visitor: Wean off __org.qemu_x-Union1
test_clone_complex3() uses simple union __org.qemu_x-Union1 to cover
arrays.  Use UserDefOneList instead.  Unions are still covered by
test_clone_complex1().

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210917143134.412106-19-armbru@redhat.com>
2021-09-27 08:23:25 +02:00
Markus Armbruster
5450186337 tests/qapi-schema: Rewrite simple union TestIfUnion to be flat
Simple unions predate flat unions.  Having both complicates the QAPI
schema language and the QAPI generator.  We haven't been using simple
unions in new code for a long time, because they are less flexible and
somewhat awkward on the wire.

To prepare for their removal, rewrite TestIfUnion to be flat.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210917143134.412106-18-armbru@redhat.com>
2021-09-27 08:23:25 +02:00
Markus Armbruster
def1d1f571 tests/qapi-schema: Simple union UserDefListUnion is now unused, drop
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210917143134.412106-17-armbru@redhat.com>
2021-09-27 08:23:25 +02:00
Markus Armbruster
1e65e16ca3 tests/qapi-schema: Wean off UserDefListUnion
Command boxed-union uses simple union UserDefListUnion to cover
unions.  Use UserDefFlatUnion instead.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210917143134.412106-16-armbru@redhat.com>
2021-09-27 08:23:25 +02:00
Markus Armbruster
16821fc85b test-clone-visitor: Wean off UserDefListUnion
test_clone_complex1() uses simple union UserDefListUnion to cover
unions.  Use UserDefFlatUnion instead.  Arrays are still covered by
test_clone_complex3().

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210917143134.412106-15-armbru@redhat.com>
2021-09-27 08:23:25 +02:00
Markus Armbruster
00e6832f41 test-qobject-output-visitor: Wean off UserDefListUnion
The test_visitor_out_list_union_FOO() use simple union
UserDefListUnion to cover lists of builtin types.  Rewrite as
test_visitor_out_list_struct(), using struct ArrayStruct and a lot
less code.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210917143134.412106-14-armbru@redhat.com>
2021-09-27 08:23:25 +02:00
Markus Armbruster
e7a76fe25a test-qobject-input-visitor: Wean off UserDefListUnion
The test_visitor_in_list_union_FOO() use simple union UserDefListUnion
to cover lists of builtin types.  Rewrite as
test_visitor_in_list_struct(), using struct ArrayStruct and a lot less
code.

test_visitor_in_fail_union_list() uses UserDefListUnion to cover
"variant members don't match the discriminator value".  Cover that in
test_visitor_in_fail_union_flat() instead, and drop
test_visitor_in_fail_union_list().  Appropriating the former for this
purpose is okay, because it actually failed due to missing
discriminator, which is still covered by
test_visitor_in_fail_union_flat_no_discrim().

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210917143134.412106-13-armbru@redhat.com>
2021-09-27 08:23:25 +02:00
Markus Armbruster
dffd8ff9e6 tests/qapi-schema: Prepare for simple union UserDefListUnion removal
Simple unions predate flat unions.  Having both complicates the QAPI
schema language and the QAPI generator.  We haven't been using simple
unions in new code for a long time, because they are less flexible and
somewhat awkward on the wire.

To prepare for their removal, simple union UserDefListUnion has to go.
It is used to cover arrays.  The next few commits will eliminate its
uses, and then it gets deleted.  As a first step, provide struct
ArrayStruct for the tests to be rewritten.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210917143134.412106-12-armbru@redhat.com>
2021-09-27 08:23:25 +02:00
Markus Armbruster
935a867c87 qapi: Convert simple union SocketAddressLegacy to flat one
Simple unions predate flat unions.  Having both complicates the QAPI
schema language and the QAPI generator.  We haven't been using simple
unions in new code for a long time, because they are less flexible and
somewhat awkward on the wire.

To prepare for their removal, convert simple union SocketAddressLegacy
to an equivalent flat one, with existing enum SocketAddressType
replacing implicit enum type SocketAddressLegacyKind.  Adds some
boilerplate to the schema, which is a bit ugly, but a lot easier to
maintain than the simple union feature.

Cc: "Daniel P. Berrangé" <berrange@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210917143134.412106-9-armbru@redhat.com>
2021-09-27 08:23:25 +02:00
Markus Armbruster
8ebc3120e1 qapi: Stop enforcing "type name should not end in 'Kind'
I'm about to convert simple unions to flat unions, then drop simple
union support.  The conversion involves making the implict enum types
explicit.  To reduce churn, I'd like to name them exactly like the
implicit types they replace.  However, these names are reserved for
the generator's use.  They won't be once simple unions are gone.  Stop
enforcing this naming rule now rather than then.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210917143134.412106-3-armbru@redhat.com>
2021-09-25 06:55:54 +02:00
Markus Armbruster
4cfd6537e4 qapi: Tidy up unusual line breaks
Break lines between members instead of within members.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20210917143134.412106-2-armbru@redhat.com>
2021-09-25 06:55:54 +02:00
Eric Blake
1899bf4737 qemu-img: Add -F shorthand to convert
Although we have long supported 'qemu-img convert -o
backing_file=foo,backing_fmt=bar', the fact that we have a shortcut -B
for backing_file but none for backing_fmt has made it more likely that
users accidentally run into:

qemu-img: warning: Deprecated use of backing file without explicit backing format

when using -B instead of -o.  For similarity with other qemu-img
commands, such as create and compare, add '-F $fmt' as the shorthand
for '-o backing_fmt=$fmt'.  Update iotest 122 for coverage of both
spellings.

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210913131735.1948339-1-eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
2021-09-15 18:42:38 +02:00
Vladimir Sementsov-Ogievskiy
ff812c5563 qcow2: handle_dependencies(): relax conflict detection
There is no conflict and no dependency if we have parallel writes to
different subclusters of one cluster when the cluster itself is already
allocated. So, relax extra dependency.

Measure performance:
First, prepare build/qemu-img-old and build/qemu-img-new images.

cd scripts/simplebench
./img_bench_templater.py

Paste the following to stdin of running script:

qemu_img=../../build/qemu-img-{old|new}
$qemu_img create -f qcow2 -o extended_l2=on /ssd/x.qcow2 1G
$qemu_img bench -c 100000 -d 8 [-s 2K|-s 2K -o 512|-s $((1024*2+512))] \
        -w -t none -n /ssd/x.qcow2

The result:

All results are in seconds

------------------  ---------  ---------
                    old        new
-s 2K               6.7 ± 15%  6.2 ± 12%
                                 -7%
-s 2K -o 512        13 ± 3%    11 ± 5%
                                 -16%
-s $((1024*2+512))  9.5 ± 4%   8.4
                                 -12%
------------------  ---------  ---------

So small writes are more independent now and that helps to keep deeper
io queue which improves performance.

271 iotest output becomes racy for three allocation in one cluster.
Second and third writes may finish in different order. Second and
third requests don't depend on each other any more. Still they both
depend on first request anyway. Filter out second and third write
offsets to cover both possible outputs.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20210824101517.59802-4-vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
[hreitz: s/ an / and /]
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
2021-09-15 15:54:07 +02:00
Vladimir Sementsov-Ogievskiy
a13de40a05 block: bdrv_inactivate_recurse(): check for permissions and fix crash
We must not inactivate child when parent has write permissions on
it.

Calling .bdrv_inactivate() doesn't help: actually only qcow2 has this
handler and it is used to flush caches, not for permission
manipulations.

So, let's simply check cumulative parent permissions before
inactivating the node.

This commit fixes a crash when we do migration during backup: prior to
the commit nothing prevents all nodes inactivation at migration finish
and following backup write to the target crashes on assertion
"assert(!(bs->open_flags & BDRV_O_INACTIVE));" in
bdrv_co_write_req_prepare().

After the commit, we rely on the fact that copy-before-write filter
keeps write permission on target node to be able to write to it. So
inactivation fails and migration fails as expected.

Corresponding test now passes, so, enable it.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20210911120027.8063-3-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
2021-09-15 15:54:07 +02:00
Vladimir Sementsov-Ogievskiy
2f43482733 tests: add migrate-during-backup
Add a simple test which tries to run migration during backup.
bdrv_inactivate_all() should fail. But due to bug (see next commit with
fix) it doesn't, nodes are inactivated and continued backup crashes
on assertion "assert(!(bs->open_flags & BDRV_O_INACTIVE));" in
bdrv_co_write_req_prepare().

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20210911120027.8063-2-vsementsov@virtuozzo.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
2021-09-15 15:54:07 +02:00
Hanna Reitz
098d983ea5 iotests/297: Cover tests/
297 so far does not check the named tests, which reside in the tests/
directory (i.e. full path tests/qemu-iotests/tests).  Fix it.

Thanks to the previous two commits, all named tests pass its scrutiny,
so we do not have to add anything to SKIP_FILES.

Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20210902094017.32902-6-hreitz@redhat.com>
2021-09-15 15:54:07 +02:00