Commit Graph

84388 Commits

Author SHA1 Message Date
Eric Blake
c3033fd372 qapi: Use QAPI_LIST_APPEND in trivial cases
The easiest spots to use QAPI_LIST_APPEND are where we already have an
obvious pointer to the tail of a list.  While at it, consistently use
the variable name 'tail' for that purpose.

Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210113221013.390592-5-eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-01-28 08:08:45 +01:00
Eric Blake
dc13f40c6b qapi: Introduce QAPI_LIST_APPEND
Similar to the existing QAPI_LIST_PREPEND, but designed for use where
we want to preserve insertion order.  Callers will be added in
upcoming patches.  Note the difference in signature: PREPEND takes
List*, APPEND takes List**.

Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20210113221013.390592-4-eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-01-28 08:08:45 +01:00
Eric Blake
240ee8bd31 qapi: A couple more QAPI_LIST_PREPEND() stragglers
Commit 54aa3de72e switched multiple sites to use QAPI_LIST_PREPEND
instead of open-coding, but missed a couple of spots.

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210113221013.390592-3-eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-01-28 08:08:45 +01:00
Eric Blake
e9d635ea18 net: Clarify early exit condition
On first glance, the loop in qmp_query_rx_filter() has early return
paths that could leak any allocation of filter_list from a previous
iteration.  But on closer inspection, it is obvious that all of the
early exits are guarded by has_name, and that the bulk of the loop
body can be executed at most once if the user is filtering by name,
thus, any early exit coincides with an empty list.  Add asserts to
make this obvious.

Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210113221013.390592-2-eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-01-28 08:08:45 +01:00
Vladimir Sementsov-Ogievskiy
a44be0334b iotests: rename and move 169 and 199 tests
Rename bitmaps migration tests and move them to tests subdirectory to
demonstrate new human-friendly test naming.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20210125185056.129513-7-vsementsov@virtuozzo.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2021-01-27 20:53:14 +01:00
Vladimir Sementsov-Ogievskiy
f203080bbd iotests: rewrite check into python
Just use classes introduced in previous three commits. Behavior
difference is described in these three commits.

Drop group file, as it becomes unused.

Drop common.env: now check is in python, and for tests we use same
python interpreter that runs the check itself. Use build environment
PYTHON in check-block instead, to keep "make check" use the same
python.

Checking for virtio-blk moved to iotests.py, as it actually iotests.py
dependency. Actually not all python iotests depend on it, so in future
it may be refactored to checked only when really needed.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20210125185056.129513-6-vsementsov@virtuozzo.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2021-01-27 20:53:14 +01:00
Vladimir Sementsov-Ogievskiy
d74c754c92 iotests: add testrunner.py
Add TestRunner class, which will run tests in a new python iotests
running framework.

There are some differences with current ./check behavior, most
significant are:
- Consider all tests self-executable, just run them, don't run python
  by hand.
- Elapsed time is cached in json file
- Elapsed time precision increased a bit
- Instead of using "diff -w" which ignores all whitespace differences,
  manually strip whitespace at line end then use python difflib, which
  no longer ignores spacing mid-line

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20210125185056.129513-5-vsementsov@virtuozzo.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2021-01-27 20:53:14 +01:00
Vladimir Sementsov-Ogievskiy
2e5a2f57db iotests: add testenv.py
Add TestEnv class, which will handle test environment in a new python
iotests running framework.

Don't add compat=1.1 for qcow2 IMGOPTS, as v3 is default anyway.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20210125185056.129513-4-vsementsov@virtuozzo.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2021-01-27 20:53:14 +01:00
Vladimir Sementsov-Ogievskiy
b25a948875 iotests: add findtests.py
Add python script with new logic of searching for tests:

Current ./check behavior:
 - tests are named [0-9][0-9][0-9]
 - tests must be registered in group file (even if test doesn't belong
   to any group, like 142)

Behavior of findtests.py:
 - group file is dropped
 - tests are all files in tests/ subdirectory (except for .out files),
   so it's not needed more to "register the test", just create it with
   appropriate name in tests/ subdirectory. Old names like
   [0-9][0-9][0-9] (in root iotests directory) are supported too, but
   not recommended for new tests
 - groups are parsed from '# group: ' line inside test files
 - optional file group.local may be used to define some additional
   groups for downstreams
 - 'disabled' group is used to temporary disable tests. So instead of
   commenting tests in old 'group' file you now can add them to
   disabled group with help of 'group.local' file
 - selecting test ranges like 5-15 are not supported more
   (to support restarting failed ./check command from the middle of the
    process, new argument is added: --start-from)

Benefits:
 - no rebase conflicts in group file on patch porting from branch to
   branch
 - no conflicts in upstream, when different series want to occupy same
   test number
 - meaningful names for test files
   For example, with digital number, when some person wants to add some
   test about block-stream, he most probably will just create a new
   test. But if there would be test-block-stream test already, he will
   at first look at it and may be just add a test-case into it.
   And anyway meaningful names are better.

This commit doesn't update check behavior (which will be done in
further commit), still, the documentation changed like new behavior is
already here.  Let's live with this small inconsistency for the
following few commits, until final change.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210125185056.129513-3-vsementsov@virtuozzo.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2021-01-27 20:53:14 +01:00
Vladimir Sementsov-Ogievskiy
362ef77f9b iotests: 146: drop extra whitespaces from .out file
check script will be stricter soon about whitespaces, so fix 146.out
now.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20210125185056.129513-2-vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2021-01-27 20:53:02 +01:00
Kevin Wolf
720d10d67b virtio-scsi-test: Test writing to scsi-cd device
This tests that trying to write to a (read-only) scsi-cd device backed
by a read-write image file doesn't crash and results in the correct
error.

This is a regression test for https://bugs.launchpad.net/bugs/1906693.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20210118123448.307825-3-kwolf@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2021-01-27 20:45:20 +01:00
Kevin Wolf
86b1cf3227 block: Separate blk_is_writable() and blk_supports_write_perm()
Currently, blk_is_read_only() tells whether a given BlockBackend can
only be used in read-only mode because its root node is read-only. Some
callers actually try to answer a slightly different question: Is the
BlockBackend configured to be writable, by taking write permissions on
the root node?

This can differ, for example, for CD-ROM devices which don't take write
permissions, but may be backed by a writable image file. scsi-cd allows
write requests to the drive if blk_is_read_only() returns false.
However, the write request will immediately run into an assertion
failure because the write permission is missing.

This patch introduces separate functions for both questions.
blk_supports_write_perm() answers the question whether the block
node/image file can support writable devices, whereas blk_is_writable()
tells whether the BlockBackend is currently configured to be writable.

All calls of blk_is_read_only() are converted to one of the two new
functions.

Fixes: https://bugs.launchpad.net/bugs/1906693
Cc: qemu-stable@nongnu.org
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20210118123448.307825-2-kwolf@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2021-01-27 20:45:20 +01:00
Peter Maydell
af47e82269 * Patches to speed up and improve the gitlab-CI
* Documentation for the decorators in the "acceptance" tests
 * One small rework of a libqtest function
 -----BEGIN PGP SIGNATURE-----
 
 iQJFBAABCAAvFiEEJ7iIR+7gJQEY8+q5LtnXdP5wLbUFAmARBpMRHHRodXRoQHJl
 ZGhhdC5jb20ACgkQLtnXdP5wLbXrmg//ap1z2taLpgUdMVxImdMieqsfzSxMUt75
 vkmQvk+rquK8VqwVKGaMsps7aANXDFD2etjAx0zp/9uS6OApNRegRFNb5eOr2oUR
 RBhnNYkabmAvabn7lFmb2FZ84cCA3c3rY0JsZ44eYcdOOQVLuKMH6//m7gPztmkA
 BNYh8riAmOTVjL81cJVmA/hKUO7P1eR2+604gRr4t7mP9/I72QG3f12o062c+dWA
 xXNK8gtqSjgfofZeqU8HYHkJoyi/3N6aGHE4XmojPbE0fzj1gjxlHm0LpIhrpwbI
 hxkj3fwVh2e89dhMXCYhFKrYjUEQuJPUdLmMRl2KcD9AmNFtM3S+WSi6SX8m9CXR
 FBYpz5zOk7HHDZKJZ17U0xr5NZ8E0iULc0uLAVm5BqkmiT3DqjaWb0JYe/4rUT7Y
 XxOX7MHA5AByRfv6IBfKCmRPUXM16W4tvWYHsCzU7ViV7eOxNLl4yY+3YvyfigHf
 5MENOAkr3kdL54sDOsARCLs8D6zyXyKyspt13SAoRg7cGxS3WpSKYVKxYeX+VXbs
 S/mUoKFT1IE8PkiDnQzw5PypPxgc3700AlrIAk9ZQiWol5tfq8QKBf4a9LSpV5Uu
 02OlI3ufkdZwALgw9ZJNsSR0hX/rcALwOxlQ4kBcWsT45kxwch0337H+U6bUVnca
 AJpJfDzlXI0=
 =CNCA
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/huth-gitlab/tags/pull-request-2021-01-27' into staging

* Patches to speed up and improve the gitlab-CI
* Documentation for the decorators in the "acceptance" tests
* One small rework of a libqtest function

# gpg: Signature made Wed 27 Jan 2021 06:22:11 GMT
# gpg:                using RSA key 27B88847EEE0250118F3EAB92ED9D774FE702DB5
# gpg:                issuer "thuth@redhat.com"
# gpg: Good signature from "Thomas Huth <th.huth@gmx.de>" [full]
# gpg:                 aka "Thomas Huth <thuth@redhat.com>" [full]
# gpg:                 aka "Thomas Huth <huth@tuxfamily.org>" [full]
# gpg:                 aka "Thomas Huth <th.huth@posteo.de>" [unknown]
# Primary key fingerprint: 27B8 8847 EEE0 2501 18F3  EAB9 2ED9 D774 FE70 2DB5

* remotes/huth-gitlab/tags/pull-request-2021-01-27:
  libqtest: Rework qtest_rsp()
  docs/devel: Explain how acceptance tests can be skipped
  gitlab-ci.yml: Avoid recompiling the sources in the test jobs
  gitlab-ci.yml: Exclude some redundant targets in build-without-default-features
  meson: Do not build optional libraries by default
  configure: Only check for audio drivers if system-mode is selected
  gitlab-ci.yml: Avoid some submodules to speed up the CI a little bit
  gitlab-ci: Test building linux-user targets on CentOS 7
  tests/docker: Install static libc package in CentOS 7

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-01-27 19:02:03 +00:00
Peter Maydell
bf159f0bdc For upstream
-----BEGIN PGP SIGNATURE-----
 
 iQEzBAABCgAdFiEErET+3BT38evtv0FRKcWWeA9ryoMFAmARGSAACgkQKcWWeA9r
 yoOxjAgA2lR3ENUw8nVNsgV7CnEz3o5SCC078iPd9YPGYqEKUs+NCKRAC/KJJ8zq
 wkyPoitxXMPHUMEv8F34t1p6Im3Q4sGCkNYI0fH2eDZ4rz6lwfa2mbom2DhEV8wi
 uyt53iqfPLpcHN7EUAy1yz29KULJ74ot9rNlwIuDjhhzItvmO5+IInWKFXlNdceE
 cFC875kWv6wQH20nxx5fis9sgGcZI9OMXoAmTcJIUhSTb0duUF016AZzlg71lbyG
 AlFa8COTsd+6DWTjSEJw7XPpTNBtntPwW1vJXo+RKUDxz9T1xc3TS9edy5+1SF1w
 SOM8vPDXkEuRfISB5XP/w7gdPzMfWA==
 =Icjb
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/edgar/tags/edgar/xilinx-next-2021-01-27.for-upstream' into staging

For upstream

# gpg: Signature made Wed 27 Jan 2021 07:41:20 GMT
# gpg:                using RSA key AC44FEDC14F7F1EBEDBF415129C596780F6BCA83
# gpg: Good signature from "Edgar E. Iglesias (Xilinx key) <edgar.iglesias@xilinx.com>" [unknown]
# gpg:                 aka "Edgar E. Iglesias <edgar.iglesias@gmail.com>" [full]
# Primary key fingerprint: AC44 FEDC 14F7 F1EB EDBF  4151 29C5 9678 0F6B CA83

* remotes/edgar/tags/edgar/xilinx-next-2021-01-27.for-upstream:
  target/microblaze: Add security attributes on memory transactions
  target/microblaze: use MMUAccessType instead of int in mmu_translate
  target/microblaze: Add use-non-secure property

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-01-27 17:40:25 +00:00
Philippe Mathieu-Daudé
5fa6ab7ecc tcg/tci: Restrict tci_write_reg16() to 64-bit hosts
Restrict tci_write_reg16() to 64-bit hosts to fix on 32-bit ones:

  [520/1115] Compiling C object libqemu-arm-linux-user.fa.p/tcg_tci.c.o
  FAILED: libqemu-arm-linux-user.fa.p/tcg_tci.c.o
  tcg/tci.c:132:1: error: 'tci_write_reg16' defined but not used [-Werror=unused-function]
   tci_write_reg16(tcg_target_ulong *regs, TCGReg index, uint16_t value)
   ^~~~~~~~~~~~~~~

Fixes: 2f160e0f97 ("tci: Add implementation for INDEX_op_ld16u_i64")
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Stefan Weil <sw@weilnetz.de>
Message-Id: <20210123094107.2340222-1-f4bug@amsat.org>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
2021-01-27 17:15:40 +01:00
Greg Kurz
1a00195335 MAINTAINERS: Update 9pfs tree URL
I've already moved my repositories to gitlab for extra CI coverage,
and I won't use the ones at github anymore.

Signed-off-by: Greg Kurz <groug@kaod.org>
Reviewed-by: Christian Schoenebeck <qemu_oss@crudebyte.com>
Message-Id: <161071814430.152031.14540382419012818908.stgit@bahia.lan>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
2021-01-27 17:07:01 +01:00
Peter Maydell
15b08119ad hw/display/vmware_vga: Remove dependency on VNC header
In commit 2f487a3d40 we fixed a problem observed with using the
vmware-vga device and the VNC UI frontend in a belt-and-braces
manner:
 * we made the VNC frontend handle non-multiple-of-16 surface widths
 * we rounded up the vmware-vga display width to a multiple of 16

However this introduced a spurious dependency of a device model on a
UI frontend header.  vmware-vga isn't special and should not care
about what UI frontend it is using, and the VNC frontend needs to
handle arbitrary surface widths because other display device models
could use them.  Moreover, even if the maximum width in vmware-vga is
made a multiple of 16, the guest itself can always program a
different width.

Remove the dependency on the VNC header.  Since we have been using
the rounded-up width value since 2014, stick with it rather than
introducing a behaviour change, but don't calculate it by rounding up
to VNC_DIRTY_BITS_PER_PIXEL any more.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Message-Id: <20210112161608.16055-1-peter.maydell@linaro.org>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
2021-01-27 09:48:04 +01:00
Gerd Hoffmann
104b8d1932 vnc: send extended desktop resize on update requests
Unlike other pseudo-encodings these don't break gtk-vnc
because older versions don't suport the extended desktop
resize extension in the first place.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Tested-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20210125104041.495274-3-kraxel@redhat.com>
2021-01-27 09:47:07 +01:00
Gerd Hoffmann
d239726c53 Revert "vnc: move initialization to framebuffer_update_request"
This reverts commit 9e1632ad07.

Older gtk-vnc versions can't deal with non-incremental update
requests sending pseudo-encodings, so trying to send full server
state (including desktop size, cursor etc. which is done using
pseudo-encodings) doesn't fly.  Return to old behavior to send
those only for new connects and when changes happen.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Tested-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20210125104041.495274-2-kraxel@redhat.com>
2021-01-27 09:47:02 +01:00
Joe Komlodi
43a9ede1ef target/microblaze: Add security attributes on memory transactions
Using the cfg.use_non_secure bitfield and the MMU access type, we can determine
if the access should be secure or not.

Signed-off-by: Joe Komlodi <komlodi@xilinx.com>
Reviewed-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com>
Tested-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com>
Message-Id: <1611274735-303873-4-git-send-email-komlodi@xilinx.com>
Signed-off-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com>
2021-01-27 08:32:55 +01:00
Joe Komlodi
671a0a1265 target/microblaze: use MMUAccessType instead of int in mmu_translate
Using MMUAccessType makes it more clear what the variable's use is.
No functional change.

Signed-off-by: Joe Komlodi <komlodi@xilinx.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com>
Tested-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com>
Message-Id: <1611274735-303873-3-git-send-email-komlodi@xilinx.com>
Signed-off-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com>
2021-01-27 08:32:55 +01:00
Joe Komlodi
ea2ccb6561 target/microblaze: Add use-non-secure property
This property is used to control the security of the following interfaces
on MicroBlaze:
M_AXI_DP - data interface
M_AXI_IP - instruction interface
M_AXI_DC - dcache interface
M_AXI_IC - icache interface

It works by enabling or disabling the use of the non_secure[3:0] signals.

Interfaces and their corresponding values are taken from:
https://www.xilinx.com/support/documentation/sw_manuals/xilinx2020_2/ug984-vivado-microblaze-ref.pdf
page 153.

Signed-off-by: Joe Komlodi <komlodi@xilinx.com>
Reviewed-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com>
Tested-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com>
Message-Id: <1611274735-303873-2-git-send-email-komlodi@xilinx.com>
Signed-off-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com>
2021-01-27 08:32:55 +01:00
Markus Armbruster
f8a9b4c665 libqtest: Rework qtest_rsp()
qtest_rsp() is used in two different ways: (1) return some arguments
to caller, which the caller must free, and (2) return no arguments to
caller.  Passing non-zero @expected_args gets you (1), and passing
zero gets you (2).

Having "the return value must be freed" depend on an argument this way
is less than ideal.

Provide separate functions for the two ways: (1) qtest_rsp_args()
takes @expected_args (possibly zero), and returns that number of
arguments.  Caller must free the return value always.  (2) qtest_rsp()
assumes zero, and returns nothing.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20210126151649.2220902-1-armbru@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-01-27 07:18:13 +01:00
Wainer dos Santos Moschetta
cf5891ec53 docs/devel: Explain how acceptance tests can be skipped
Documented under the "Acceptance tests using the Avocado Framework"
section in testing.rst how environment variables are used to skip tests.

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20210115210022.417996-1-wainersm@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-01-27 04:47:40 +01:00
Thomas Huth
12a917a14f gitlab-ci.yml: Avoid recompiling the sources in the test jobs
Currently, our check-system-* jobs are recompiling the whole sources
again. This happens due to the fact that the jobs are checking out
the whole source tree and required submodules again, and only try
to use the "build" directory with the binaries and object files as an
artifact from the previous stage - which simply does not work right
anymore (with the current version of meson). Due to some changed
time stamps, meson/ninja are always trying to rebuild the whole tree.

In the long run, we could likely use "meson test --no-rebuild", but
there is still some work going on in that area to improve the user
experience. So until this has been done, simply avoid recompiling the
sources with a trick: pass NINJA=":" to the make process in the test
jobs. Also check out the submodules manually before updating the
timestamps in the build folder, so that the binaries are definitely
newer that all the source files.
This saves ca. 10 - 15 minutes of precious CI cycles in each run.

Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20210126065757.403853-1-thuth@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-01-27 04:47:40 +01:00
Thomas Huth
8cd95aec4c gitlab-ci.yml: Exclude some redundant targets in build-without-default-features
The build-without-default-features job is running quite long and sometimes
already hits the 1h time limit. Exclude some targets which do not provide
additional test coverage here (since we e.g. also already test other targets
of the same type, just with different endianess, or a 64-bit superset) to
avoid that we hit the timeout here so easily.

Message-Id: <20210126172345.15947-1-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-01-27 04:47:13 +01:00
Peter Maydell
565c86af51 Block patches:
- Make backup block jobs use asynchronous requests with the block-copy
   module
 - Use COR filter node for stream block jobs
 - Make coroutine-sigaltstack’s qemu_coroutine_new() function thread-safe
 - Report error string when file locking fails with an unexpected errno
 - iotest fixes, additions, and some refactoring
 -----BEGIN PGP SIGNATURE-----
 
 iQFGBAABCAAwFiEEkb62CjDbPohX0Rgp9AfbAGHVz0AFAmAQJFsSHG1yZWl0ekBy
 ZWRoYXQuY29tAAoJEPQH2wBh1c9AWZEIALwZwQU82RE+gezeDK7XPqBYuGMOVzbJ
 wdvQLnAx+vEo2idLlWm4RE7GhibhTB5iRaiIGmpZhvja/+2BrZHve4GHD5+GBolF
 QvLoIdfkPqhORhuAMc1+8rkP4yqv1bLBuOB6TLh15Bw6B6Q7clQbDGqubK2HIwgb
 h/RcmMj/93ZN27WM589Zdk5i/1kE11sOyCKNP+UQ3hnKz7ecqtY7pHYvFS3Q9e6l
 xpc/QyFhTw/OHfaUViL6W0MWhdqhf/mBlXqzvFoqVWEfdJs+56E1BeUsHRUSxTjs
 2v6y7AC3s9gZC/9Icpui3mXvxAQQPTsaF0UfjEPiW+q+NiC07FtdqHo=
 =S1Dc
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2021-01-26' into staging

Block patches:
- Make backup block jobs use asynchronous requests with the block-copy
  module
- Use COR filter node for stream block jobs
- Make coroutine-sigaltstack’s qemu_coroutine_new() function thread-safe
- Report error string when file locking fails with an unexpected errno
- iotest fixes, additions, and some refactoring

# gpg: Signature made Tue 26 Jan 2021 14:16:59 GMT
# gpg:                using RSA key 91BEB60A30DB3E8857D11829F407DB0061D5CF40
# gpg:                issuer "mreitz@redhat.com"
# gpg: Good signature from "Max Reitz <mreitz@redhat.com>" [full]
# Primary key fingerprint: 91BE B60A 30DB 3E88 57D1  1829 F407 DB00 61D5 CF40

* remotes/maxreitz/tags/pull-block-2021-01-26: (53 commits)
  iotests/178: Pass value to invalid option
  iotests/118: Drop 'change' test
  iotests: Add test for the regression fixed in c8bf9a9169
  block: report errno when flock fcntl fails
  simplebench: add bench-backup.py
  simplebench: bench_block_job: add cmd_options argument
  simplebench/bench_block_job: use correct shebang line with python3
  block/block-copy: drop unused argument of block_copy()
  block/block-copy: drop unused block_copy_set_progress_callback()
  qapi: backup: disable copy_range by default
  backup: move to block-copy
  block/backup: drop extra gotos from backup_run()
  block/block-copy: make progress_bytes_callback optional
  iotests: 257: prepare for backup over block-copy
  iotests: 219: prepare for backup over block-copy
  iotests: 185: prepare for backup over block-copy
  iotests/129: Limit backup's max-chunk/max-workers
  iotests: 56: prepare for backup over block-copy
  qapi: backup: add max-chunk and max-workers to x-perf struct
  job: call job_enter from job_pause
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-01-26 21:08:13 +00:00
Philippe Mathieu-Daudé
610e7e0e69 meson: Do not build optional libraries by default
The following libraries will be selected if a feature requires it:

- capstone
- fdt
- SLiRP

Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210122204441.2145197-5-philmd@redhat.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-01-26 18:38:37 +01:00
Philippe Mathieu-Daudé
90520ee4ca configure: Only check for audio drivers if system-mode is selected
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
Message-Id: <20210122204441.2145197-2-philmd@redhat.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-01-26 18:38:37 +01:00
Thomas Huth
db0108d5d8 gitlab-ci.yml: Avoid some submodules to speed up the CI a little bit
Since the meson build system rework, the configure script prefers the
git submodules over the system libraries. So we are testing compilation
with capstone, fdt and libslirp as a submodule all over the place,
burning CPU cycles by recompiling these third party modules and wasting
some network bandwidth in the CI by cloning the submodules each time.
Let's stop doing that in at least a couple of jobs and use the system
libraries instead.

While we're at it, also install meson in the Fedora container, since
it is new enough already, so we do not need to check out the meson
submodule here.

Message-Id: <20210121174451.658924-1-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-01-26 18:38:37 +01:00
Philippe Mathieu-Daudé
4e41d4a34e gitlab-ci: Test building linux-user targets on CentOS 7
Add a configuration tested by Peter Maydell (see [1] and [2])
but not covered in our CI [3]:

  [705/2910] Compiling C object libqemu-arm-linux-user.fa.p/linux-user_strace.c.o
  FAILED: libqemu-arm-linux-user.fa.p/linux-user_strace.c.o
  ../linux-user/strace.c: In function 'do_print_sockopt':
  ../linux-user/strace.c:2831:14: error: 'IPV6_ADDR_PREFERENCES' undeclared (first use in this function)
           case IPV6_ADDR_PREFERENCES:
                ^

This job currently takes 31 minutes 32 seconds ([4]).

[1] https://lists.gnu.org/archive/html/qemu-devel/2021-01/msg05086.html
[2] https://lists.gnu.org/archive/html/qemu-devel/2021-01/msg05379.html
[3] https://gitlab.com/philmd/qemu/-/jobs/977408284
[4] https://gitlab.com/philmd/qemu/-/jobs/978223286

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20210121172829.1643620-3-f4bug@amsat.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-01-26 15:39:34 +01:00
Philippe Mathieu-Daudé
4762c82cbd tests/docker: Install static libc package in CentOS 7
We need to install the static libc package to be able to run
the TCG tests:

  $ make check-tcg
  ...
    BUILD   TCG tests for x86_64-softmmu
    BUILD   x86_64-softmmu guest-tests with cc
  /usr/bin/ld: hello: warning: allocated section `.notes' not in segment
  /usr/bin/ld: memory: warning: allocated section `.notes' not in segment
    BUILD   TCG tests for x86_64-linux-user
    BUILD   x86_64-linux-user guest-tests with cc
  /usr/bin/ld: cannot find -lpthread
  /usr/bin/ld: cannot find -lc
  collect2: error: ld returned 1 exit status
  make[2]: *** [threadcount] Error 1
  make[1]: *** [cross-build-guest-tests] Error 2
  make: *** [build-tcg-tests-x86_64-linux-user] Error 2

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20210121172829.1643620-2-f4bug@amsat.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-01-26 15:39:34 +01:00
Max Reitz
bb24cdc5ef iotests/178: Pass value to invalid option
ccd3b3b811 has deprecated short-hand boolean options (i.e., options
with values).  All options without values are interpreted as boolean
options, so this includes the invalid option "snapshot.foo" used in
iotest 178.

So after ccd3b3b811, 178 fails with:

  +qemu-img: warning: short-form boolean option 'snapshot.foo' deprecated
  +Please use snapshot.foo=on instead

Suppress that deprecation warning by passing some value to it (it does
not matter which, because the option is invalid anyway).

Fixes: ccd3b3b811
       ("qemu-option: warn for short-form boolean options")
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210126123834.115915-1-mreitz@redhat.com>
2021-01-26 14:36:37 +01:00
Max Reitz
014bb4b0e5 iotests/118: Drop 'change' test
Commit 0afec75734 removed the 'change' QMP command, so we can no
longer test it in 118.

Fixes: 0afec75734
       ('qmp: remove deprecated "change" command')
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210126104833.57026-1-mreitz@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2021-01-26 14:36:37 +01:00
Alberto Garcia
0f46147e10 iotests: Add test for the regression fixed in c8bf9a9169
Signed-off-by: Alberto Garcia <berto@igalia.com>
Suggested-by: Maxim Levitsky <mlevitsk@redhat.com>
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Message-Id: <20210112170540.2912-1-berto@igalia.com>
[mreitz: Add "# group:" line]
Signed-off-by: Max Reitz <mreitz@redhat.com>
2021-01-26 14:36:37 +01:00
David Edmondson
797e3e3805 block: report errno when flock fcntl fails
When a call to fcntl(2) for the purpose of adding file locks fails
with an error other than EAGAIN or EACCES, report the error returned
by fcntl.

EAGAIN or EACCES are elided as they are considered to be common
failures, indicating that a conflicting lock is held by another
process.

No errors are elided when removing file locks.

Signed-off-by: David Edmondson <david.edmondson@oracle.com>
Message-Id: <20210113164447.2545785-1-david.edmondson@oracle.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2021-01-26 14:36:37 +01:00
Vladimir Sementsov-Ogievskiy
c701f59253 simplebench: add bench-backup.py
Add script to benchmark new backup architecture.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20210116214705.822267-24-vsementsov@virtuozzo.com>
[mreitz: s/not unsupported/not supported/]
Signed-off-by: Max Reitz <mreitz@redhat.com>
2021-01-26 14:36:37 +01:00
Vladimir Sementsov-Ogievskiy
b2fcb0c575 simplebench: bench_block_job: add cmd_options argument
Add argument to allow additional block-job options.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-23-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2021-01-26 14:36:37 +01:00
Vladimir Sementsov-Ogievskiy
2096de521e simplebench/bench_block_job: use correct shebang line with python3
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-22-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2021-01-26 14:36:37 +01:00
Vladimir Sementsov-Ogievskiy
143a6384f5 block/block-copy: drop unused argument of block_copy()
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-21-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2021-01-26 14:36:37 +01:00
Vladimir Sementsov-Ogievskiy
5b49c2bdc1 block/block-copy: drop unused block_copy_set_progress_callback()
Drop unused code.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-20-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2021-01-26 14:36:37 +01:00
Vladimir Sementsov-Ogievskiy
6a30f663d4 qapi: backup: disable copy_range by default
Further commit will add a benchmark
(scripts/simplebench/bench-backup.py), which will show that backup
works better with async parallel requests (previous commit) and
disabled copy_range. So, let's disable copy_range by default.

Note: the option was added several commits ago with default to true,
to follow old behavior (the feature was enabled unconditionally), and
only now we are going to change the default behavior.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-19-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2021-01-26 14:36:37 +01:00
Vladimir Sementsov-Ogievskiy
71eed4cebe backup: move to block-copy
This brings async request handling and block-status driven chunk sizes
to backup out of the box, which improves backup performance.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-18-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2021-01-26 14:36:37 +01:00
Vladimir Sementsov-Ogievskiy
511e7d31bf block/backup: drop extra gotos from backup_run()
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-17-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2021-01-26 14:36:37 +01:00
Vladimir Sementsov-Ogievskiy
d51590fc3e block/block-copy: make progress_bytes_callback optional
We are going to stop use of this callback in the following commit.
Still the callback handling code will be dropped in a separate commit.
So, for now let's make it optional.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-16-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2021-01-26 14:36:37 +01:00
Vladimir Sementsov-Ogievskiy
2d0f32e3fc iotests: 257: prepare for backup over block-copy
Iotest 257 dumps a lot of in-progress information of backup job, such
as offset and bitmap dirtiness. Further commit will move backup to be
one block-copy call, which will introduce async parallel requests
instead of plain cluster-by-cluster copying. To keep things
deterministic, allow only one worker (only one copy request at a time)
for this test.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-15-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2021-01-26 14:36:37 +01:00
Vladimir Sementsov-Ogievskiy
34a5de525a iotests: 219: prepare for backup over block-copy
The further change of moving backup to be a one block-copy call will
make copying chunk-size and cluster-size two separate things. So, even
with 64k cluster sized qcow2 image, default chunk would be 1M.
Test 219 depends on specified chunk-size. Update it for explicit
chunk-size for backup as for mirror.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-14-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2021-01-26 14:36:37 +01:00
Vladimir Sementsov-Ogievskiy
61623f8215 iotests: 185: prepare for backup over block-copy
The further change of moving backup to be a one block-copy call will
make copying chunk-size and cluster-size two separate things. So, even
with 64k cluster sized qcow2 image, default chunk would be 1M.
185 test however assumes, that with speed limited to 64K, one iteration
would result in offset=64K. It will change, as first iteration would
result in offset=1M independently of speed.

So, let's explicitly specify, what test wants: set max-chunk to 64K, so
that one iteration is 64K. Note, that we don't need to limit
max-workers, as block-copy rate limiter will handle the situation and
wouldn't start new workers when speed limit is obviously reached.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-13-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2021-01-26 14:36:37 +01:00
Max Reitz
67a066fbe4 iotests/129: Limit backup's max-chunk/max-workers
Right now, this does not change anything, because backup ignores
max-chunk and max-workers.  However, as soon as backup is switched over
to block-copy for the background copying process, we will need it to
keep 129 passing.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210120102043.28346-1-mreitz@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2021-01-26 14:36:37 +01:00
Vladimir Sementsov-Ogievskiy
55f3e5cb3b iotests: 56: prepare for backup over block-copy
After introducing parallel async copy requests instead of plain
cluster-by-cluster copying loop, we'll have to wait for paused status,
as we need to wait for several parallel request. So, let's gently wait
instead of just asserting that job already paused.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-12-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2021-01-26 14:36:37 +01:00