Commit Graph

5849 Commits

Author SHA1 Message Date
Peter Maydell
e021e6fe52 QAPI patches for 2020-01-14
-----BEGIN PGP SIGNATURE-----
 
 iQJGBAABCAAwFiEENUvIs9frKmtoZ05fOHC0AOuRhlMFAl4dlLoSHGFybWJydUBy
 ZWRoYXQuY29tAAoJEDhwtADrkYZTKGoQAKXNP/+ezGJvczU/RVtBhSQXRXQyF+zh
 VFznrB8gL4+MkD20bZv1jhSeNHJgQn4Frarq7IfglNy8NSLOwkVWleY9QFZV+t7U
 puXbO4AfIoMyvKYDEweDaHhAkQ1lnd+lDkN8HPm83EdM9ot+sGmppwnONtJm2xoC
 2/+Ff+A7lmr9mU/+KVMgtnhdu8l/NdSJ4dk3gp0zPqcRtM9PanwBjA4ve+4asBxg
 Hec5lQlqkj09ngNtFMXTQdYvpWXXcKyBwsoJymFh6kSYA7ZACvchfvqq0SS1JQfV
 4yCBfro64ZwUeWuwD5Blzb5iwrrVURRm2ecLq9uy62xmEqX72OWpl1h2+Q2HBZoZ
 rSbo85PJ2DTGNK+xRyRF6N7E1RGlVD4tIcE0nH/+JfW1Fkf7wCGe7AR7ywXScDav
 nW9UtpwPMLYHz/jO0khYAlpfY+SGtO7JFokv7R+9Zpi9IFUziyY9Y74/EIUDE8QV
 vk/GFbGPDUg1+aZgI3Q2/iPmgXDYWZIoBNhmebI8jwNktIyCCGWL07/I6jOX8K+/
 f1njp4Bma8sFm1xoSwgJqams1SsuXdKj33V0k1Yw88RvprXBsRRMHBVtlz443/gh
 LtKGgqJMCU+uuDIJBZ7t1r/iNY+2bgsrgPVNf9AMtPoY+A0dP4FiUuGKEY3fkCAy
 h8WF8U/rkJ4/
 =iRJu
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/armbru/tags/pull-qapi-2020-01-14' into staging

QAPI patches for 2020-01-14

# gpg: Signature made Tue 14 Jan 2020 10:15:22 GMT
# gpg:                using RSA key 354BC8B3D7EB2A6B68674E5F3870B400EB918653
# gpg:                issuer "armbru@redhat.com"
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>" [full]
# gpg:                 aka "Markus Armbruster <armbru@pond.sub.org>" [full]
# Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867  4E5F 3870 B400 EB91 8653

* remotes/armbru/tags/pull-qapi-2020-01-14:
  qapi: Simplify QAPISchemaModularCVisitor
  qapi: Fix code generation for empty modules
  qapi: Proper intermediate representation for modules
  qapi: Generate command registration stuff into separate files
  tests/Makefile.include: Fix missing test-qapi-emit-events.[ch]
  qapi: Tweak "command returns a nice type" check for clarity

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-01-14 16:42:27 +00:00
Markus Armbruster
3e7fb5811b qapi: Fix code generation for empty modules
When a sub-module doesn't contain any definitions, we don't generate
code for it, but we do generate the #include.

We generate code only for modules that get visited.
QAPISchema.visit() visits only modules that have definitions.  It can
visit modules multiple times.

Clean this up as follows.  Collect entities in their QAPISchemaModule.
Have QAPISchema.visit() call QAPISchemaModule.visit() for each module.
Have QAPISchemaModule.visit() call .visit_module() for itself, and
QAPISchemaEntity.visit() for each of its entities.  This way, we visit
each module exactly once.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191120182551.23795-6-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2020-01-14 11:01:58 +01:00
Markus Armbruster
00ca24ff9e qapi: Generate command registration stuff into separate files
Having to include qapi-commands.h just for qmp_init_marshal() is
suboptimal.  Generate it into separate files.  This lets
monitor/misc.c, qga/main.c, and the generated qapi-commands-FOO.h
include less.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191120182551.23795-4-armbru@redhat.com>
[Typos in docs/devel/qapi-code-gen.txt fixed]
Reviewed-by: Eric Blake <eblake@redhat.com>
2020-01-14 11:01:58 +01:00
Markus Armbruster
0cc0e26894 tests/Makefile.include: Fix missing test-qapi-emit-events.[ch]
Commit 5d75648b56 "qapi: Generate QAPIEvent stuff into separate files"
added tests/test-qapi-emit-events.[ch] to the set of generated files,
but neglected to update tests/.gitignore and tests/Makefile.include.
Commit a0af8cee3c "tests/.gitignore: ignore test-qapi-emit-events.[ch]
for in-tree builds" fixed the former.  Now fix the latter.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191120182551.23795-3-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2020-01-14 11:01:58 +01:00
Peter Maydell
981c9b88e6 * Move qtests into a separate directory
* Build index.html for docs
 -----BEGIN PGP SIGNATURE-----
 
 iQJFBAABCAAvFiEEJ7iIR+7gJQEY8+q5LtnXdP5wLbUFAl4bAUURHHRodXRoQHJl
 ZGhhdC5jb20ACgkQLtnXdP5wLbUSQQ/+MU9vOoDrctISyxN8YQ1a0YdYKmuXKHyT
 YI/z/CV3xFnTw3sIHzXWXg/MzwgpTUXOZn5TG0P46qeNBPC416FrKoi+4J91VWud
 80si37fwmXiEMB67yK4KqkZzEeffoei4EWC8fWHU//KEaEYiLxNLjvPGSV5+Gjw7
 6gh4QcGecRectBCb2BqPwr8Qp7B5k+tPCpWsWff+eFZSbHYe6+g00x6wOTOLTd1h
 MccEoyq1LrZshSO6iThISgIUpLyMyCy9KMJXIb7mztxCGLDffLVnJFhmH5Yn6I6e
 7SVHoa7cKjHMN1DC9p+5+cjqo5VZg7iPy346zPEPitdUw1e0tnYNyqZDh5d8xoUR
 WlGpc4USuHbysznbpkaFaMABkKyXPddVQUUtIgHcfav2UVAUH2KmZ38POnY+W9wL
 S1Yv9yhUFT0aPvj8QsSRBaQXfcn73CFlJV7XsOd7opFH/M5kWVtOzVpzSINLqlQC
 BJOW1ePyhJ8LqKdTRtSe5BHM8RtwDRukDGpGmJjKJeCvv6uE2wrcjZcwbTxghatj
 AFelcSlTdIJZBSc7+rWm/Y5nm857erXs5rt2bLjJBa1/gWg19yAj8aY08zzael3T
 juyvNH/Au79sCmOznGvX4TrltpnhU4kdLKYZ3zpDpbmc4l33kaOz+3xn9NdMnWA2
 zb9nSBePXkI=
 =CchM
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/huth-gitlab/tags/pull-request-2020-01-12' into staging

* Move qtests into a separate directory
* Build index.html for docs

# gpg: Signature made Sun 12 Jan 2020 11:21:41 GMT
# gpg:                using RSA key 27B88847EEE0250118F3EAB92ED9D774FE702DB5
# gpg:                issuer "thuth@redhat.com"
# gpg: Good signature from "Thomas Huth <th.huth@gmx.de>" [full]
# gpg:                 aka "Thomas Huth <thuth@redhat.com>" [full]
# gpg:                 aka "Thomas Huth <huth@tuxfamily.org>" [full]
# gpg:                 aka "Thomas Huth <th.huth@posteo.de>" [unknown]
# Primary key fingerprint: 27B8 8847 EEE0 2501 18F3  EAB9 2ED9 D774 FE70 2DB5

* remotes/huth-gitlab/tags/pull-request-2020-01-12:
  docs: build an index page for the HTML docs
  tests/libqos: Move the libqos files under tests/qtest/
  tests/Makefile: Move qtest-related settings to a separate Makefile.include
  test: Move qtests to a separate directory
  tests/Makefile: Separate unit test dependencies from qtest dependencies
  tests/Makefile: Remove 'tests/' and '$(EXESUF)' from the check-qtest variables
  tests/ptimer: Remove unnecessary inclusion of libqtest.h
  tests/Makefile: test-char does not need libqtest

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-01-13 13:06:49 +00:00
Thomas Huth
1cf4323ecd tests/libqos: Move the libqos files under tests/qtest/
The qos stuff belongs to qtest, so move it into that directory, too.

Message-Id: <20191218103059.11729-8-thuth@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-01-12 11:42:41 +01:00
Thomas Huth
833884f37a tests/Makefile: Move qtest-related settings to a separate Makefile.include
tests/Makefile.include is pretty much overcrowded. Now that we have a
dedicated folder for the qtests, let's move the related settings
to a Makefile.include file in that directory instead.

Message-Id: <20191218103059.11729-7-thuth@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-01-12 11:42:41 +01:00
Thomas Huth
1e8a1fae74 test: Move qtests to a separate directory
The tests directory itself is pretty overcrowded, and it's hard to
see which test belongs to which test subsystem (unit, qtest, ...).
Let's move the qtests to a separate folder for more clarity.

Message-Id: <20191218103059.11729-6-thuth@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-01-12 11:42:41 +01:00
Thomas Huth
10ae5b303a tests/Makefile: Separate unit test dependencies from qtest dependencies
Some of the unit test dependencies are declared right in the block of
the qtest dependencies. Let's move them to the other unit tests instead.

Message-Id: <20191218103059.11729-5-thuth@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-01-12 11:42:40 +01:00
Thomas Huth
55e997a724 tests/Makefile: Remove 'tests/' and '$(EXESUF)' from the check-qtest variables
It's much easier if we simply add the folder prefix and the exe suffix
later via a substitution instead.

Message-Id: <20191218103059.11729-4-thuth@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-01-12 11:42:40 +01:00
Thomas Huth
6f5b752e96 tests/ptimer: Remove unnecessary inclusion of libqtest.h
ptimer-test is a unit test, not a qtest, so libqtest.h is not
required here.

Message-Id: <20191218103059.11729-3-thuth@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-01-12 11:42:40 +01:00
Thomas Huth
3db9f6a324 tests/Makefile: test-char does not need libqtest
No need to link the libqtest objects here.

Message-Id: <20191218103059.11729-2-thuth@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-01-12 11:42:40 +01:00
Peter Maydell
b952544fe8 * Compat machines fix (Denis)
* Command line parsing fixes (Michal, Peter, Xiaoyao)
 * Cooperlake CPU model fixes (Xiaoyao)
 * i386 gdb fix (mkdolata)
 * IOEventHandler cleanup (Philippe)
 * icount fix (Pavel)
 * RR support for random number sources (Pavel)
 * Kconfig fixes (Philippe)
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v2.0.22 (GNU/Linux)
 
 iQEcBAABAgAGBQJeFbG8AAoJEL/70l94x66DCpMIAKBwxBL+VegqI+ySKgmtIBQX
 LtU+ardEeZ37VfWfvuWzTFe+zQ0hsFpz/e0LHE7Ae+LVLMNWXixlmMrTIm+Xs762
 hJzxBjhUhkdrMioVYTY16Kqap4Nqaxu70gDQ32Ve2sY6xYGxYLSaJooBOU5bXVgb
 HPspHFVpeP6ZshBd1n2LXsgURE6v3AjTwqcsPCkL/AESFdkdOsoHeXjyKWJG1oPy
 W7btzlUEqVsauZI8/PhhW/8hZUvUsJVHonYLTZTyy8aklU7aOILSyT2uPXFBVUVQ
 irkQjLtD4dWlogBKO4i/QHMuwV+Asa57WNPmqv3EcIWPUWmTY84H0g2AxRgcc2M=
 =48jx
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into staging

* Compat machines fix (Denis)
* Command line parsing fixes (Michal, Peter, Xiaoyao)
* Cooperlake CPU model fixes (Xiaoyao)
* i386 gdb fix (mkdolata)
* IOEventHandler cleanup (Philippe)
* icount fix (Pavel)
* RR support for random number sources (Pavel)
* Kconfig fixes (Philippe)

# gpg: Signature made Wed 08 Jan 2020 10:41:00 GMT
# gpg:                using RSA key BFFBD25F78C7AE83
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>" [full]
# gpg:                 aka "Paolo Bonzini <pbonzini@redhat.com>" [full]
# Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4  E2F7 7E15 100C CD36 69B1
#      Subkey fingerprint: F133 3857 4B66 2389 866C  7682 BFFB D25F 78C7 AE83

* remotes/bonzini/tags/for-upstream: (38 commits)
  chardev: Use QEMUChrEvent enum in IOEventHandler typedef
  chardev: use QEMUChrEvent instead of int
  chardev/char: Explicit we ignore some QEMUChrEvent in IOEventHandler
  monitor/hmp: Explicit we ignore a QEMUChrEvent in IOEventHandler
  monitor/qmp: Explicit we ignore few QEMUChrEvent in IOEventHandler
  virtio-console: Explicit we ignore some QEMUChrEvent in IOEventHandler
  vhost-user-blk: Explicit we ignore few QEMUChrEvent in IOEventHandler
  vhost-user-net: Explicit we ignore few QEMUChrEvent in IOEventHandler
  vhost-user-crypto: Explicit we ignore some QEMUChrEvent in IOEventHandler
  ccid-card-passthru: Explicit we ignore QEMUChrEvent in IOEventHandler
  hw/usb/redirect: Explicit we ignore few QEMUChrEvent in IOEventHandler
  hw/usb/dev-serial: Explicit we ignore few QEMUChrEvent in IOEventHandler
  hw/char/terminal3270: Explicit ignored QEMUChrEvent in IOEventHandler
  hw/ipmi: Explicit we ignore some QEMUChrEvent in IOEventHandler
  hw/ipmi: Remove unnecessary declarations
  target/i386: Add missed features to Cooperlake CPU model
  target/i386: Add new bit definitions of MSR_IA32_ARCH_CAPABILITIES
  target/i386: Fix handling of k_gs_base register in 32-bit mode in gdbstub
  hw/rtc/mc146818: Add missing dependency on ISA Bus
  hw/nvram/Kconfig: Restrict CHRP NVRAM to machines using OpenBIOS or SLOF
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-01-10 17:16:49 +00:00
Alex Bennée
486e58b188 tests/tcg: add user version of dumb-as-bricks semiconsole test
There are linux-user users of semihosting so we'd better check things
work for them as well.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
2020-01-09 11:41:29 +00:00
Alex Bennée
f05938090b tests/tcg: extract __semi_call into a header and expand
There are two types of ARM semicall - lets test them both. Putting the
logic in a header will make re-using the functions easier later.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
2020-01-09 11:41:29 +00:00
Alex Bennée
9c354591ca tests/tcg: add a dumb-as-bricks semihosting console test
We don't run this during check-tcg as we would need to check stuff is
echoed back. However we can still build the binary so people can test
it manually.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
2020-01-09 11:41:29 +00:00
Alex Bennée
3618e3a640 testing: don't nest build for fp-test
Re-calling the main make is counter-productive and really messes up
with parallel builds. Just ensure we have built the pre-requisites
before we build the fp-test bits. If the user builds manually just
complain if the parent build hasn't got the bits we need.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Emilio G. Cota <cota@braap.org>
2020-01-09 11:41:29 +00:00
Gerd Hoffmann
bc2bf7f348 freebsd: use python37
FreeBSD seems to use python37 by default now, which breaks the build
script.  Add python to the package list, to explicitly pick the version,
and also adapt the configure command line.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200106123746.18201-1-kraxel@redhat.com>
2020-01-09 11:41:29 +00:00
Gerd Hoffmann
22db66a436 tests/vm: update openbsd to release 6.6
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20191018102443.26469-1-kraxel@redhat.com>
2020-01-09 11:41:29 +00:00
Philippe Mathieu-Daudé
083b266f69 chardev: Use QEMUChrEvent enum in IOEventHandler typedef
The Chardev events are listed in the QEMUChrEvent enum.

By using the enum in the IOEventHandler typedef we:

- make the IOEventHandler type more explicit (this handler
  process out-of-band information, while the IOReadHandler
  is in-band),
- help static code analyzers.

This patch was produced with the following spatch script:

  @match@
  expression backend, opaque, context, set_open;
  identifier fd_can_read, fd_read, fd_event, be_change;
  @@
  qemu_chr_fe_set_handlers(backend, fd_can_read, fd_read, fd_event,
                           be_change, opaque, context, set_open);

  @depends on match@
  identifier opaque, event;
  identifier match.fd_event;
  @@
   static
  -void fd_event(void *opaque, int event)
  +void fd_event(void *opaque, QEMUChrEvent event)
   {
   ...
   }

Then the typedef was modified manually in
include/chardev/char-fe.h.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Acked-by: Corey Minyard <cminyard@mvista.com>
Acked-by: Cornelia Huck <cohuck@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20191218172009.8868-15-philmd@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-01-08 11:15:35 +01:00
Peter Maydell
973d306dd6 virtio, pci, pc: fixes, features
Bugfixes all over the place.
 HMAT support.
 New flags for vhost-user-blk utility.
 Auto-tuning of seg max for virtio storage.
 
 Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
 -----BEGIN PGP SIGNATURE-----
 
 iQFDBAABCAAtFiEEXQn9CHHI+FuUyooNKB8NuNKNVGkFAl4TaMEPHG1zdEByZWRo
 YXQuY29tAAoJECgfDbjSjVRpvzgH/2LyDAzCa9h93ikSJjmyUk5FUaqve38daEb3
 S3JYjwKxQx7u1ydooKhvBQnBCZ2i3S+k62gfYyKB+nBv8xvjs0Eg5D1YJ5E8hciy
 lf5OFGWWtX2iPDjZwQwT13kiJe0o3JRGxJJ6XqTEG+1EYOp7cky/FEv4PD030b9m
 I2wROZ/Am+onB9YJX8c0Vv1CG+AryuJNXnvwQzTXEjj4U7bEYUyJwVZaCRyAdWQ3
 uYXIZN9VwjVX6BFvy9ZAJbEsUVJvOM1/aQaDqcrLz+VlzRT7bRkKHi2G3vakrm1I
 r5OpgyLo84132awCncbSykKDH5o8WaxLaJBjGmuBfasMz9wPzAg=
 =uL1o
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/mst/tags/for_upstream' into staging

virtio, pci, pc: fixes, features

Bugfixes all over the place.
HMAT support.
New flags for vhost-user-blk utility.
Auto-tuning of seg max for virtio storage.

Signed-off-by: Michael S. Tsirkin <mst@redhat.com>

# gpg: Signature made Mon 06 Jan 2020 17:05:05 GMT
# gpg:                using RSA key 5D09FD0871C8F85B94CA8A0D281F0DB8D28D5469
# gpg:                issuer "mst@redhat.com"
# gpg: Good signature from "Michael S. Tsirkin <mst@kernel.org>" [full]
# gpg:                 aka "Michael S. Tsirkin <mst@redhat.com>" [full]
# Primary key fingerprint: 0270 606B 6F3C DF3D 0B17  0970 C350 3912 AFBE 8E67
#      Subkey fingerprint: 5D09 FD08 71C8 F85B 94CA  8A0D 281F 0DB8 D28D 5469

* remotes/mst/tags/for_upstream: (32 commits)
  intel_iommu: add present bit check for pasid table entries
  intel_iommu: a fix to vtd_find_as_from_bus_num()
  virtio-net: delete also control queue when TX/RX deleted
  virtio: reset region cache when on queue deletion
  virtio-mmio: update queue size on guest write
  tests: add virtio-scsi and virtio-blk seg_max_adjust test
  virtio: make seg_max virtqueue size dependent
  hw: fix using 4.2 compat in 5.0 machine types for i440fx/q35
  vhost-user-scsi: reset the device if supported
  vhost-user: add VHOST_USER_RESET_DEVICE to reset devices
  hw/pci/pci_host: Let pci_data_[read/write] use unsigned 'size' argument
  hw/pci/pci_host: Remove redundant PCI_DPRINTF()
  virtio-mmio: Clear v2 transport state on soft reset
  ACPI: add expected files for HMAT tests (acpihmat)
  tests/bios-tables-test: add test cases for ACPI HMAT
  tests/numa: Add case for QMP build HMAT
  hmat acpi: Build Memory Side Cache Information Structure(s)
  hmat acpi: Build System Locality Latency and Bandwidth Information Structure(s)
  hmat acpi: Build Memory Proximity Domain Attributes Structure(s)
  numa: Extend CLI to provide memory side cache information
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-01-07 16:25:00 +00:00
Peter Maydell
0cb0406172 tests/iothread: Always connect iothread GSource to a GMainContext
On older versions of glib (anything prior to glib commit 0f056ebe
from May 2019), the implementation of g_source_ref() and
g_source_unref() is not threadsafe for a GSource which is not
attached to a GMainContext.

QEMU's real iothread.c implementation always attaches its
iothread->ctx's GSource to a GMainContext created for that iothread,
so it is OK, but the simple test framework implementation in
tests/iothread.c was not doing this.  This was causing intermittent
assertion failures in the test-aio-multithread subtest
"/aio/multi/mutex/contended" test on the BSD hosts.  (It's unclear
why only BSD seems to have been affected -- perhaps a combination of
the specific glib version being used in the VMs and their happening
to run on a host with a lot of CPUs).

Borrow the iothread_init_gcontext() from the real iothread.c
and add the corresponding cleanup code and the calls to
g_main_context_push/pop_thread_default() so we actually use
the GMainContext we create.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Message-id: 20200106144552.7205-1-peter.maydell@linaro.org
2020-01-07 14:32:57 +00:00
Peter Maydell
c4d1069c25 Add dbus-vmstate
Hi,
 
 With external processes or helpers participating to the VM support, it
 becomes necessary to handle their migration. Various options exist to
 transfer their state:
 1) as the VM memory, RAM or devices (we could say that's how
    vhost-user devices can be handled today, they are expected to
    restore from ring state)
 2) other "vmstate" (as with TPM emulator state blobs)
 3) left to be handled by management layer
 
 1) is not practical, since an external processes may legitimatelly
 need arbitrary state date to back a device or a service, or may not
 even have an associated device.
 
 2) needs ad-hoc code for each helper, but is simple and working
 
 3) is complicated for management layer, QEMU has the migration timing
 
 The proposed "dbus-vmstate" object will connect to a given D-Bus
 address, and save/load from org.qemu.VMState1 owners on migration.
 
 Thus helpers can easily have their state migrated with QEMU, without
 implementing ad-hoc support (such as done for TPM emulation)
 
 D-Bus is ubiquitous on Linux (it is systemd IPC), and can be made to
 work on various other OSes. There are several implementations and good
 bindings for various languages.  (the tests/dbus-vmstate-test.c is a
 good example of how simple the implementation of services can be, even
 in C)
 
 dbus-vmstate is put into use by the libvirt series "[PATCH 00/23] Use
 a slirp helper process".
 
 v2:
  - fix build with broken mingw-glib
 -----BEGIN PGP SIGNATURE-----
 
 iQJQBAABCAA6FiEEh6m9kz+HxgbSdvYt2ujhCXWWnOUFAl4TR5ccHG1hcmNhbmRy
 ZS5sdXJlYXVAcmVkaGF0LmNvbQAKCRDa6OEJdZac5R6EEACFTd4hDG8i/GnxCFut
 MGcTusJr+2IklIT/K0qpLf0axNUoIqycwv8m0T9QhoG8h+9lMykOd1YJpNetT5qK
 gifOF2gcPK/9WIdFbX7dLSUAWpzO6fG/RzKK65Nc1uJSnXlb8JV0BU/6FrfCE+3U
 Bg5PvVtxxtwejQfQPOI7bPxOqxr/SmjUGcbFgacMAMG0Lm/VG/92kdoC6Z4Xf/bd
 FcAeiO2CiPoGXG5zD4WF1emwxnSu65PgcFpSpqvvFlmDbYlTwoMt4VWxTfkAzbAM
 IES7j2IbhUEe3p0hvMTqmmsmds1QNCBgnQI/LtQiXPTnbfpBcZ0wT6QsSZXWvHz8
 ClA9OAimxyELblTGjD9vsi3G5m2DQS+NdfPOX7hfHouVQzDJJaS8jxDItpPgXwSO
 fZ9mUO8ps3N2YTakuKNBP/IzDOuyExrBg80GF+HbEc59Uhj8Yq/awyz1XsqjQzVP
 54+TUjwC8HZxVWgMeqiJ1njPTfRJo6uAnguLbfAXj8P9vaXLtsy/3JGsmKiziXXW
 XzvQDzhfOMjm7Uo7vN7Hp3X/UYJxnaQ3dViqZnv/gqG6yv+igVlqyrTx2IBhN2NW
 DZt3c7VqVUBYFShLgfy0zDjzM/s7mFkQKCFHUsBqIwODugYEc3TTdAa60QYjX5i9
 negngax45KM6nF3tq74fJpwWVw==
 =M4kD
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/elmarco/tags/dbus-vmstate7-pull-request' into staging

Add dbus-vmstate

Hi,

With external processes or helpers participating to the VM support, it
becomes necessary to handle their migration. Various options exist to
transfer their state:
1) as the VM memory, RAM or devices (we could say that's how
   vhost-user devices can be handled today, they are expected to
   restore from ring state)
2) other "vmstate" (as with TPM emulator state blobs)
3) left to be handled by management layer

1) is not practical, since an external processes may legitimatelly
need arbitrary state date to back a device or a service, or may not
even have an associated device.

2) needs ad-hoc code for each helper, but is simple and working

3) is complicated for management layer, QEMU has the migration timing

The proposed "dbus-vmstate" object will connect to a given D-Bus
address, and save/load from org.qemu.VMState1 owners on migration.

Thus helpers can easily have their state migrated with QEMU, without
implementing ad-hoc support (such as done for TPM emulation)

D-Bus is ubiquitous on Linux (it is systemd IPC), and can be made to
work on various other OSes. There are several implementations and good
bindings for various languages.  (the tests/dbus-vmstate-test.c is a
good example of how simple the implementation of services can be, even
in C)

dbus-vmstate is put into use by the libvirt series "[PATCH 00/23] Use
a slirp helper process".

v2:
 - fix build with broken mingw-glib

# gpg: Signature made Mon 06 Jan 2020 14:43:35 GMT
# gpg:                using RSA key 87A9BD933F87C606D276F62DDAE8E10975969CE5
# gpg:                issuer "marcandre.lureau@redhat.com"
# gpg: Good signature from "Marc-André Lureau <marcandre.lureau@redhat.com>" [full]
# gpg:                 aka "Marc-André Lureau <marcandre.lureau@gmail.com>" [full]
# Primary key fingerprint: 87A9 BD93 3F87 C606 D276  F62D DAE8 E109 7596 9CE5

* remotes/elmarco/tags/dbus-vmstate7-pull-request:
  tests: add dbus-vmstate-test
  tests: add migration-helpers unit
  dockerfiles: add dbus-daemon to some of latest distributions
  configure: add GDBUS_CODEGEN
  Add dbus-vmstate object
  util: add dbus helper unit
  docs: start a document to describe D-Bus usage
  vmstate: replace DeviceState with VMStateIf
  vmstate: add qom interface to get id

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-01-06 18:22:42 +00:00
Peter Maydell
f4d8cf148e Block patches:
- Minor fixes and tests from the freeze period (too minor to be included
   in 4.2)
 - Allow many bash iotests to test qcow2's external data file feature
 - Add compress filter driver
 - Fix Python iotests after 6f6e1698a6
 - Fix for the backup job
 -----BEGIN PGP SIGNATURE-----
 
 iQFGBAABCAAwFiEEkb62CjDbPohX0Rgp9AfbAGHVz0AFAl4TRSISHG1yZWl0ekBy
 ZWRoYXQuY29tAAoJEPQH2wBh1c9AOiUIAJKWyzc+d7Fe4mwYamrNq5kf3dadw0qp
 t4HGjQ5vn+u4I4FH7NwSlx7qpWcerqxijeOJUYPMzCAS2/cm6OG4ETcPzfM+TeZL
 sJupvu1aloUnJiQxs424KD5AClHftByu+KpDOYmt8HmuSG+PkXvrz3q5bfJEjBQ3
 ZVKp7Ye8XkSLMPFlOavJGiniEx9huzQ7d1PHxZjpq+GrowFN+vGBuRu2+GfYQSIl
 +F1XbojSA7X91DDLgLbgZbTrwA4ph9qUJF1ObzhCToL2JcTt8H4Wleujaah2czi5
 2tOt1xKGtXy0mSx8ToJSI9JYIy/R/cGp7J9RNWgWa9OvNftgCJ10mAs=
 =EHAE
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2020-01-06' into staging

Block patches:
- Minor fixes and tests from the freeze period (too minor to be included
  in 4.2)
- Allow many bash iotests to test qcow2's external data file feature
- Add compress filter driver
- Fix Python iotests after 6f6e1698a6
- Fix for the backup job

# gpg: Signature made Mon 06 Jan 2020 14:33:06 GMT
# gpg:                using RSA key 91BEB60A30DB3E8857D11829F407DB0061D5CF40
# gpg:                issuer "mreitz@redhat.com"
# gpg: Good signature from "Max Reitz <mreitz@redhat.com>" [full]
# Primary key fingerprint: 91BE B60A 30DB 3E88 57D1  1829 F407 DB00 61D5 CF40

* remotes/maxreitz/tags/pull-block-2020-01-06: (34 commits)
  backup-top: Begin drain earlier
  tests/qemu-iotests: Update tests to recent desugarized -accel option
  tests/qemu-iotests: add case to write compressed data of multiple clusters
  qcow2: Allow writing compressed data of multiple clusters
  block: introduce compress filter driver
  iotests: Allow check -o data_file
  iotests: Disable data_file where it cannot be used
  iotests: Make 198 work with data_file
  iotests: Make 137 work with data_file
  iotests: Make 110 work with data_file
  iotests: Make 091 work with data_file
  iotests: Avoid cp/mv of test images
  iotests: Use _rm_test_img for deleting test images
  iotests: Avoid qemu-img create
  iotests: Drop IMGOPTS use in 267
  iotests: Replace IMGOPTS='' by --no-opts
  iotests: Replace IMGOPTS= by -o
  iotests: Inject space into -ocompat=0.10 in 051
  iotests: Add -o and --no-opts to _make_test_img
  iotests: Let _make_test_img parse its parameters
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-01-06 17:44:22 +00:00
Denis Plotnikov
cec0242ab4 tests: add virtio-scsi and virtio-blk seg_max_adjust test
It tests proper seg_max_adjust settings for all machine types except
'none', 'isapc', 'microvm'

Signed-off-by: Denis Plotnikov <dplotnikov@virtuozzo.com>
Message-Id: <20191220140905.1718-3-dplotnikov@virtuozzo.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2020-01-06 12:04:51 -05:00
Marc-André Lureau
586ca6ba3c tests: add dbus-vmstate-test
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-01-06 18:41:32 +04:00
Marc-André Lureau
d77799ccda tests: add migration-helpers unit
Move a few helper functions from migration-test.c to migration-helpers.c

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-01-06 18:41:32 +04:00
Marc-André Lureau
1409c047c1 dockerfiles: add dbus-daemon to some of latest distributions
To get dbus-vmstate test covered.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-01-06 18:41:32 +04:00
Marc-André Lureau
107b59698f vmstate: add qom interface to get id
Add an interface to get the instance id, instead of depending on
Device and qdev_get_dev_path().

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Acked-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2020-01-06 18:41:32 +04:00
Philippe Mathieu-Daudé
75ab574b4c tests/qemu-iotests: Update tests to recent desugarized -accel option
Commit 6f6e1698a6 desugarized "-machine accel=" to a list
of "-accel" options. Since now "-machine accel" and "-accel"
became incompatible, update the iotests to the new format.

Error reported here:
https://gitlab.com/qemu-project/qemu/-/jobs/385801004#L3400

Reported-by: GitLab CI
Fixes: 6f6e1698a6 (vl: configure accelerators from -accel options)
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-id: 20200106130951.29873-1-philmd@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-01-06 14:14:41 +01:00
Andrey Shinkevich
19959445f5 tests/qemu-iotests: add case to write compressed data of multiple clusters
Add the case to the iotest #214 that checks possibility of writing
compressed data of more than one cluster size. The test case involves
the compress filter driver showing a sample usage of that.

Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 1575288906-551879-4-git-send-email-andrey.shinkevich@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-01-06 13:43:07 +01:00
Max Reitz
1b35b85abb iotests: Allow check -o data_file
The problem with allowing the data_file option is that you want to use a
different data file per image used in the test.  Therefore, we need to
allow patterns like -o data_file='$TEST_IMG.data_file'.

Then, we need to filter it out from qemu-img map, qemu-img create, and
remove the data file in _rm_test_img.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Message-id: 20191107163708.833192-23-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-01-06 13:43:07 +01:00
Max Reitz
3be2024aef iotests: Disable data_file where it cannot be used
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id: 20191107163708.833192-22-mreitz@redhat.com
[mreitz: Also disable 273]
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-01-06 13:43:07 +01:00
Max Reitz
39d2c7dc8f iotests: Make 198 work with data_file
We do not care about the json:{} filenames here, so we can just filter
them out and thus make the test work both with and without external data
files.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Message-id: 20191107163708.833192-21-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-01-06 13:43:07 +01:00
Max Reitz
e66566e6a7 iotests: Make 137 work with data_file
When using an external data file, there are no refcounts for data
clusters.  We thus have to adjust the corruption test in this patch to
not be based around a data cluster allocation, but the L2 table
allocation (L2 tables are still refcounted with external data files).

Furthermore, we should not print qcow2.py's list of incompatible
features because it differs depending on whether there is an external
data file or not.

With those two changes, the test will work both with and without
external data files (once that options works with the iotests at all).

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Message-id: 20191107163708.833192-20-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-01-06 13:43:07 +01:00
Max Reitz
d327a942aa iotests: Make 110 work with data_file
The only difference is that the json:{} filename of the image looks
different.  We actually do not care about that filename in this test, we
are only interested in (1) that there is a json:{} filename, and (2)
whether the backing filename can be constructed.

So just filter out the json:{} data, thus making this test pass both
with and without data_file.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Message-id: 20191107163708.833192-19-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-01-06 13:43:07 +01:00
Max Reitz
990f9bcc73 iotests: Make 091 work with data_file
The image end offset as reported by qemu-img check is different when
using an external data file; we do not care about its value here, so we
can just filter it.  Incidentally, common.rc already has _check_test_img
for us which does exactly that.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Message-id: 20191107163708.833192-18-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-01-06 13:43:07 +01:00
Max Reitz
d88bef1921 iotests: Avoid cp/mv of test images
This will not work with external data files, so try to get tests working
without it as far as possible.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Message-id: 20191107163708.833192-17-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-01-06 13:43:07 +01:00
Max Reitz
f91ecbd74e iotests: Use _rm_test_img for deleting test images
Just rm will not delete external data files.  Use _rm_test_img every
time we delete a test image.

(In the process, clean up the indentation of every _cleanup() this patch
touches.)

((Also, use quotes consistently.  I am happy to see unquoted instances
like "rm -rf $TEST_DIR/..." go.))

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Message-id: 20191107163708.833192-16-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-01-06 13:43:07 +01:00
Max Reitz
52a97b5a42 iotests: Avoid qemu-img create
Use _make_test_img whenever possible.  This way, we will not ignore
user-specified image options.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Message-id: 20191107163708.833192-15-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-01-06 13:43:07 +01:00
Max Reitz
ecb4c1d116 iotests: Drop IMGOPTS use in 267
Overwriting IMGOPTS means ignoring all user-supplied options, which is
not what we want.  Replace the current IMGOPTS use by a new BACKING_FILE
variable.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Message-id: 20191107163708.833192-14-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-01-06 13:43:07 +01:00
Max Reitz
10b612565c iotests: Replace IMGOPTS='' by --no-opts
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Message-id: 20191107163708.833192-13-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-01-06 13:43:07 +01:00
Max Reitz
407fb56a8e iotests: Replace IMGOPTS= by -o
Tests should not overwrite all user-supplied image options, but only add
to it (which will effectively overwrite conflicting values).  Accomplish
this by passing options to _make_test_img via -o instead of $IMGOPTS.

For some tests, there is no functional change because they already only
appended options to IMGOPTS.  For these, this patch is just a
simplification.

For others, this is a change, so they now heed user-specified $IMGOPTS.
Some of those tests do not work with all image options, though, so we
need to disable them accordingly.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Message-id: 20191107163708.833192-12-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-01-06 13:43:07 +01:00
Max Reitz
1c6d2f2128 iotests: Inject space into -ocompat=0.10 in 051
It did not matter before, but now that _make_test_img understands -o, we
should use it properly here.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Message-id: 20191107163708.833192-11-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-01-06 13:43:07 +01:00
Max Reitz
8b6d7be6a6 iotests: Add -o and --no-opts to _make_test_img
Blindly overriding IMGOPTS is suboptimal as this discards user-specified
options.  Whatever options the test needs should simply be appended.

Some tests do this (with IMGOPTS=$(_optstr_add "$IMGOPTS" "...")), but
that is cumbersome.  It’s simpler to just give _make_test_img an -o
parameter with which tests can add options.

Some tests actually must override the user-specified options, though,
for example when creating an image in a different format than the test
$IMGFMT.  For such cases, --no-opts allows clearing the current option
list.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Message-id: 20191107163708.833192-10-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-01-06 13:43:07 +01:00
Max Reitz
eea871d047 iotests: Let _make_test_img parse its parameters
This will allow us to add more options than just -b.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Message-id: 20191107163708.833192-9-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-01-06 13:43:07 +01:00
Max Reitz
8d1c81379c iotests: Drop compat=1.1 in 050
IMGOPTS can never be empty for qcow2, because the check scripts adds
compat=1.1 unless the user specified any compat option themselves.
Thus, this block does not do anything and can be dropped.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Message-id: 20191107163708.833192-8-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-01-06 13:43:07 +01:00
Max Reitz
b043b07ce3 iotests: Replace IMGOPTS by _unsupported_imgopts
Some tests require compat=1.1 and thus set IMGOPTS='compat=1.1'
globally.  That is not how it should be done; instead, they should
simply set _unsupported_imgopts to compat=0.10 (compat=1.1 is the
default anyway).

This makes the tests heed user-specified $IMGOPTS.  Some do not work
with all image options, though, so we need to disable them accordingly.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Maxim Levitsky <mlevitsky@redhat.com>
Message-id: 20191107163708.833192-7-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-01-06 13:43:07 +01:00
Max Reitz
7ab2a25890 iotests: Filter refcount_order in 036
This test can run just fine with other values for refcount_bits, so we
should filter the value from qcow2.py's dump-header.  In fact, we can
filter everything but the feature bits and header extensions, because
that is what the test is about.

(036 currently ignores user-specified image options, but that will be
fixed in the next patch.)

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Message-id: 20191107163708.833192-6-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-01-06 13:43:06 +01:00
Max Reitz
a75b7b5757 iotests: Add _filter_json_filename
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
Message-id: 20191107163708.833192-5-mreitz@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-01-06 13:43:06 +01:00