Commit Graph

2216 Commits

Author SHA1 Message Date
Thomas Huth
a5730b8bd3 block/file-posix: Simplify the XFS_IOC_DIOINFO handling
The handling for the XFS_IOC_DIOINFO ioctl is currently quite excessive:
This is not a "real" feature like the other features that we provide with
the "--enable-xxx" and "--disable-xxx" switches for the configure script,
since this does not influence lots of code (it's only about one call to
xfsctl() in file-posix.c), so people don't gain much with the ability to
disable this with "--disable-xfsctl".
It's also unfortunate that the ioctl will be disabled on Linux in case
the user did not install the right xfsprogs-devel package before running
configure. Thus let's simplify this by providing the ioctl definition
on our own, so we can completely get rid of the header dependency and
thus the related code in the configure script.

Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20211215125824.250091-1-thuth@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-01-12 14:09:04 +01:00
Stefan Weil
9ebfc5a583 simplebench: Fix Python syntax error (reported by LGTM)
Fixes: b2fcb0c575
Signed-off-by: Stefan Weil <sw@weilnetz.de>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-id: 20220107153019.504124-1-sw@weilnetz.de
Signed-off-by: John Snow <jsnow@redhat.com>
2022-01-10 18:23:10 -05:00
Martin Wilck
719fab3afa qemu-binfmt-conf.sh: fix -F option
qemu-binfmt-conf.sh should use "-F" as short option for "--qemu-suffix".
Fix the getopt call to make this work.

Fixes: 7155be7cda ("qemu-binfmt-conf.sh: allow to provide a suffix to the interpreter name")
Signed-off-by: Martin Wilck <mwilck@suse.com>
Reviewed-by: Laurent Vivier <laurent@vivier.eu>
Message-Id: <20211129135100.3934-1-mwilck@suse.com>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
2022-01-06 11:40:40 +01:00
Richard Henderson
f18155a207 Replace tap-driver.pl with "meson test".
-----BEGIN PGP SIGNATURE-----
 
 iQFIBAABCAAyFiEE8TM4V0tmI4mGbHaCv/vSX3jHroMFAmHEPBoUHHBib256aW5p
 QHJlZGhhdC5jb20ACgkQv/vSX3jHroMXVQf+PIngPelbV1Rtjpw7s4HOrkJ00Tc8
 mXzvFVWA+/uSxHPu6BsuZ0SvkvPGCEtc2rdmTfQPjo5mtcs5dytZNrwrsIxjfE0x
 g/Sjbd2tqZTUwQlPtbmzspcPj+d63cWivqbC8GoKbQA7Z1S71s+5fXObFiFaVgty
 fbrSOJkmhLT7GkvcN59RFk7CxgQ8d4YPiLeuFHYSJpYMm/SboL/WOlg5z93NVizW
 TZhXJUml26cK4YZYTi+d2U4+5rca+ObB//7pTHsu18SF5hVrTsz3bnki22JufPvi
 +WtMw9BXgblQ0uBU6lJChlDS7fuqCB1k98AFr/QsTmzO5eFunLU/XhiVLQ==
 =TitE
 -----END PGP SIGNATURE-----

Merge tag 'for-upstream-mtest' of https://gitlab.com/bonzini/qemu into staging

Replace tap-driver.pl with "meson test".

# gpg: Signature made Thu 23 Dec 2021 01:06:34 AM PST
# gpg:                using RSA key F13338574B662389866C7682BFFBD25F78C7AE83
# gpg:                issuer "pbonzini@redhat.com"
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>" [unknown]
# gpg:                 aka "Paolo Bonzini <pbonzini@redhat.com>" [unknown]
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg:          There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4  E2F7 7E15 100C CD36 69B1
#      Subkey fingerprint: F133 3857 4B66 2389 866C  7682 BFFB D25F 78C7 AE83

* tag 'for-upstream-mtest' of https://gitlab.com/bonzini/qemu:
  build: use "meson test" as the test harness

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-12-23 11:35:48 -08:00
Paolo Bonzini
3d2f73ef75 build: use "meson test" as the test harness
"meson test" starting with version 0.57 is just as capable and easy to
use as QEMU's own TAP driver.  All existing options for "make check"
work.  The only required code change involves how to mark "slow" tests;
they need to belong to an additional "slow" suite.

The rules for .tap output are replaced by JUnit XML; GitLab is able
to parse that output and present it in the CI pipeline report.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-12-23 10:06:19 +01:00
Marc-André Lureau
142ca628a7 ui: add a D-Bus display backend
The "dbus" display backend exports the QEMU consoles and other
UI-related interfaces over D-Bus.

By default, the connection is established on the session bus, but you
can specify a different bus with the "addr" option.

The backend takes the "org.qemu" service name, while still allowing
further instances to queue on the same name (so you can lookup all the
available instances too). It accepts any number of clients at this
point, although this is expected to evolve with options to restrict
clients, or only accept p2p via fd passing.

The interface is intentionally very close to the internal QEMU API,
and can be introspected or interacted with busctl/dfeet etc:

$ ./qemu-system-x86_64 -name MyVM -display dbus
$ busctl --user introspect org.qemu /org/qemu/Display1/Console_0

org.qemu.Display1.Console           interface -         -               -
.RegisterListener                   method    h         -               -
.SetUIInfo                          method    qqiiuu    -               -
.DeviceAddress                      property  s         "pci/0000/01.0" emits-change
.Head                               property  u         0               emits-change
.Height                             property  u         480             emits-change
.Label                              property  s         "VGA"           emits-change
.Type                               property  s         "Graphic"       emits-change
.Width                              property  u         640             emits-change
[...]

See the interfaces XML source file and Sphinx docs for the generated API
documentations.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
2021-12-21 10:50:22 +04:00
Marc-André Lureau
20f19713ef scripts: teach modinfo to skip non-C sources
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
2021-12-21 10:50:21 +04:00
Philippe Mathieu-Daudé
ce2ff9cccf checkpatch: Do not allow deprecated g_memdup()
g_memdup() is insecure and as been deprecated in GLib 2.68.
QEMU provides the safely equivalent g_memdup2() wrapper.

Do not allow more g_memdup() calls in the repository, provide
a hint to use g_memdup2().

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210903174510.751630-29-philmd@redhat.com>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
2021-12-17 11:57:35 +01:00
Richard Henderson
29eb5c2c86 * improve compatibility for macOS scripts/entitlement.sh (Evan)
* add support for KVM_GUESTDBG_BLOCKIRQ (Maxim)
 * update linux-headers to Linux 5.16 (myself)
 * configure cleanups (myself)
 * lsi53c895a assertion failure fix (Philippe)
 * fix incorrect description for die-id (Yanan)
 * support for NUMA in SGX enclave memory (Yang Zhong)
 -----BEGIN PGP SIGNATURE-----
 
 iQFIBAABCAAyFiEE8TM4V0tmI4mGbHaCv/vSX3jHroMFAmG5yEgUHHBib256aW5p
 QHJlZGhhdC5jb20ACgkQv/vSX3jHroOFkwf9Glt3tnHfO/gWZectiMAjjM8vg3cR
 yEsWU0fa0iu8MO8NiOW0b6DT0yosZ8rl2SYSZBQGqwa/Lq2MZLO2v0JG4LblBom7
 WVmzNOG4t0OFoE2gwMlZ2J+ppaHV5zN1Yc6lMXOVQcefY0pXoyze+0Fq/cwKf2+p
 6WDZTuOpjtxMwt9n71b93kNAHagk422MpNi1xi+o37hNsm/Zh3BTtaSJ7WCkCmWq
 YmwwQZgyB/V0Ah42wchuf5zxEnQPMbjaQSu8jQU4iyCB7+2biBuh3Cqw5eR8tqC1
 cGNINGlo2Orl9ASyEunHyuDj8p0wN8J37wNuBo4kghhrTVsVKQYF0wmsqw==
 =U1Xy
 -----END PGP SIGNATURE-----

Merge tag 'for-upstream' of https://gitlab.com/bonzini/qemu into staging

* improve compatibility for macOS scripts/entitlement.sh (Evan)
* add support for KVM_GUESTDBG_BLOCKIRQ (Maxim)
* update linux-headers to Linux 5.16 (myself)
* configure cleanups (myself)
* lsi53c895a assertion failure fix (Philippe)
* fix incorrect description for die-id (Yanan)
* support for NUMA in SGX enclave memory (Yang Zhong)

# gpg: Signature made Wed 15 Dec 2021 02:49:44 AM PST
# gpg:                using RSA key F13338574B662389866C7682BFFBD25F78C7AE83
# gpg:                issuer "pbonzini@redhat.com"
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>" [unknown]
# gpg:                 aka "Paolo Bonzini <pbonzini@redhat.com>" [unknown]
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg:          There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4  E2F7 7E15 100C CD36 69B1
#      Subkey fingerprint: F133 3857 4B66 2389 866C  7682 BFFB D25F 78C7 AE83

* tag 'for-upstream' of https://gitlab.com/bonzini/qemu:
  configure: remove dead variables
  doc: Add the SGX numa description
  numa: Support SGX numa in the monitor and Libvirt interfaces
  numa: Enable numa for SGX EPC sections
  kvm: add support for KVM_GUESTDBG_BLOCKIRQ
  gdbstub, kvm: let KVM report supported singlestep flags
  gdbstub: reject unsupported flags in handle_set_qemu_sstep
  linux-headers: update to 5.16-rc1
  virtio-gpu: do not byteswap padding
  scripts/entitlement.sh: Use backward-compatible cp flags
  qapi/machine.json: Fix incorrect description for die-id
  tests/qtest: Add fuzz-lsi53c895a-test
  hw/scsi/lsi53c895a: Do not abort when DMA requested and no data queued

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-12-16 08:39:20 -08:00
Thomas Huth
e6a52b3651 Move the libssh setup from configure to meson.build
It's easier to do this in meson.build now.

Message-Id: <20211209144801.148388-1-thuth@redhat.com>
Acked-by: Richard W.M. Jones <rjones@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-12-15 08:08:59 +01:00
Evan Miller
4006a27c5e scripts/entitlement.sh: Use backward-compatible cp flags
Older versions of Mac OS X do not support cp -a. The cp man page indicates
that -a is equivalent to -pPR.

Signed-off-by: Evan Miller <emmiller@gmail.com>
Message-Id: <40635C6E-059A-4146-B1E2-F6376700EE85@gmail.com>
[Leave out -R, these are files and not directories. - Paolo]
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-11-30 22:25:58 +01:00
John Snow
206439cd89 scripts/device-crash-test: Use a QMP timeout
Despite all the previous fixes, it's still possible for
device-crash-test to wedge itself in the case that QEMU terminates *so
quickly* that it doesn't even begin a connection attempt to our QMP
client. Python will just joyfully wait ad infinitum for a connection
that will now never arrive.

The real fix is to use asyncio to simultaneously poll both the health of
the launched process AND the connection attempt. That's quite a bit more
invasive than just setting a connection timeout, though.

Do the very simplest thing for now.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20211118204620.1897674-7-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-11-22 18:41:21 -05:00
Thomas Huth
418ce0201f Revert "device-crash-test: Ignore errors about a bus not being available"
This reverts commit ca89d15f8e.

There is already an entry for this kind of messages earlier in the
ERROR_RULE_LIST - when I added this patch, I just got fooled by
the other errors that occur due to a race between QMP connection
and QEMU terminating early (which still spit out the 'No bus found'
messages in their backtrace), but these other problems have now
fortunately been tackled by John Snow, so we certainly don't need
this duplicated entry here anymore.

Message-Id: <20211112072220.108580-1-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-11-17 10:20:17 +01:00
Richard Henderson
8d5fcb1990 Pull request
-----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEE+ber27ys35W+dsvQfe+BBqr8OQ4FAmGUTcIACgkQfe+BBqr8
 OQ6rvBAAqj2+TqbRcYbBGGCdxbIG7no6GvXGAElfBGZGfDhfn0QY8FelYn4tDwBg
 6QBMgNQSzDDSIs47dFZaLPE1yifZ4XqKA3Gg7B9ll5wT/qwbyv5NJOXyLEImtvrh
 9lxWddEzNX8FwlArCAHTEWdgkt6O2eHMcT/uk08Ik9RPYAk5ePZLU4awc0VFhDv8
 uHDGtgOy8QrpIWORZC3CbVxx/onhxBj24wq5x+3FmDpYIZvI2SFAwKrk/bsJD3X7
 BMtNJ5KXRrpvZH6Qkz/xUS/8frAreOeV7EN3EdB31VX60LnUbuYXxAer/vSBT1LR
 VALxcm+xHJLW38xhgzcWO/Z/PFGyj3R/M2xvsX+SucG93FbVqdNspUkTBIFVIxkv
 BMpfIRvsiUgBZt2CX70eKs9wmwPwG8w8kcoTqmaSkf8kvVX8xqA19k+U3j/67dkV
 mn3YQj5ub5Zh024t4yMmMcNGxPWUd29nXLm2RerEA27IDRgrDMfjUZj6HJNB9Gfm
 4h/Ij0nPTkxjNCtNvEh0mvzjrpLCVHdrR7RkomLxOZDYFMoTD+Kyw2apn4AuAu/7
 ba0BW4bvs+048q2XW8AfprUaZ8wJQy+ALteZOOrq7rvgBR7iXYjPoHtir5Jqptzw
 4NHMNihF+fYVELuuWEJLAMkViA/nhMHxOiHprpcdRMQFaClt5iM=
 =2l59
 -----END PGP SIGNATURE-----

Merge tag 'python-pull-request' of https://gitlab.com/jsnow/qemu into staging

Pull request

# gpg: Signature made Wed 17 Nov 2021 01:33:06 AM CET
# gpg:                using RSA key F9B7ABDBBCACDF95BE76CBD07DEF8106AAFC390E
# gpg: Good signature from "John Snow (John Huston) <jsnow@redhat.com>" [full]

* tag 'python-pull-request' of https://gitlab.com/jsnow/qemu:
  scripts/device-crash-test: hide tracebacks for QMP connect errors
  scripts/device-crash-test: don't emit AQMP connection errors to stdout
  scripts/device-crash-test: simplify Exception handling
  python/aqmp: fix ConnectError string method
  python/aqmp: Fix disconnect during capabilities negotiation

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-11-17 07:41:08 +01:00
John Snow
c398a241ec scripts/device-crash-test: hide tracebacks for QMP connect errors
Generally, the traceback for a connection failure is uninteresting and
all we need to know is that the connection attempt failed.

Reduce the verbosity in these cases, except when debugging.

Signed-off-by: John Snow <jsnow@redhat.com>
Reported-by: Thomas Huth <thuth@redhat.com>
Tested-by: Thomas Huth <thuth@redhat.com>
Message-id: 20211111143719.2162525-6-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-11-16 14:26:36 -05:00
John Snow
76f86e78b2 scripts/device-crash-test: don't emit AQMP connection errors to stdout
These errors are expected, so they shouldn't clog up terminal output. In
the event that they're *not* expected, we'll be seeing an awful lot more
output concerning the nature of the failure.

Reported-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Tested-by: Thomas Huth <thuth@redhat.com>
Message-id: 20211111143719.2162525-5-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-11-16 14:26:36 -05:00
John Snow
47b43acd57 scripts/device-crash-test: simplify Exception handling
We don't need to handle KeyboardInterruptError specifically; we can
instead tighten the scope of the broad Exception handlers to only catch
"Exception", which has the effect of allowing all BaseException classes
that do not inherit from Exception to be raised through.

KeyboardInterruptError and a few other important ones are
BaseExceptions, so this does the same thing with less code.

Signed-off-by: John Snow <jsnow@redhat.com>
Reported-by: Thomas Huth <thuth@redhat.com>
Tested-by: Thomas Huth <thuth@redhat.com>
Message-id: 20211111143719.2162525-4-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-11-16 14:26:36 -05:00
Richard Henderson
56f4f41e02 nbd patches for 2021-11-16
- Rich Jones: Add 'qemu-nbd --selinux-label' option for running Unix
   socket with appropriate SELinux labeling
 - Eric Blake: Address clang sanitizer warning
 -----BEGIN PGP SIGNATURE-----
 
 iQEzBAABCAAdFiEEccLMIrHEYCkn0vOqp6FrSiUnQ2oFAmGT3RoACgkQp6FrSiUn
 Q2oFEQf/XPOCJHluoE/BN9iL2kggqXQR65hEWPkITBGTxkADlqo1u9XwQRL8YbCF
 KvYp+tE1sCg4JmD81bUvAtu+foIE3GE/E3U6K8FhBsVI8kV3i/HJdUycmzxSbjU9
 yrf6uhnnmc95I3po50tojZM93L69JVcUKmIahj1c1BnwwdGuMaEZJ+rzpSLB5eiF
 iPCCjgpcZGv4LqbPG240foAF7bTeyMf0l4MjftiB+CBvgeO4NIxHSiN+IE3B5hHX
 +caNY9mcTGESwWf4bXVIlAhWAuHhP8YxlZZKQRQnUopKDF846oYhXJWLhgm9Q43z
 MHP8cn0oVJIKz+w1e/8MPC95tji+QA==
 =1qRZ
 -----END PGP SIGNATURE-----

Merge tag 'pull-nbd-2021-11-16' of https://repo.or.cz/qemu/ericb into staging

nbd patches for 2021-11-16

- Rich Jones: Add 'qemu-nbd --selinux-label' option for running Unix
  socket with appropriate SELinux labeling
- Eric Blake: Address clang sanitizer warning

# gpg: Signature made Tue 16 Nov 2021 05:32:26 PM CET
# gpg:                using RSA key 71C2CC22B1C4602927D2F3AAA7A16B4A2527436A
# gpg: Good signature from "Eric Blake <eblake@redhat.com>" [full]
# gpg:                 aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>" [full]
# gpg:                 aka "[jpeg image of size 6874]" [full]

* tag 'pull-nbd-2021-11-16' of https://repo.or.cz/qemu/ericb:
  nbd/server: Add --selinux-label option
  nbd/server: Silence clang sanitizer warning

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-11-16 18:55:36 +01:00
Cleber Rosa
d7c2e2b3f4 Jobs based on custom runners: add CentOS Stream 8
This introduces three different parts of a job designed to run
on a custom runner managed by Red Hat.  The goals include:

  a) propose a model for other organizations that want to onboard
     their own runners, with their specific platforms, build
     configuration and tests.

  b) bring awareness to the differences between upstream QEMU and the
     version available under CentOS Stream, which is "A preview of
     upcoming Red Hat Enterprise Linux minor and major releases".

  c) because of b), it should be easier to identify and reduce the gap
     between Red Hat's downstream and upstream QEMU.

The components of this custom job are:

  I) OS build environment setup code:

     - additions to the existing "build-environment.yml" playbook
       that can be used to set up CentOS/EL 8 systems.

     - a CentOS Stream 8 specific "build-environment.yml" playbook
       that adds to the generic one.

 II) QEMU build configuration: a script that will produce binaries with
     features as similar as possible to the ones built and packaged on
     CentOS stream 8.

III) Scripts that define the minimum amount of testing that the
     binaries built with the given configuration (point II) under the
     given OS build environment (point I) should be subjected to.

 IV) Job definition: GitLab CI jobs that will dispatch the build/test
     jobs (see points #II and #III) to the machine specifically
     configured according to #I.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Tested-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20211111160501.862396-2-crosa@redhat.com>
Message-Id: <20211115142915.3797652-6-alex.bennee@linaro.org>
2021-11-16 16:19:53 +00:00
Richard W.M. Jones
3d212b41e9 nbd/server: Add --selinux-label option
Under SELinux, Unix domain sockets have two labels.  One is on the
disk and can be set with commands such as chcon(1).  There is a
different label stored in memory (called the process label).  This can
only be set by the process creating the socket.  When using SELinux +
SVirt and wanting qemu to be able to connect to a qemu-nbd instance,
you must set both labels correctly first.

For qemu-nbd the options to set the second label are awkward.  You can
create the socket in a wrapper program and then exec into qemu-nbd.
Or you could try something with LD_PRELOAD.

This commit adds the ability to set the label straightforwardly on the
command line, via the new --selinux-label flag.  (The name of the flag
is the same as the equivalent nbdkit option.)

A worked example showing how to use the new option can be found in
this bug: https://bugzilla.redhat.com/show_bug.cgi?id=1984938

Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1984938
Signed-off-by: Richard W.M. Jones <rjones@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>

[eblake: rebase to configure changes, reject --selinux-label if it is
not compiled in or not used on a Unix socket]
Note that we may relax some of these restrictions at a later date,
such as making it possible to label a TCP socket, although it may be
smarter to do so as a generic QMP action rather than more one-off
command lines in qemu-nbd.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20211115202944.615966-1-eblake@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
[eblake: adjust meson output as suggested by thuth]
Signed-off-by: Eric Blake <eblake@redhat.com>
2021-11-16 10:16:38 -06:00
Philippe Mathieu-Daudé
f1f727ac8a tcg: Remove TCI experimental status
The following commits (released in v6.0.0) made raised the
quality of the TCI backend to the other TCG architectures,
thus is is not considerated experimental anymore:
- c6fbea47664..2f74f45e32b
- dc09f047edd..9e9acb7b348
- b6139eb0578..2fc6f16ca5e
- dbcbda2cd84..5e8892db93f

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20211106111457.517546-1-f4bug@amsat.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-11-11 11:47:01 +01:00
Thomas Huth
eea9453a01 Move the l2tpv3 test from configure to meson.build
And while we're at it, also provide a proper entry for this feature
in meson_options.txt, so that people who don't need it have a knob
to disable this feature.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20211028185910.1729744-3-thuth@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-11-02 15:57:28 +01:00
Paolo Bonzini
6638cae5f6 meson: bump submodule to 0.59.3
This gains some bugfixes, especially:

- it fixes the introspection of array options.  While technically we
still support Meson 0.58.2, this issue only appears when adding a new
option and not if the user is just building QEMU.  In the relatively
rare case of a contributor using --meson to point to a 0.58 version,
review can catch spurious changes to scripts/meson-buildoptions.sh
easily.

- it fixes "meson test" when it is not the process group leader.  Make is
the process group leader when "make check" invokes "meson test", so this
is a requirement for using it as a test harness.

Tested-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-11-02 15:57:28 +01:00
John Snow
3bd559467d iotests: Accommodate async QMP Exception classes
(But continue to support the old ones for now, too.)

There are very few cases of any user of QEMUMachine or a subclass
thereof relying on a QMP Exception type. If you'd like to check for
yourself, you want to grep for all of the derivatives of QMPError,
excluding 'AQMPError' and its derivatives. That'd be these:

- QMPError
- QMPConnectError
- QMPCapabilitiesError
- QMPTimeoutError
- QMPProtocolError
- QMPResponseError
- QMPBadPortError

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-id: 20211026175612.4127598-5-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-11-01 11:54:59 -04:00
Markus Armbruster
57df0dff1a qapi: Extend -compat to set policy for unstable interfaces
New option parameters unstable-input and unstable-output set policy
for unstable interfaces just like deprecated-input and
deprecated-output set policy for deprecated interfaces (see commit
6dd75472d5 "qemu-options: New -compat to set policy for deprecated
interfaces").  This is intended for testing users of the management
interfaces.  It is experimental.

For now, this covers only syntactic aspects of QMP, i.e. stuff tagged
with feature 'unstable'.  We may want to extend it to cover semantic
aspects, or the command line.

Note that there is no good way for management application to detect
presence of these new option parameters: they are not visible output
of query-qmp-schema or query-command-line-options.  Tolerable, because
it's meant for testing.  If running with -compat fails, skip the test.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Acked-by: John Snow <jsnow@redhat.com>
Message-Id: <20211028102520.747396-10-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[Doc comments fixed up]
2021-10-29 21:28:01 +02:00
Markus Armbruster
c868876043 qapi: Generalize enum member policy checking
The code to check enumeration value policy can see special feature
flag 'deprecated' in QEnumLookup member flags[value].  I want to make
feature flag 'unstable' visible there as well, so I can add policy for
it.

Instead of extending flags[], replace it by @special_features (a
bitset of QapiSpecialFeature), because that's how special features get
passed around elsewhere.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Acked-by: John Snow <jsnow@redhat.com>
Message-Id: <20211028102520.747396-8-armbru@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2021-10-29 18:24:56 +02:00
Markus Armbruster
6604e4757a qapi: Generalize command policy checking
The code to check command policy can see special feature flag
'deprecated' as command flag QCO_DEPRECATED.  I want to make feature
flag 'unstable' visible there as well, so I can add policy for it.

To let me make it visible, add member @special_features (a bitset of
QapiSpecialFeature) to QmpCommand, and adjust the generator to pass it
through qmp_register_command().  Then replace "QCO_DEPRECATED in
@flags" by QAPI_DEPRECATED in @special_features", and drop
QCO_DEPRECATED.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Acked-by: John Snow <jsnow@redhat.com>
Message-Id: <20211028102520.747396-7-armbru@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2021-10-29 18:24:46 +02:00
Markus Armbruster
a130728554 qapi: Generalize struct member policy checking
The generated visitor functions call visit_deprecated_accept() and
visit_deprecated() when visiting a struct member with special feature
flag 'deprecated'.  This makes the feature flag visible to the actual
visitors.  I want to make feature flag 'unstable' visible there as
well, so I can add policy for it.

To let me make it visible, replace these functions by
visit_policy_reject() and visit_policy_skip(), which take the member's
special features as an argument.  Note that the new functions have the
opposite sense, i.e. the return value flips.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20211028102520.747396-6-armbru@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
[Unbreak forward visitor]
2021-10-29 18:23:09 +02:00
Markus Armbruster
c67db1ed16 qapi: Tools for sets of special feature flags in generated code
New enum QapiSpecialFeature enumerates the special feature flags.

New helper gen_special_features() returns code to represent a
collection of special feature flags as a bitset.

The next few commits will put them to use.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-Id: <20211028102520.747396-5-armbru@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2021-10-29 15:56:29 +02:00
Markus Armbruster
9bafe07bc8 qapi: Eliminate QCO_NO_OPTIONS for a slight simplification
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-Id: <20211028102520.747396-4-armbru@redhat.com>
2021-10-29 15:55:52 +02:00
Markus Armbruster
aa2370444b qapi: Implement deprecated-input={reject,crash} for enum values
This copies the code implementing the policy from qapi/qmp-dispatch.c
to qapi/qobject-input-visitor.c.  Tolerable, but if we acquire more
copies, we should look into factoring them out.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Tested-by: Peter Krempa <pkrempa@redhat.com>
Acked-by: Peter Krempa <pkrempa@redhat.com>
Message-Id: <20211025042405.3762351-5-armbru@redhat.com>
2021-10-27 17:19:34 +02:00
Markus Armbruster
b6c18755e4 qapi: Add feature flags to enum members
This is quite similar to commit 84ab008687 "qapi: Add feature flags to
struct members", only for enums instead of structs.

Special feature flag 'deprecated' is silently ignored there.  This is
okay only because it will be implemented shortly.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20211025042405.3762351-3-armbru@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
2021-10-27 17:18:55 +02:00
Markus Armbruster
75ecee7262 qapi: Enable enum member introspection to show more than name
The next commit will add feature flags to enum members.  There's a
problem, though: query-qmp-schema shows an enum type's members as an
array of member names (SchemaInfoEnum member @values).  If it showed
an array of objects with a name member, we could simply add more
members to these objects.  Since it's just strings, we can't.

I can see three ways to correct this design mistake:

1. Do it the way we should have done it, plus compatibility goo.

   We want a ['SchemaInfoEnumMember'] member in SchemaInfoEnum.  Since
   changing @values would be a compatibility break, add a new member
   @members instead.

   @values is now redundant.  In my testing, output of
   qemu-system-x86_64's query-qmp-schema grows by 11% (18.5KiB).

   We can deprecate @values now and drop it later.  This will break
   outmoded clients.  Well-behaved clients such as libvirt are
   expected to break cleanly.

2. Like 1, but omit "boring" elements of @member, and empty @member.

   @values does not become redundant.  @members augments it.  Somewhat
   cumbersome, but output of query-qmp-schema grows only as we make
   enum members non-boring.

   There is nothing to deprecate here.

3. Versioned query-qmp-schema.

   query-qmp-schema provides either @values or @members.  The QMP
   client can select which version it wants.  There is no redundant
   output.

   We can deprecate old versions and eventually drop them.  This will
   break outmoded clients.  Breaking cleanly is easier than for 1.

   While 1 and 2 operate within the common rules for compatible
   evolution apply (section "Compatibility considerations" in
   docs/devel/qapi-code-gen.rst), 3 bypasses them.  Attractive when
   operating within the rules is just too awkward.  Not the case here.

This commit implements 1.  Libvirt developers prefer it.

Deprecate @values in favour of @members.  Since query-qmp-schema
compatibility is pretty fundamental for management applications, an
extended grace period is advised.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Tested-by: Peter Krempa <pkrempa@redhat.com>
Acked-by: Peter Krempa <pkrempa@redhat.com>
Message-Id: <20211025042405.3762351-2-armbru@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
2021-10-27 17:18:43 +02:00
Laurent Vivier
2c92be50bc analyze-migration.py: fix extract contents ('-x') errors
When we try to use 'analyze-migration.py -x' with python3,
we have the following errors:

  Traceback (most recent call last):
    File "scripts/analyze-migration.py", line 593, in <module>
      f.write(jsonenc.encode(dump.vmsd_desc))
  TypeError: a bytes-like object is required, not 'str'

  Traceback (most recent call last):
    File "scripts/analyze-migration.py", line 601, in <module>
      f.write(jsonenc.encode(dict))
  TypeError: a bytes-like object is required, not 'str'

This happens because the file 'f' is open in binary mode while
jsonenc.encode() returns a string.

The results are human-readable files, 'desc.json' and 'state.json',
so there is no reason to use the binary mode.

Signed-off-by: Laurent Vivier <lvivier@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20211015131645.501281-3-lvivier@redhat.com>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
2021-10-23 20:28:56 +02:00
Laurent Vivier
f98d372aef analyze-migration.py: fix a long standing typo
The parameters of '-d' can be either 'state' or 'desc', not 'dump'
as it is reported in the error message.

Fixes: b17425701d ("Add migration stream analyzation script")
Signed-off-by: Laurent Vivier <lvivier@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20211015131645.501281-2-lvivier@redhat.com>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
2021-10-23 20:28:56 +02:00
Philippe Mathieu-Daudé
bd0da3a3d4 roms/edk2: Only initialize required submodules
The EDK2 firmware images built to test QEMU do not require
the following submodules:

 - MdeModulePkg/Universal/RegularExpressionDxe/oniguruma
 - UnitTestFrameworkPkg/Library/CmockaLib/cmocka

The only submodules required are:

 - ArmPkg/Library/ArmSoftFloatLib/berkeley-softfloat-3
 - BaseTools/Source/C/BrotliCompress/brotli
 - CryptoPkg/Library/OpensslLib/openssl
 - MdeModulePkg/Library/BrotliCustomDecompressLib/brotli

Adapt the buildsys machinery to only initialize the required
submodules.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
Message-Id: <20211018105816.2663195-3-philmd@redhat.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-10-20 16:26:19 -07:00
Vladimir Sementsov-Ogievskiy
7d55a3bbad block-coroutine-wrapper.py: support BlockBackend first argument
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20211006131718.214235-7-vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
2021-10-15 15:51:33 -05:00
Paolo Bonzini
3b4da13293 configure: automatically parse command line for meson -D options
Right now meson_options.txt lists about 90 options.  Each option
needs code in configure to parse it and pass the option down to Meson as
a -D command-line argument; in addition the default must be duplicated
between configure and meson_options.txt.  This series tries to remove
the code duplication by generating the case statement for those --enable
and --disable options, as well as the corresponding help text.

About 80% of the options can be handled completely by the new mechanism.
Eight meson options are not of the --enable/--disable kind.  Six more need
to be parsed in configure for various reasons documented in the patch,
but they still have their help automatically generated.

The advantages are:

- less code in configure

- parsing and help is more consistent (for example --enable-blobs was
  not supported)

- options are described entirely in one place, meson_options.txt.
  This make it more attractive to use Meson options instead of
  hand-crafted configure options and config-host.mak

A few options change name: --enable-tcmalloc and --enable-jemalloc
become --enable-malloc={tcmalloc,jemalloc}; --disable-blobs becomes
--disable-install-blobs; --enable-trace-backend becomes
--enable-trace-backends.  However, the old names are allowed
for backwards compatibility.

Message-Id: <20211007130829.632254-19-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-10-14 09:51:06 +02:00
Paolo Bonzini
8b5fb29842 meson-buildoptions: include list of tracing backends
Manually patch the introspection data to include the tracing backends.
This works around a deficiency in Meson that will be fixed by
https://github.com/mesonbuild/meson/pull/9395.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-10-14 09:50:57 +02:00
Paolo Bonzini
61d63097be configure: prepare for auto-generated option parsing
Prepare the configure script and Makefile for automatically generated
help and parsing.

Because we need to run the script to generate the full help, we
cannot rely on the user supplying the path to a Python interpreter
with --python; therefore, the introspection output is parsed into
shell functions and stored in scripts/.  The converter is written
in Python as standard for QEMU, and this commit contains a stub.

Tested-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20211007130829.632254-18-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-10-14 09:50:57 +02:00
Paolo Bonzini
9c29b74100 trace: move configuration from configure to Meson
Cc: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20211007130829.632254-4-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-10-14 09:50:56 +02:00
Paolo Bonzini
654d6b0453 meson: switch minimum meson version to 0.58.2, minimum recommended to 0.59.2
Meson 0.58.2 does not need b_staticpic=$pie anymore, and has
stabilized the keyval module.  Remove the workaround and use a few
replacements for features deprecated in the 0.57.0 release cycle.

One feature that we would like to use is passing dependencies to
summary.  However, that was broken in 0.59.0 and 0.59.1.  Therefore,
use the embedded Meson if the host has anything older than 0.59.2,
but allow --meson= to use 0.58.2.

Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-10-05 13:10:29 +02:00
John Snow
d183e0481b qapi/parser: enable pylint checks
Signed-off-by: John Snow <jsnow@redhat.com>

Message-Id: <20210930205716.1148693-14-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-10-02 07:33:42 +02:00
John Snow
18e3673e0f qapi/parser: Silence too-few-public-methods warning
Eh. Not worth the fuss today. There are bigger fish to fry.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210930205716.1148693-13-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-10-02 07:33:42 +02:00
John Snow
2e28283e41 qapi/parser: enable mypy checks
Signed-off-by: John Snow <jsnow@redhat.com>

Message-Id: <20210930205716.1148693-12-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-10-02 07:33:42 +02:00
John Snow
15acf48cfe qapi/parser: Add FIXME for consolidating JSON-related types
The fix for this comment is forthcoming in a future commit, but this
will keep me honest. The linting configuration in ./python/setup.cfg
prohibits 'FIXME' comments. A goal of this long-running series is to
move ./scripts/qapi to ./python/qemu/qapi so that the QAPI generator is
regularly type-checked by GitLab CI.

This comment is a time-bomb to force me to address this issue prior to
that step.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210930205716.1148693-11-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-10-02 07:33:42 +02:00
John Snow
5f0d9f3bc7 qapi/parser: add type hint annotations (QAPIDoc)
Annotations do not change runtime behavior.
This commit consists of only annotations.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210930205716.1148693-10-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-10-02 07:33:42 +02:00
John Snow
e7ac60fcd0 qapi/parser: add import cycle workaround
Adding static types causes a cycle in the QAPI generator:
[schema -> expr -> parser -> schema]. It exists because the QAPIDoc
class needs the names of types defined by the schema module, but the
schema module needs to import both expr.py/parser.py to do its actual
parsing.

Ultimately, the layering violation is that parser.py should not have any
knowledge of specifics of the Schema. QAPIDoc performs double-duty here
both as a parser *and* as a finalized object that is part of the schema.

In this patch, add the offending type hints alongside the workaround to
avoid the cycle becoming a problem at runtime. See
https://mypy.readthedocs.io/en/latest/runtime_troubles.html#import-cycles
for more information on this workaround technique.

I see three ultimate resolutions here:

(1) Just keep this patch and use the TYPE_CHECKING trick to eliminate
    the cycle which is only present during static analysis.

(2) Don't bother to annotate connect_member() et al, give them 'object'
    or 'Any'. I don't particularly like this, because it diminishes the
    usefulness of type hints for documentation purposes. Still, it's an
    extremely quick fix.

(3) Reimplement doc <--> definition correlation directly in schema.py,
    integrating doc fields directly into QAPISchemaMember and relieving
    the QAPIDoc class of the responsibility. Users of the information
    would instead visit the members first and retrieve their
    documentation instead of the inverse operation -- visiting the
    documentation and retrieving their members.

My preference is (3), but in the short-term (1) is the easiest way to
have my cake (strong type hints) and eat it too (Not have import
cycles). Do (1) for now, but plan for (3).

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210930205716.1148693-9-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-10-02 07:33:42 +02:00
John Snow
f4c05aaf14 qapi/parser: Introduce NullSection
Here's the weird bit. QAPIDoc generally expects -- virtually everywhere
-- that it will always have a current section. The sole exception to
this is in the case that end_comment() is called, which leaves us with
*no* section. However, in this case, we also don't expect to actually
ever mutate the comment contents ever again.

NullSection is just a Null-object that allows us to maintain the
invariant that we *always* have a current section, enforced by static
typing -- allowing us to type that field as QAPIDoc.Section instead of
the more ambiguous Optional[QAPIDoc.Section].

end_section is renamed to switch_section and now accepts as an argument
the new section to activate, clarifying that no callers ever just
unilaterally end a section; they only do so when starting a new section.

Signed-off-by: John Snow <jsnow@redhat.com>

Message-Id: <20210930205716.1148693-8-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-10-02 07:33:42 +02:00
John Snow
1e20a77576 qapi/parser: clarify _end_section() logic
The "if self._section" clause in end_section is mysterious: In which
circumstances might we end a section when we don't have one?

QAPIDoc always expects there to be a "current section", only except
after a call to end_comment(). This actually *shouldn't* ever be 'None',
so let's remove that logic so I don't wonder why it's like this again in
three months.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210930205716.1148693-7-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-10-02 07:33:42 +02:00
John Snow
cd87c14cde qapi/parser: remove FIXME comment from _append_body_line
True, we do not check the validity of this symbol -- but we don't check
the validity of definition names during parse, either -- that happens
later, during the expr check. I don't want to introduce a dependency on
expr.py:check_name_str here and introduce a cycle.

Instead, rest assured that a documentation block is required for each
definition. This requirement uses the names of each section to ensure
that we fulfilled this requirement.

e.g., let's say that block-core.json has a comment block for
"Snapshot!Info" by accident. We'll see this error message:

In file included from ../../qapi/block.json:8:
../../qapi/block-core.json: In struct 'SnapshotInfo':
../../qapi/block-core.json:38: documentation comment is for 'Snapshot!Info'

That's a pretty decent error message.

Now, let's say that we actually mangle it twice, identically:

../../qapi/block-core.json: In struct 'Snapshot!Info':
../../qapi/block-core.json:38: struct has an invalid name

That's also pretty decent. If we forget to fix it in both places, we'll
just be back to the first error.

Therefore, let's just drop this FIXME and adjust the error message to
not imply a more thorough check than is actually performed.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210930205716.1148693-6-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-10-02 07:33:42 +02:00
John Snow
012336a152 qapi/parser: fix unused check_args_section arguments
Pylint informs us we're not using these arguments. Oops, it's
right. Correct the error message and remove the remaining unused
parameter.

Fix test output now that the error message is improved.

Fixes: e151941d1b
Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210930205716.1148693-4-jsnow@redhat.com>
[Commit message formatting tweaked]
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-10-02 07:33:41 +02:00
John Snow
2adb988ed4 qapi/gen: use dict.items() to iterate over _modules
New pylint warning. I could silence it, but this is the only occurrence
in the entire tree, including everything in iotests/ and python/. Easier
to just change this one instance.

(The warning is emitted in cases where you are fetching the values
anyway, so you may as well just take advantage of the iterator to avoid
redundant lookups.)

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210930205716.1148693-3-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-10-02 07:33:41 +02:00
John Snow
1c00917409 qapi/pylintrc: ignore 'consider-using-f-string' warning
Pylint 2.11.x adds this warning. We're not yet ready to pursue that
conversion, so silence it for now.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210930205716.1148693-2-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-10-02 07:33:41 +02:00
Markus Armbruster
4e99f4b12c qapi: Drop simple unions
Simple unions predate flat unions.  Having both complicates the QAPI
schema language and the QAPI generator.  We haven't been using simple
unions in new code for a long time, because they are less flexible and
somewhat awkward on the wire.

The previous commits eliminated simple union from the tree.  Now drop
them from the QAPI schema language entirely, and update mentions of
"flat union" to just "union".

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210917143134.412106-22-armbru@redhat.com>
2021-09-27 08:23:25 +02:00
Markus Armbruster
8ebc3120e1 qapi: Stop enforcing "type name should not end in 'Kind'
I'm about to convert simple unions to flat unions, then drop simple
union support.  The conversion involves making the implict enum types
explicit.  To reduce churn, I'd like to name them exactly like the
implicit types they replace.  However, these names are reserved for
the generator's use.  They won't be once simple unions are gone.  Stop
enforcing this naming rule now rather than then.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210917143134.412106-3-armbru@redhat.com>
2021-09-25 06:55:54 +02:00
Vladimir Sementsov-Ogievskiy
5b3f7daaec simplebench: add img_bench_templater.py
Add simple grammar-parsing template benchmark. New tool consume test
template written in bash with some special grammar injections and
produces multiple tests, run them and finally print a performance
comparison table of different tests produced from one template.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20210824101517.59802-2-vsementsov@virtuozzo.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
2021-09-15 15:54:07 +02:00
Markus Armbruster
62f27589f8 qapi: Fix bogus error for 'if': { 'not': '' }
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210908045428.2689093-6-armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
[check_infix()'s type hint fixed]
2021-09-08 15:30:30 +02:00
Markus Armbruster
e2ff14a574 qapi: Bury some unused code in class Indentation
.__int__() has never been used.  Drop it.

.decrease() raises ArithmeticError when asked to decrease indentation
level below zero.  Nothing catches it.  It's a programming error.
Dumb down to assert.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210908045428.2689093-4-armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
2021-09-08 15:30:10 +02:00
Markus Armbruster
916fca17c7 qapi: Drop Indentation.__bool__()
Intentation.__bool__() is not worth its keep: it has just one user,
which can just as well check .__str__() instead.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210908045428.2689093-3-armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
2021-09-08 15:30:10 +02:00
Markus Armbruster
7b275cdd69 qapi: Fix a botched type annotation
Mypy is unhappy:

    $ mypy --config-file=scripts/qapi/mypy.ini `git-ls-files scripts/qapi/\*py`
    scripts/qapi/common.py:208: error: Function is missing a return type annotation
    scripts/qapi/common.py:227: error: Returning Any from function declared to return "str"

Messed up in commit ccea6a8637 "qapi: Factor common recursion out of
cgen_ifcond(), docgen_ifcond()".  Tidy up.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210908045428.2689093-2-armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
2021-09-08 15:30:10 +02:00
Thomas Huth
bf6a618556 scripts: Remove the "show-fixed-bugs.sh" file
Since we are not using Launchpad anymore, there is no more need for
this script.

Message-Id: <20210825142143.142037-1-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Laurent Vivier <laurent@vivier.eu>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-09-06 10:00:14 +02:00
Markus Armbruster
34f7b25e57 qapi: Tweak error messages for unknown / conflicting 'if' keys
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210831123809.1107782-13-armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
2021-09-03 17:09:10 +02:00
Markus Armbruster
6dcf03719a qapi: Tweak error messages for missing / conflicting meta-type
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210831123809.1107782-12-armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
2021-09-03 17:09:10 +02:00
Markus Armbruster
555dd1aaa6 qapi: Use re.fullmatch() where appropriate
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210831123809.1107782-10-armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
2021-09-03 17:09:10 +02:00
Markus Armbruster
d0830ee443 qapi: Use "not COND" instead of "!COND" for generated documentation
Generated documentation uses operators "and", "or", and "!".  Change
the latter to "not".

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210831123809.1107782-9-armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
2021-09-03 17:09:10 +02:00
Markus Armbruster
a7987799d1 qapi: Avoid redundant parens in code generated for conditionals
Commit 6cc2e4817f "qapi: introduce QAPISchemaIfCond.cgen()" caused a
minor regression: redundant parenthesis.  Subsequent commits
eliminated of many of them, but not all.  Get rid of the rest now.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210831123809.1107782-8-armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
2021-09-03 17:09:10 +02:00
Markus Armbruster
ccea6a8637 qapi: Factor common recursion out of cgen_ifcond(), docgen_ifcond()
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210831123809.1107782-7-armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
2021-09-03 17:09:10 +02:00
Markus Armbruster
82ca72c023 qapi: Fix C code generation for 'if'
When commit 5d83b9a130 "qapi: replace if condition list with dict
{'all': [...]}" made cgen_ifcond() and docgen_ifcond() recursive, it
messed up parenthesises in the former, and got them right in the
latter, as the previous commit demonstrates.

To fix, adopt the latter's working code for the former.  This
generates the correct code from the previous commit's commit message.

Fixes: 5d83b9a130
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210831123809.1107782-6-armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
2021-09-03 17:09:10 +02:00
Markus Armbruster
e46c930cdd qapi: Simplify how QAPISchemaIfCond represents "no condition"
None works fine, there is no need to replace it by {} in .__init__().

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210831123809.1107782-3-armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
2021-09-03 17:09:10 +02:00
Markus Armbruster
1889e57a71 qapi: Simplify QAPISchemaIfCond's interface for generating C
QAPISchemaIfCond.cgen() is only ever used like

    gen_if(ifcond.cgen())

and

    gen_endif(ifcond.cgen())

Simplify to

    ifcond.gen_if()

and

    ifcond.gen_endif()

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210831123809.1107782-2-armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
[Import statements tidied up with isort]
2021-09-03 17:06:40 +02:00
Alexander Bulekov
dfc86c0f25 fuzz: add an instrumentation filter
By default, -fsanitize=fuzzer instruments all code with coverage
information. However, this means that libfuzzer will track coverage over
hundreds of source files that are unrelated to virtual-devices. This
means that libfuzzer will optimize inputs for coverage observed in timer
code, memory APIs etc. This slows down the fuzzer and stores many inputs
that are not relevant to the actual virtual-devices.

With this change, clang versions that support the
"-fsanitize-coverage-allowlist" will only instrument a subset of the
compiled code, that is directly related to virtual-devices.

Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
Reviewed-by: Darren Kenny <darren.kenny@oracle.com>
2021-09-01 07:33:13 -04:00
Marc-André Lureau
8a9f1e1d9c qapi: make 'if' condition strings simple identifiers
Change the 'if' condition strings to be C-agnostic. It will accept
'[A-Z][A-Z0-9_]*' identifiers. This allows to express configuration
conditions in other languages (Rust or Python for ex) or other more
suitable forms.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Tested-by: John Snow <jsnow@redhat.com>
Message-Id: <20210804083105.97531-11-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Rebased with semantic conflict in redefined-event.json]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-08-26 13:53:56 +02:00
Marc-André Lureau
2b7d214536 qapi: add 'not' condition operation
For the sake of completeness, introduce the 'not' condition.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20210804083105.97531-10-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Long line broken in tests/qapi-schema/qapi-schema-test.json]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-08-26 13:53:56 +02:00
Marc-André Lureau
3ad64edfad qapi: add 'any' condition
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20210804083105.97531-8-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-08-26 13:53:56 +02:00
Marc-André Lureau
5d83b9a130 qapi: replace if condition list with dict {'all': [...]}
Replace the simple list sugar form with a recursive structure that will
accept other operators in the following commits (all, any or not).

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20210804083105.97531-7-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Accidental code motion undone.  Degenerate :forms: comment dropped.
Helper _check_if() moved.  Error messages tweaked.  ui.json updated.
Accidental changes to qapi-schema-test.json dropped.]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-08-26 13:53:56 +02:00
Marc-André Lureau
d806f89f87 qapidoc: introduce QAPISchemaIfCond.docgen()
Instead of building the condition documentation from a list of string,
use the result generated from QAPISchemaIfCond.docgen().

This changes the generated documentation from:
- COND1, COND2... (where COND1, COND2 are Literal nodes, and ',' is Text)
to:
- COND1 and COND2 (the whole string as a Literal node)

This will allow us to generate more complex conditions in the following
patches, such as "(COND1 and COND2) or COND3".

Adding back the differentiated formatting is left to the wish list.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20210804083105.97531-6-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[TODO comment added]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-08-26 13:53:56 +02:00
Marc-André Lureau
6cc2e4817f qapi: introduce QAPISchemaIfCond.cgen()
Instead of building prepocessor conditions from a list of string, use
the result generated from QAPISchemaIfCond.cgen() and hide the
implementation details.

Note: this patch introduces a minor regression, generating a redundant
pair of parenthesis. This is mostly fixed in a later patch in this
series ("qapi: replace if condition list with dict [..]")

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20210804083105.97531-5-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Commit message tweaked]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-08-26 13:53:56 +02:00
Marc-André Lureau
33aa3267ba qapi: add QAPISchemaIfCond.is_present()
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210804083105.97531-4-marcandre.lureau@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-08-26 13:53:56 +02:00
Marc-André Lureau
f17539c80d qapi: wrap Sequence[str] in an object
Mechanical change, except for a new assertion in
QAPISchemaEntity.ifcond().

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20210804083105.97531-3-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Rebased with obvious conflicts, commit message adjusted]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-08-26 13:53:56 +02:00
Markus Armbruster
b32abbb2f5 qapi: Fix crash on redefinition with a different condition
QAPISchema._make_implicit_object_type() asserts that when an implicit
object type is used multiple times, @ifcond is the same for all uses.
It will be for legitimate uses, i.e. simple union branch wrapper
types.  A comment explains this.

The assertion fails when a command or event is redefined with a
different condition.  The redefinition is an error, but it's flagged
only later.

Fixing the assertion would complicate matters further.  Not
worthwhile, drop it instead.  We really need to get rid of simple
unions.

Tweak test case redefined-event to cover redefinition with a different
condition.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210806120510.2367124-1-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2021-08-26 13:53:56 +02:00
Alexander Bulekov
3973e7ae63 fuzz: avoid building twice, when running on gitlab
On oss-fuzz, we build twice, to put together a build that is portable to
the runner containers. On gitlab ci, this is wasteful and contributes to
timeouts on the build-oss-fuzz job. Avoid building twice on gitlab, at
the remote cost of potentially missing some cases that break oss-fuzz
builds.

Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
Reviewed-by: Darren Kenny <darren.kenny@oracle.com>
Message-Id: <20210809111621.54454-1-alxndr@bu.edu>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-08-11 13:40:01 +02:00
Paolo Bonzini
e17bdaab2b coverity-model: write models fully for non-array allocation functions
Coverity seems to have issues figuring out the properties of g_malloc0
and other non *_n functions.  While this was "fixed" by removing the
custom second argument to __coverity_mark_as_afm_allocated__, inline
the code from the array-based allocation functions to avoid future
issues.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-07-30 12:04:01 +02:00
Paolo Bonzini
0da41187df coverity-model: constrain g_malloc/g_malloc0/g_realloc as never returning NULL
g_malloc/g_malloc0/g_realloc only return NULL if the size is 0; we do not need
to cover that in the model, and so far have expected __coverity_alloc__
to model a non-NULL return value.  But that apparently does not work
anymore, so add some extra conditionals that invoke __coverity_panic__
for NULL pointers.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-07-30 12:04:01 +02:00
Paolo Bonzini
05ad6857a5 coverity-model: clean up the models for array allocation functions
sz is only used in one place, so replace it with nmemb * size in
that one place.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-07-30 12:04:01 +02:00
Paolo Bonzini
96915d638c coverity-model: remove model for more allocation functions
These models are not needed anymore now that Coverity does not check
anymore that the result is used with "g_free".  Coverity understands
GCC attributes and uses them to detect leaks.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-07-30 12:04:01 +02:00
Paolo Bonzini
243a545bff coverity-model: make g_free a synonym of free
Recently, Coverity has started complaining about using g_free() to free
memory areas allocated by GLib functions not included in model.c,
such as g_strfreev.  This unfortunately goes against the GLib
documentation, which suggests that g_malloc() should be matched
with g_free() and plain malloc() with free(); since GLib 2.46 however
g_malloc() is hardcoded to always use the system malloc implementation,
and g_free is just "free" plus a tracepoint.  Therefore, this
should not cause any problem in practice.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-07-30 12:04:01 +02:00
Paolo Bonzini
d4b3d152ee coverity-model: update address_space_read/write models
Use void * for consistency with the actual function; provide a model
for MemoryRegionCache functions and for address_space_rw.  These
let Coverity understand the bounds of the data that various functions
read and write even at very high levels of inlining (e.g. pci_dma_read).

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-07-30 12:04:01 +02:00
Peter Maydell
9ad4c7c9b6 QAPI patches patches for 2021-07-15
-----BEGIN PGP SIGNATURE-----
 
 iQJGBAABCAAwFiEENUvIs9frKmtoZ05fOHC0AOuRhlMFAmDwKlYSHGFybWJydUBy
 ZWRoYXQuY29tAAoJEDhwtADrkYZTR2UP+wVKKbY09lvPlpb25DJe073EQspRqBqg
 hG8R4D82Xics3XMfAJ/QcCLmtEbYKT02BCyY069rP6fnD8ZS5DfUeiIXrXBajN5K
 eVFuIO/mV7qNvx8mWyvZ7X5BWoolEDxkx+TjFfu6ZgSDMFPCjKE729/IXpAPhHll
 lYW2j2RLHUJCTRq+IRv36E11LFKI1zDewA8q5NHARxIAyMBLT9UFKLpnw1Fe5V9C
 w/2aB1Uh78oAfB6BtazvTHVWT75pca/W/jzyMcdXMVaPCJrsODEvz6Gn50gKYG+L
 J7/m6ivjCjitLGk9H+h8TaC/Q/wIE36gMjd2XT4MhgxF14/bMZpX1xlMO8De9DMn
 XhONKRGtXYV9r2uLhc0C8kID6TEidIDBNXDrtX1hDL172AfJ+Q7fDTqqCSUk5C2j
 dbNQISOlEWTJ1moOiRPjozGrcap5/Z/CViUrrrWXvAZoHScEoHgX+3QyRYD0MfeC
 0Q55kdx0Pu5W4qwg/EEBSzMUSy82VzJzgQTBmfmTxeEOqfUpfKu53ZNBqOO+1ZJd
 FsPzqnekNDcDrXtsg4TOuAnvShF8NW9nVnx3JeNUilcrjN0lIsnf7wrWUQQD66wA
 ZyboB9r/b0WdTQ9f+Zpi/i1ufEu9CSBseRrLac2noOOOrHLiURR8TYr3p6i6g0Fk
 jY9uCC/WnPto
 =dZGh
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/armbru/tags/pull-qapi-2021-07-15' into staging

QAPI patches patches for 2021-07-15

# gpg: Signature made Thu 15 Jul 2021 13:30:14 BST
# gpg:                using RSA key 354BC8B3D7EB2A6B68674E5F3870B400EB918653
# gpg:                issuer "armbru@redhat.com"
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>" [full]
# gpg:                 aka "Markus Armbruster <armbru@pond.sub.org>" [full]
# Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867  4E5F 3870 B400 EB91 8653

* remotes/armbru/tags/pull-qapi-2021-07-15:
  qapi: Fix crash on missing enum member name

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-07-16 10:56:03 +01:00
Markus Armbruster
a0c7b99bf7 qapi: Fix crash on missing enum member name
New test case enum-dict-no-name.json crashes:

    $ python3 scripts/qapi-gen.py tests/qapi-schema/enum-dict-no-name.json
    Traceback (most recent call last):
    [...]
      File "/work/armbru/qemu/scripts/qapi/expr.py", line 458, in check_enum
	member_name = member['name']
    KeyError: 'name'

Root cause: we try to retrieve member 'name' before we check for
missing members.  With that fixed, we get the expected error "'data'
member misses key 'name'".

Fixes: 0825f62c84
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210616072121.626431-1-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
2021-07-15 12:56:41 +02:00
Daniel P. Berrangé
51f5c849c1 hw/usb/ccid: remove references to NSS
The NSS package was previously pre-requisite for building CCID related
features, however, this became obsolete when the libcacard library was
spun off to a separate project:

    commit 7b02f5447c
    Author: Marc-André Lureau <marcandre.lureau@redhat.com>
    Date:   Sun Aug 30 11:48:40 2015 +0200

        libcacard: use the standalone project

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210623142245.307776-2-berrange@redhat.com>
Message-Id: <20210709143005.1554-10-alex.bennee@linaro.org>
2021-07-14 14:33:53 +01:00
Cleber Rosa
40de78c284 Jobs based on custom runners: docs and gitlab-runner setup playbook
To have the jobs dispatched to custom runners, gitlab-runner must
be installed, active as a service and properly configured.  The
variables file and playbook introduced here should help with those
steps.

The playbook introduced here covers the Linux distributions and
has been primarily tested on OS/machines that the QEMU project
has available to act as runners, namely:

 * Ubuntu 20.04 on aarch64
 * Ubuntu 18.04 on s390x

But, it should work on all other Linux distributions.  Earlier
versions were tested on FreeBSD too, so chances of success are
high.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Willian Rampazzo <willianr@redhat.com>
Tested-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210630012619.115262-4-crosa@redhat.com>
Message-Id: <20210709143005.1554-4-alex.bennee@linaro.org>
2021-07-14 14:31:48 +01:00
Cleber Rosa
159c5d177b Jobs based on custom runners: build environment docs and playbook
To run basic jobs on custom runners, the environment needs to be
properly set up.  The most common requirement is having the right
packages installed.

The playbook introduced here covers the QEMU's project s390x and
aarch64 machines.  At the time this is being proposed, those machines
have already had this playbook applied to them.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210630012619.115262-3-crosa@redhat.com>
Message-Id: <20210709143005.1554-3-alex.bennee@linaro.org>
2021-07-14 14:31:48 +01:00
Peter Maydell
2a54fc454c Linux-user pull request 20210713
Update headers to linux v5.13
 cleanup errno target headers
 Fix race condition on fd translation table
 -----BEGIN PGP SIGNATURE-----
 
 iQJGBAABCAAwFiEEzS913cjjpNwuT1Fz8ww4vT8vvjwFAmDtmAUSHGxhdXJlbnRA
 dml2aWVyLmV1AAoJEPMMOL0/L748BFEP/jl0taLq3vqwUuAmsirJadf4K5+4iFiO
 /FhhAw5kzNasrlO2WrvTazDpJsAotuPeI2kQpnuEruDPOJulf21SUWRVeOUmzYBQ
 V/mKKkazY7FnmI8jhO8FhFrNGrOQWEGtU5adNX0bJwArbhfSWMfBTCfaoWePUmVO
 26X2gNchHwvymoLyqKG7fA1AahSQEqgE6wG9L3n6eXxPy8xBEj0WDQoISakTKJJ4
 J7Z2QchsYC51A2kpkA77TF+W2yLRrOwO8XNLQNOjSuPyvi6v0nsbe6mTSxVOmEcN
 3PYezsAWHg0rptIWuGwnVN2+juQt2pp8T3sADRWAMrgnkCrBkiwgWmBNzWuPXLX6
 xoXd1//lZcmGRwmJzvvWs1+1IZsVcRyWq9nAg/+CTbrTe+TC2q5a90jkEPup9bkx
 up/xKJJwFyl4j/eeUfmWBruAjX0PCCtgpcVTMafWSTD9maooHVTOJmy6ZETgCCXC
 KvRoWNK1Cx94KblHrGU/mWxBT9OhqPJxlvPswKDYidS0bMOoVt8eqFI7GRKxX7qK
 wldbNiqLBB0ZxHLZPP3TgE96qVr+PSP9UfShWC+X5Cl5o4/m0tvq+V0dq3PBsz4O
 Zq7NQ1cU7z7rBcyqDFiREVzucScCY3rch3E4xGJASulrqm3HFJHJXuaBcFUMJhUt
 61puHc6tOyBp
 =Stii
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/vivier2/tags/linux-user-for-6.1-pull-request' into staging

Linux-user pull request 20210713

Update headers to linux v5.13
cleanup errno target headers
Fix race condition on fd translation table

# gpg: Signature made Tue 13 Jul 2021 14:41:25 BST
# gpg:                using RSA key CD2F75DDC8E3A4DC2E4F5173F30C38BD3F2FBE3C
# gpg:                issuer "laurent@vivier.eu"
# gpg: Good signature from "Laurent Vivier <lvivier@redhat.com>" [full]
# gpg:                 aka "Laurent Vivier <laurent@vivier.eu>" [full]
# gpg:                 aka "Laurent Vivier (Red Hat) <lvivier@redhat.com>" [full]
# Primary key fingerprint: CD2F 75DD C8E3 A4DC 2E4F  5173 F30C 38BD 3F2F BE3C

* remotes/vivier2/tags/linux-user-for-6.1-pull-request:
  linux-user: update syscall.tbl to Linux v5.13
  linux-user, mips: update syscall-args-o32.c.inc to Linux v5.13
  linux-user: update syscall_nr.h to Linux v5.13
  fd-trans: Fix race condition on reallocation of the translation table.
  linux-user/syscall: Remove ERRNO_TABLE_SIZE check
  linux-user: Simplify host <-> target errno conversion using macros
  linux-user/mips: Move errno definitions to 'target_errno_defs.h'
  linux-user/hppa: Move errno definitions to 'target_errno_defs.h'
  linux-user/alpha: Move errno definitions to 'target_errno_defs.h'
  linux-user: Extract target errno to 'target_errno_defs.h'
  linux-user/sparc: Rename target_errno.h -> target_errno_defs.h
  linux-user/syscall: Fix RF-kill errno (typo in ERFKILL)

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-07-13 21:58:57 +01:00
Laurent Vivier
e2dcdcea23 linux-user, mips: update syscall-args-o32.c.inc to Linux v5.13
Updated running scripts/update-mips-syscall-args.sh

scripts/update-mips-syscall-args.sh has been updated to reflect
file directory changes in strace repository.

Signed-off-by: Laurent Vivier <laurent@vivier.eu>
Message-Id: <20210708215756.268805-3-laurent@vivier.eu>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
2021-07-13 13:59:59 +02:00
Gerd Hoffmann
2adf216491 qemu-trace-stap: changing SYSTEMTAP_TAPSET considered harmful.
Setting SYSTEMTAP_TAPSET to some value other than
/usr/share/systemtap/tapsets results in systemtap not finding the
standard tapset library any more, which in turn breaks tracing because
pid() and other standard systemtap functions are not available any more.

So using SYSTEMTAP_TAPSET to point systemtap to the qemu probes will
only work for the prefix=/usr installs because both qemu and system
tapsets in the same directory then.  All other prefixes are broken.

Fix that by using the "-I $tapsetdir" command line switch instead.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-id: 20210601132414.432430-2-kraxel@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2021-07-12 17:37:06 +01:00
Akihiko Odaki
411ad8dd80 meson: Use input/output for entitlements target
input/output parameters respect dependencies.

Signed-off-by: Akihiko Odaki <akihiko.odaki@gmail.com>
Message-Id: <20210709012533.58262-1-akihiko.odaki@gmail.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-07-09 18:21:34 +02:00
Jose R. Ziviani
af19eecf84 modules: check if all dependencies can be satisfied
Verifies if all dependencies are correctly listed in the modinfo.c too
and stop the builds if they're not.

Signed-off-by: Jose R. Ziviani <jziviani@suse.de>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Jose R. Ziviani <jziviani@suse.de>
Message-Id: <20210624103836.2382472-5-kraxel@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-07-09 18:20:27 +02:00
Gerd Hoffmann
5ebbfecc3e modules: generate modinfo.c
Add script to generate C source with a small
database containing the module meta-data.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Jose R. Ziviani <jziviani@suse.de>
Message-Id: <20210624103836.2382472-4-kraxel@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-07-09 18:20:27 +02:00
Gerd Hoffmann
f5723ab665 modules: collect module meta-data
Add script to collect the module meta-data from the source code,
store the results in *.modinfo files.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Jose R. Ziviani <jziviani@suse.de>
Message-Id: <20210624103836.2382472-3-kraxel@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-07-09 18:20:27 +02:00
John Arbuckle
3983a767e3 Set icon for QEMU binary on Mac OS
Before switching the build system over to Meson, an icon was
added to the QEMU binary on Mac OS. This patch adds back that
feature; it piggybacks on the existing scripts/entitlement.sh,
which already does in-place changes to the executable on Darwin.

Signed-off-by: John Arbuckle <programmingkidx@gmail.com>
Message-Id: <20210705195328.36442-1-programmingkidx@gmail.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-07-06 09:35:08 +02:00
Paolo Bonzini
dd52af17ec coverity-scan: switch to vpath build
This is the patch that has been running on the coverity cronjob
for a few weeks now.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-07-06 07:54:50 +02:00
Philippe Mathieu-Daudé
dff5f68224 coverity-scan: Remove lm32 / unicore32 targets
lm32 has been removed in commit 9d49bcf699 ("Drop the deprecated
lm32 target"), and unicore32 in 4369223902 ("Drop the deprecated
unicore32 target").

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20210619091342.3660495-2-f4bug@amsat.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-07-06 07:49:41 +02:00
Alex Bennée
66cf70149a scripts/checkpatch: roll diff tweaking into checkpatch itself
Rather than relying on external tweaks lets just do it inside
checkpatch's direct commitish handling which is QEMU specific code
anyway.

Suggested-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20210623102749.25686-4-alex.bennee@linaro.org>
2021-06-25 10:08:33 +01:00
Peter Maydell
0add99ea3e Python Pull request
Moves QMP-related tools not used for build or automatic testing from
 scripts/ to python/qemu/qmp/ where they will be protected from bitrot by
 the check-python-* CI jobs.
 
 stub forwarders are left in the old locations for now.
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEE+ber27ys35W+dsvQfe+BBqr8OQ4FAmDNJhAACgkQfe+BBqr8
 OQ5uZg/8CxwwDVskXya2BWa0WHrqQGmods2XMVupp9qI5HQAPE6AzlypUzvoA+6o
 jr5uenG/CGSNZgWUfGN3UxCw85knEGLekU9u3Rx/N+CmlHvNPZz8UUg0oOy9+QIV
 owFYXPq4Uc9siUjvFhFNmC6ZIUGf2y5vXOECqjTZL4SM9Qq0F3mdrFUdT78AR2jw
 HTUipCiaZQLFArlxqIFvIv/wR4i+zh9cXs4A31THjoLBz6kkgX89D+yb9cUDwcjp
 fDBYWuAN7dJzt1jqxuD1yUj3rqwKj0MZa6QXd0pKBQt3fHJCE4geY+mGzmGd9nTx
 ZZ8RI64CEjH946eNpRFCl+DPqLNE88pTbRpUHXy3YOvNj1ZWGjPtA6aDqcpQdySL
 UcQxc5K0zKROh9vUIMmmXW4oGlv9xgilO0BnMq5oMTvolaflaF1JdPpmjd+0pHJ2
 u9/I/+Bc5tJEmSYPu4ASh0UVllAVSSjnopLcjLJv9g8yApDxuodfyNafdctbsVmL
 YjiSedsPI2Hawm7O54cE1D6fkjopxwFaEuv1CyTqiDcULk6qWSDj3KtnrDOp0BNV
 qpN6ReK75UHBzhkdgoKWLm3WuZZk9s2r58e9GAPb7UxR3EiaPOBLOYrKawooqiI7
 +LDTqXNbVQJBpe1IgkygZiy2GuQ8unQCsxqhjy7nn6M01/SrPEc=
 =YTLz
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/jsnow-gitlab/tags/python-pull-request' into staging

Python Pull request

Moves QMP-related tools not used for build or automatic testing from
scripts/ to python/qemu/qmp/ where they will be protected from bitrot by
the check-python-* CI jobs.

stub forwarders are left in the old locations for now.

# gpg: Signature made Sat 19 Jun 2021 00:02:40 BST
# gpg:                using RSA key F9B7ABDBBCACDF95BE76CBD07DEF8106AAFC390E
# gpg: Good signature from "John Snow (John Huston) <jsnow@redhat.com>" [full]
# Primary key fingerprint: FAEB 9711 A12C F475 812F  18F2 88A9 064D 1835 61EB
#      Subkey fingerprint: F9B7 ABDB BCAC DF95 BE76  CBD0 7DEF 8106 AAFC 390E

* remotes/jsnow-gitlab/tags/python-pull-request: (72 commits)
  scripts/qmp-shell: add redirection shim
  python: add qmp-shell entry point
  scripts/qmp-shell: move to python/qemu/qmp/qmp_shell.py
  scripts/qmp-shell: add docstrings
  scripts/qmp-shell: make QMPShellError inherit QMPError
  scripts/qmp-shell: remove double-underscores
  scripts/qmp-shell: convert usage comment to docstring
  scripts/qmp-shell: Remove too-broad-exception
  scripts/qmp-shell: Fix empty-transaction invocation
  scripts/qmp-shell: remove TODO
  scripts/qmp-shell: use logging to show warnings
  scripts/qmp-shell: Use context manager instead of atexit
  python/qmp: return generic type from context manager
  scripts/qmp-shell: unprivatize 'pretty' property
  scripts/qmp-shell: Accept SocketAddrT instead of string
  scripts/qmp-shell: add mypy types
  python/qmp: add QMPObject type alias
  scripts/qmp-shell: initialize completer early
  scripts/qmp-shell: refactor QMPCompleter
  scripts/qmp-shell: Fix "FuzzyJSON" parser
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-06-21 16:11:33 +01:00
Peter Maydell
53f306f316 x86 queue, 2021-06-18
Features:
 * Add ratelimit for bus locks acquired in guest (Chenyi Qiang)
 
 Documentation:
 * SEV documentation updates (Tom Lendacky)
 * Add a table showing x86-64 ABI compatibility levels (Daniel P. Berrangé)
 
 Automated changes:
 * Update Linux headers to 5.13-rc4 (Eduardo Habkost)
 -----BEGIN PGP SIGNATURE-----
 
 iQJIBAABCAAyFiEEWjIv1avE09usz9GqKAeTb5hNxaYFAmDM+T4UHGVoYWJrb3N0
 QHJlZGhhdC5jb20ACgkQKAeTb5hNxabUrQ/+PtiJjd1cW9nhA0kWu8dVGq3xXJb4
 Nbma86tRPKBauTeQCLccXEvUjLqgFejeQlArhq4QKErLisXu4TDuQ+GeAfdR7h5P
 MTMSo0C665cT2/NbrwQizSPQdrNEgZAYRaDRafZLQTJ1TStzWDB1Vg79rzpWPcn0
 76XjIfSdGZUa4B1OvjNvUFq/SXf+0hW75soCwRhDNh5tfzfyct0XCSRF/wTXqyR/
 7yxDtfTzUAvT+6l3qb8ky+wqUTIY58BgjbdIGhyAUr5/N8y5YystF41TUVoy772k
 pmCXHniMmgmhH7HVwGujtc6mPe5y1VFJVaA08Pzb7KwSfdO9F/3Gk3DHpKW8/whi
 tCGluBqz0qlyhsnP9wDRJb6BzCBl2hVqu50DL+uSNsJOSIW60LLMJV4ANlDYdDM3
 s33S5NrM0DsRAjrtczPdvKPWwaVE4NB2bYX1I3yYGgflwzQYOjBmswM/UgymhlZk
 5dxtF9CX2p+Vre6UoLDKum1DJDCcWjHouJAAqZzxxEko56yWgTUSzTcK4GVOlsAc
 qX4gJbFpOzDlSdpDTG/fcnQlCnwc1jxCzsB8Wy2KJiBif3Sa3Wh1s00Cp7oGNQt+
 P/z2Fp1agl8u83bbvlIjZnsv0O2g5Ks4r5tBhXmqI36aiU26F/x39SUfp7/7OAUd
 CQBGBGXqpnOmUcE=
 =YWGX
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/ehabkost-gl/tags/x86-next-pull-request' into staging

x86 queue, 2021-06-18

Features:
* Add ratelimit for bus locks acquired in guest (Chenyi Qiang)

Documentation:
* SEV documentation updates (Tom Lendacky)
* Add a table showing x86-64 ABI compatibility levels (Daniel P. Berrangé)

Automated changes:
* Update Linux headers to 5.13-rc4 (Eduardo Habkost)

# gpg: Signature made Fri 18 Jun 2021 20:51:26 BST
# gpg:                using RSA key 5A322FD5ABC4D3DBACCFD1AA2807936F984DC5A6
# gpg:                issuer "ehabkost@redhat.com"
# gpg: Good signature from "Eduardo Habkost <ehabkost@redhat.com>" [full]
# Primary key fingerprint: 5A32 2FD5 ABC4 D3DB ACCF  D1AA 2807 936F 984D C5A6

* remotes/ehabkost-gl/tags/x86-next-pull-request:
  scripts: helper to generate x86_64 CPU ABI compat info
  docs: add a table showing x86-64 ABI compatibility levels
  docs/interop/firmware.json: Add SEV-ES support
  docs: Add SEV-ES documentation to amd-memory-encryption.txt
  doc: Fix some mistakes in the SEV documentation
  i386: Add ratelimit for bus locks acquired in guest
  Update Linux headers to 5.13-rc4

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-06-21 11:26:04 +01:00
John Snow
d08caefe66 scripts/qmp-shell: add redirection shim
qmp-shell has a new home, add a redirect for a little while as the dust
settles.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-43-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:07 -04:00
John Snow
6be7206efc scripts/qmp-shell: move to python/qemu/qmp/qmp_shell.py
The script will be unavailable for a commit or two, which will help
preserve development history attached to the new file. A forwarder will
be added shortly afterwards.

With qmp_shell in the python qemu.qmp package, now it is fully type
checked, linted, etc. via the Python CI. It will be quite a bit harder
to accidentally break it again in the future.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-41-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:07 -04:00
John Snow
e359c5a8b8 scripts/qmp-shell: add docstrings
Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-40-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:07 -04:00
John Snow
6a1105adba scripts/qmp-shell: make QMPShellError inherit QMPError
In preparation for moving qmp-shell into the qemu.qmp package, make
QMPShellError inherit from QMPError so that all custom errors in this
package all derive from QMPError.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-39-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:07 -04:00
John Snow
a64fe44d5a scripts/qmp-shell: remove double-underscores
They're not needed; single underscore is enough to express intent that
these methods are "internal". double underscore is used as a weak name
mangling, but that isn't beneficial for us here.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-38-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:07 -04:00
John Snow
7fc29896d2 scripts/qmp-shell: convert usage comment to docstring
The nice usage comment should be a docstring instead of a comment, so
that it's visible from other python tooling.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-37-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:07 -04:00
John Snow
26d3ce9e5e scripts/qmp-shell: Remove too-broad-exception
We are only anticipating QMPShellErrors here, for syntax we weren't able
to understand. Other errors, if any, should be allowed to percolate
upwards.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-36-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:07 -04:00
John Snow
c83055ef1d scripts/qmp-shell: Fix empty-transaction invocation
calling "transaction( )" is pointless, but valid. Rework the parser to
allow this kind of invocation. This helps clean up exception handling
later by removing accidental breakages of the parser that aren't
explicitly forbidden.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-35-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:07 -04:00
John Snow
74688377fe scripts/qmp-shell: remove TODO
We still want to revamp qmp-shell again, but there's much more to the
idea than the comment now intuits. Remove it.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-34-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:07 -04:00
John Snow
be19c6a712 scripts/qmp-shell: use logging to show warnings
A perfect candidate is non-fatal shell history messages.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-33-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:07 -04:00
John Snow
d1d14e5989 scripts/qmp-shell: Use context manager instead of atexit
We can invoke the shell history writing when we leave the QMPShell scope
instead of relying on atexit. Doing so may be preferable to avoid global
state being registered from within a class instead of from the
application logic directly.

Use QMP's context manager to hook this history saving at close time,
which gets invoked when we leave the context block.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-32-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:07 -04:00
John Snow
6e24a7edb8 scripts/qmp-shell: unprivatize 'pretty' property
Similar to verbose, there's no reason this needs to be hidden.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-30-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:07 -04:00
John Snow
b0b8ca17e5 scripts/qmp-shell: Accept SocketAddrT instead of string
Don't "extend" QEMUMonitorProtocol by changing the argument types. Move
the string parsing just outside of the class instead.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-29-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:07 -04:00
John Snow
1eab88723c scripts/qmp-shell: add mypy types
As per my usual, this patch is annotations only. Any changes with side
effects are done elsewhere.

Note: pylint does not understand the subscripts for Collection in Python 3.6,
so use the stronger Sequence type as a workaround.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-28-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:07 -04:00
John Snow
4157429582 scripts/qmp-shell: initialize completer early
Add an empty completer as a more type-safe placeholder instead of
'None'.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-26-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:07 -04:00
John Snow
db12abc208 scripts/qmp-shell: refactor QMPCompleter
list is a generic type, but we expect to use strings directly. We could
subclass list[str], but pylint does not presently understand that
invocation.

Change this class to envelop a list instead of *being* a list, for
simpler mypy typing.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-25-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:07 -04:00
John Snow
6faf2384ec scripts/qmp-shell: Fix "FuzzyJSON" parser
I'm not sure when this regressed (Or maybe if it was ever working right
to begin with?), but the Python AST requires you to change "Names" to
"Constants" in order to truly convert `false` to `False`.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-24-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:07 -04:00
John Snow
ad4eebee00 scripts/qmp-shell: move the REPL functionality into QMPShell
Instead of doing this in main, move it into the class itself. (This
makes it easier to put into the qemu.qmp package later by removing as
much as we can from the main() function.)

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-23-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:07 -04:00
John Snow
1215a1fbef scripts/qmp-shell: remove prompt argument from read_exec_command
It's only ever used by one caller, we can just absorb that logic.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-22-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:07 -04:00
John Snow
1caa505766 scripts/qmp-shell: move get_prompt() to prompt property
Small tidying; treat "prompt" like an immutable property instead of
function/method/routine.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-21-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:07 -04:00
John Snow
2ac3f3786e scripts/qmp-shell: Make verbose a public attribute
No real reason to hide this behind an underscore; make it part of the
initializer and make it a regular RW attribute.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-20-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:07 -04:00
John Snow
ad459132c0 scripts/qmp-shell: Add pretty attribute to HMP shell
It's less useful, but it makes the initialization methods LSP
consistent, which quiets a mypy complaint.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-19-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:07 -04:00
John Snow
17329be2fd scripts/qmp-shell: use argparse
Use argparse instead of an open-coded CLI parser, for consistency with
everything else.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-18-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:07 -04:00
John Snow
90bd8eb8dc scripts/qmp-shell: use isinstance() instead of type()
A bit more idiomatic, and quiets some linter warnings.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-17-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:07 -04:00
John Snow
73f699c903 scripts/qmp-shell: remove if-raise-else patterns
Shushes pylint. I don't always mind these patterns personally, but I'm
not as sure that I want to remove the warning from pylint's repertoire
entirely. Oh well.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-16-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:06 -04:00
John Snow
d962ec85ed scripts/qmp-shell: fix shell history exception handling
We want to remove exceptions that are too broad here; we only want to
catch IOErrors that get raised as a direct result of the open call.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-15-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:06 -04:00
John Snow
628b92dd67 scripts/qmp-shell: rename one and two-letter variables
A bit of churn and housekeeping for pylint, flake8 et al.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-14-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:06 -04:00
John Snow
2813dee053 scripts/qmp-shell: make QMPCompleter returns explicit
This function returns None when it doesn't find a match; do that
explicitly.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-13-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:06 -04:00
John Snow
50f6f1c308 scripts/qmp-shell: ignore visit_Name name
Not something I control, sorry, pylint.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-12-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:06 -04:00
John Snow
c6be2bf86e scripts/qmp-shell: use triple-double-quote docstring style
(2014 me had never written python before.)

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-11-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:06 -04:00
John Snow
31226369ab scripts/qmp-shell: declare verbose in __init__
Linters get angry when we don't define state at init time.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-10-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:06 -04:00
John Snow
5cb0233861 scripts/qmp-shell: Use python3-style super()
Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-9-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:06 -04:00
John Snow
c4a1447fc1 scripts/qmp-shell: use @classmethod where appropriate
Methods with no self-use should belong to the class.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-8-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:06 -04:00
John Snow
9669c8289c scripts/qmp-shell: remove shadowed variable from _print()
Don't use 'qmp' here, which shadows the qmp module.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-7-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:06 -04:00
John Snow
f2daa2d489 scripts/qmp-shell: fix connect method signature
It needs to match the parent's signature -- the negotiate parameter must
be optional.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-6-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:06 -04:00
John Snow
de14ba24f3 scripts/qmp-shell: fix exception handling
Fixes: 50d189c

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-5-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:06 -04:00
John Snow
70e5674018 scripts/qmp-shell: fix show_banner signature
The signatures need to match.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-4-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:06 -04:00
John Snow
169b43b367 scripts/qmp-shell: Apply flake8 rules
A lot of fiddling around to get us below 80 columns.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-3-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:06 -04:00
John Snow
badf462985 scripts/qmp-shell: apply isort rules
Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-2-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:06 -04:00
John Snow
88fb483fc5 scripts/qemu-ga-client: Add forwarder shim
Add a little forwarder shim until we are sure that everyone is
comfortable with how to use the tools in their new packaged location.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210604155532.1499282-12-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:06 -04:00
John Snow
1e129afc31 scripts/qemu-ga-client: move to python/qemu/qmp/qemu_ga_client.py
The script itself will be unavailable for a few commits before being
restored, with no way to run it right after this commit. This helps move
git history into the new file. To prevent linter regressions, though, we
do need to immediately touch up the filename to remove dashes (to make
the module importable), and remove the executable bit.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210604155532.1499282-10-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:06 -04:00
John Snow
ca683d4a2f scripts/qemu-ga-client: add mypy type hints
This script is in slightly rough shape, but it still works. A lot of
care went into its initial development. In good faith, I'm updating it
to the latest Python coding standards. If there is in interest in this
script, though, I'll be asking for a contributor to take care of it
further.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210604155532.1499282-9-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:06 -04:00
John Snow
f85d3252ef scripts/qemu-ga-client: apply (most) pylint rules
I'm only doing a very quick best-effort to preserve this script, to help
keep it from breaking further. I think there are pending ideas swirling
on the right way to implement better SDKs and better clients, and this
script might be a handy reference for those discussions. It presents
some interesting design problems, like static type safety when using a
dynamic RPC mechanism.

I believe it's worth preserving the effort and care that went into
making this script by updating it to work with our current
infrastructure. However, I am disabling the requirement for docstrings
in this file.

If you would like to help improve this script, please add docstrings
alongside any refactors or rejuvenations you might apply at that time.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210604155532.1499282-7-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:06 -04:00
John Snow
aff103b531 scripts/qemu-ga-client: add module docstring
Turn that nice usage comment into a docstring.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210604155532.1499282-6-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:06 -04:00