Commit Graph

2304 Commits

Author SHA1 Message Date
Markus Armbruster
34f7b25e57 qapi: Tweak error messages for unknown / conflicting 'if' keys
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210831123809.1107782-13-armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
2021-09-03 17:09:10 +02:00
Markus Armbruster
6dcf03719a qapi: Tweak error messages for missing / conflicting meta-type
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210831123809.1107782-12-armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
2021-09-03 17:09:10 +02:00
Markus Armbruster
555dd1aaa6 qapi: Use re.fullmatch() where appropriate
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210831123809.1107782-10-armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
2021-09-03 17:09:10 +02:00
Markus Armbruster
d0830ee443 qapi: Use "not COND" instead of "!COND" for generated documentation
Generated documentation uses operators "and", "or", and "!".  Change
the latter to "not".

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210831123809.1107782-9-armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
2021-09-03 17:09:10 +02:00
Markus Armbruster
a7987799d1 qapi: Avoid redundant parens in code generated for conditionals
Commit 6cc2e4817f "qapi: introduce QAPISchemaIfCond.cgen()" caused a
minor regression: redundant parenthesis.  Subsequent commits
eliminated of many of them, but not all.  Get rid of the rest now.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210831123809.1107782-8-armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
2021-09-03 17:09:10 +02:00
Markus Armbruster
ccea6a8637 qapi: Factor common recursion out of cgen_ifcond(), docgen_ifcond()
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210831123809.1107782-7-armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
2021-09-03 17:09:10 +02:00
Markus Armbruster
82ca72c023 qapi: Fix C code generation for 'if'
When commit 5d83b9a130 "qapi: replace if condition list with dict
{'all': [...]}" made cgen_ifcond() and docgen_ifcond() recursive, it
messed up parenthesises in the former, and got them right in the
latter, as the previous commit demonstrates.

To fix, adopt the latter's working code for the former.  This
generates the correct code from the previous commit's commit message.

Fixes: 5d83b9a130
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210831123809.1107782-6-armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
2021-09-03 17:09:10 +02:00
Markus Armbruster
e46c930cdd qapi: Simplify how QAPISchemaIfCond represents "no condition"
None works fine, there is no need to replace it by {} in .__init__().

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210831123809.1107782-3-armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
2021-09-03 17:09:10 +02:00
Markus Armbruster
1889e57a71 qapi: Simplify QAPISchemaIfCond's interface for generating C
QAPISchemaIfCond.cgen() is only ever used like

    gen_if(ifcond.cgen())

and

    gen_endif(ifcond.cgen())

Simplify to

    ifcond.gen_if()

and

    ifcond.gen_endif()

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210831123809.1107782-2-armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
[Import statements tidied up with isort]
2021-09-03 17:06:40 +02:00
Alexander Bulekov
dfc86c0f25 fuzz: add an instrumentation filter
By default, -fsanitize=fuzzer instruments all code with coverage
information. However, this means that libfuzzer will track coverage over
hundreds of source files that are unrelated to virtual-devices. This
means that libfuzzer will optimize inputs for coverage observed in timer
code, memory APIs etc. This slows down the fuzzer and stores many inputs
that are not relevant to the actual virtual-devices.

With this change, clang versions that support the
"-fsanitize-coverage-allowlist" will only instrument a subset of the
compiled code, that is directly related to virtual-devices.

Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
Reviewed-by: Darren Kenny <darren.kenny@oracle.com>
2021-09-01 07:33:13 -04:00
Marc-André Lureau
8a9f1e1d9c qapi: make 'if' condition strings simple identifiers
Change the 'if' condition strings to be C-agnostic. It will accept
'[A-Z][A-Z0-9_]*' identifiers. This allows to express configuration
conditions in other languages (Rust or Python for ex) or other more
suitable forms.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Tested-by: John Snow <jsnow@redhat.com>
Message-Id: <20210804083105.97531-11-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Rebased with semantic conflict in redefined-event.json]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-08-26 13:53:56 +02:00
Marc-André Lureau
2b7d214536 qapi: add 'not' condition operation
For the sake of completeness, introduce the 'not' condition.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20210804083105.97531-10-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Long line broken in tests/qapi-schema/qapi-schema-test.json]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-08-26 13:53:56 +02:00
Marc-André Lureau
3ad64edfad qapi: add 'any' condition
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20210804083105.97531-8-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-08-26 13:53:56 +02:00
Marc-André Lureau
5d83b9a130 qapi: replace if condition list with dict {'all': [...]}
Replace the simple list sugar form with a recursive structure that will
accept other operators in the following commits (all, any or not).

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20210804083105.97531-7-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Accidental code motion undone.  Degenerate :forms: comment dropped.
Helper _check_if() moved.  Error messages tweaked.  ui.json updated.
Accidental changes to qapi-schema-test.json dropped.]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-08-26 13:53:56 +02:00
Marc-André Lureau
d806f89f87 qapidoc: introduce QAPISchemaIfCond.docgen()
Instead of building the condition documentation from a list of string,
use the result generated from QAPISchemaIfCond.docgen().

This changes the generated documentation from:
- COND1, COND2... (where COND1, COND2 are Literal nodes, and ',' is Text)
to:
- COND1 and COND2 (the whole string as a Literal node)

This will allow us to generate more complex conditions in the following
patches, such as "(COND1 and COND2) or COND3".

Adding back the differentiated formatting is left to the wish list.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20210804083105.97531-6-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[TODO comment added]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-08-26 13:53:56 +02:00
Marc-André Lureau
6cc2e4817f qapi: introduce QAPISchemaIfCond.cgen()
Instead of building prepocessor conditions from a list of string, use
the result generated from QAPISchemaIfCond.cgen() and hide the
implementation details.

Note: this patch introduces a minor regression, generating a redundant
pair of parenthesis. This is mostly fixed in a later patch in this
series ("qapi: replace if condition list with dict [..]")

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20210804083105.97531-5-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Commit message tweaked]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-08-26 13:53:56 +02:00
Marc-André Lureau
33aa3267ba qapi: add QAPISchemaIfCond.is_present()
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210804083105.97531-4-marcandre.lureau@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-08-26 13:53:56 +02:00
Marc-André Lureau
f17539c80d qapi: wrap Sequence[str] in an object
Mechanical change, except for a new assertion in
QAPISchemaEntity.ifcond().

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20210804083105.97531-3-marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Rebased with obvious conflicts, commit message adjusted]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-08-26 13:53:56 +02:00
Markus Armbruster
b32abbb2f5 qapi: Fix crash on redefinition with a different condition
QAPISchema._make_implicit_object_type() asserts that when an implicit
object type is used multiple times, @ifcond is the same for all uses.
It will be for legitimate uses, i.e. simple union branch wrapper
types.  A comment explains this.

The assertion fails when a command or event is redefined with a
different condition.  The redefinition is an error, but it's flagged
only later.

Fixing the assertion would complicate matters further.  Not
worthwhile, drop it instead.  We really need to get rid of simple
unions.

Tweak test case redefined-event to cover redefinition with a different
condition.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210806120510.2367124-1-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2021-08-26 13:53:56 +02:00
Alexander Bulekov
3973e7ae63 fuzz: avoid building twice, when running on gitlab
On oss-fuzz, we build twice, to put together a build that is portable to
the runner containers. On gitlab ci, this is wasteful and contributes to
timeouts on the build-oss-fuzz job. Avoid building twice on gitlab, at
the remote cost of potentially missing some cases that break oss-fuzz
builds.

Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
Reviewed-by: Darren Kenny <darren.kenny@oracle.com>
Message-Id: <20210809111621.54454-1-alxndr@bu.edu>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-08-11 13:40:01 +02:00
Paolo Bonzini
e17bdaab2b coverity-model: write models fully for non-array allocation functions
Coverity seems to have issues figuring out the properties of g_malloc0
and other non *_n functions.  While this was "fixed" by removing the
custom second argument to __coverity_mark_as_afm_allocated__, inline
the code from the array-based allocation functions to avoid future
issues.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-07-30 12:04:01 +02:00
Paolo Bonzini
0da41187df coverity-model: constrain g_malloc/g_malloc0/g_realloc as never returning NULL
g_malloc/g_malloc0/g_realloc only return NULL if the size is 0; we do not need
to cover that in the model, and so far have expected __coverity_alloc__
to model a non-NULL return value.  But that apparently does not work
anymore, so add some extra conditionals that invoke __coverity_panic__
for NULL pointers.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-07-30 12:04:01 +02:00
Paolo Bonzini
05ad6857a5 coverity-model: clean up the models for array allocation functions
sz is only used in one place, so replace it with nmemb * size in
that one place.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-07-30 12:04:01 +02:00
Paolo Bonzini
96915d638c coverity-model: remove model for more allocation functions
These models are not needed anymore now that Coverity does not check
anymore that the result is used with "g_free".  Coverity understands
GCC attributes and uses them to detect leaks.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-07-30 12:04:01 +02:00
Paolo Bonzini
243a545bff coverity-model: make g_free a synonym of free
Recently, Coverity has started complaining about using g_free() to free
memory areas allocated by GLib functions not included in model.c,
such as g_strfreev.  This unfortunately goes against the GLib
documentation, which suggests that g_malloc() should be matched
with g_free() and plain malloc() with free(); since GLib 2.46 however
g_malloc() is hardcoded to always use the system malloc implementation,
and g_free is just "free" plus a tracepoint.  Therefore, this
should not cause any problem in practice.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-07-30 12:04:01 +02:00
Paolo Bonzini
d4b3d152ee coverity-model: update address_space_read/write models
Use void * for consistency with the actual function; provide a model
for MemoryRegionCache functions and for address_space_rw.  These
let Coverity understand the bounds of the data that various functions
read and write even at very high levels of inlining (e.g. pci_dma_read).

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-07-30 12:04:01 +02:00
Peter Maydell
9ad4c7c9b6 QAPI patches patches for 2021-07-15
-----BEGIN PGP SIGNATURE-----
 
 iQJGBAABCAAwFiEENUvIs9frKmtoZ05fOHC0AOuRhlMFAmDwKlYSHGFybWJydUBy
 ZWRoYXQuY29tAAoJEDhwtADrkYZTR2UP+wVKKbY09lvPlpb25DJe073EQspRqBqg
 hG8R4D82Xics3XMfAJ/QcCLmtEbYKT02BCyY069rP6fnD8ZS5DfUeiIXrXBajN5K
 eVFuIO/mV7qNvx8mWyvZ7X5BWoolEDxkx+TjFfu6ZgSDMFPCjKE729/IXpAPhHll
 lYW2j2RLHUJCTRq+IRv36E11LFKI1zDewA8q5NHARxIAyMBLT9UFKLpnw1Fe5V9C
 w/2aB1Uh78oAfB6BtazvTHVWT75pca/W/jzyMcdXMVaPCJrsODEvz6Gn50gKYG+L
 J7/m6ivjCjitLGk9H+h8TaC/Q/wIE36gMjd2XT4MhgxF14/bMZpX1xlMO8De9DMn
 XhONKRGtXYV9r2uLhc0C8kID6TEidIDBNXDrtX1hDL172AfJ+Q7fDTqqCSUk5C2j
 dbNQISOlEWTJ1moOiRPjozGrcap5/Z/CViUrrrWXvAZoHScEoHgX+3QyRYD0MfeC
 0Q55kdx0Pu5W4qwg/EEBSzMUSy82VzJzgQTBmfmTxeEOqfUpfKu53ZNBqOO+1ZJd
 FsPzqnekNDcDrXtsg4TOuAnvShF8NW9nVnx3JeNUilcrjN0lIsnf7wrWUQQD66wA
 ZyboB9r/b0WdTQ9f+Zpi/i1ufEu9CSBseRrLac2noOOOrHLiURR8TYr3p6i6g0Fk
 jY9uCC/WnPto
 =dZGh
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/armbru/tags/pull-qapi-2021-07-15' into staging

QAPI patches patches for 2021-07-15

# gpg: Signature made Thu 15 Jul 2021 13:30:14 BST
# gpg:                using RSA key 354BC8B3D7EB2A6B68674E5F3870B400EB918653
# gpg:                issuer "armbru@redhat.com"
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>" [full]
# gpg:                 aka "Markus Armbruster <armbru@pond.sub.org>" [full]
# Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867  4E5F 3870 B400 EB91 8653

* remotes/armbru/tags/pull-qapi-2021-07-15:
  qapi: Fix crash on missing enum member name

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-07-16 10:56:03 +01:00
Markus Armbruster
a0c7b99bf7 qapi: Fix crash on missing enum member name
New test case enum-dict-no-name.json crashes:

    $ python3 scripts/qapi-gen.py tests/qapi-schema/enum-dict-no-name.json
    Traceback (most recent call last):
    [...]
      File "/work/armbru/qemu/scripts/qapi/expr.py", line 458, in check_enum
	member_name = member['name']
    KeyError: 'name'

Root cause: we try to retrieve member 'name' before we check for
missing members.  With that fixed, we get the expected error "'data'
member misses key 'name'".

Fixes: 0825f62c84
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210616072121.626431-1-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
2021-07-15 12:56:41 +02:00
Daniel P. Berrangé
51f5c849c1 hw/usb/ccid: remove references to NSS
The NSS package was previously pre-requisite for building CCID related
features, however, this became obsolete when the libcacard library was
spun off to a separate project:

    commit 7b02f5447c
    Author: Marc-André Lureau <marcandre.lureau@redhat.com>
    Date:   Sun Aug 30 11:48:40 2015 +0200

        libcacard: use the standalone project

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210623142245.307776-2-berrange@redhat.com>
Message-Id: <20210709143005.1554-10-alex.bennee@linaro.org>
2021-07-14 14:33:53 +01:00
Cleber Rosa
40de78c284 Jobs based on custom runners: docs and gitlab-runner setup playbook
To have the jobs dispatched to custom runners, gitlab-runner must
be installed, active as a service and properly configured.  The
variables file and playbook introduced here should help with those
steps.

The playbook introduced here covers the Linux distributions and
has been primarily tested on OS/machines that the QEMU project
has available to act as runners, namely:

 * Ubuntu 20.04 on aarch64
 * Ubuntu 18.04 on s390x

But, it should work on all other Linux distributions.  Earlier
versions were tested on FreeBSD too, so chances of success are
high.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Willian Rampazzo <willianr@redhat.com>
Tested-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210630012619.115262-4-crosa@redhat.com>
Message-Id: <20210709143005.1554-4-alex.bennee@linaro.org>
2021-07-14 14:31:48 +01:00
Cleber Rosa
159c5d177b Jobs based on custom runners: build environment docs and playbook
To run basic jobs on custom runners, the environment needs to be
properly set up.  The most common requirement is having the right
packages installed.

The playbook introduced here covers the QEMU's project s390x and
aarch64 machines.  At the time this is being proposed, those machines
have already had this playbook applied to them.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210630012619.115262-3-crosa@redhat.com>
Message-Id: <20210709143005.1554-3-alex.bennee@linaro.org>
2021-07-14 14:31:48 +01:00
Peter Maydell
2a54fc454c Linux-user pull request 20210713
Update headers to linux v5.13
 cleanup errno target headers
 Fix race condition on fd translation table
 -----BEGIN PGP SIGNATURE-----
 
 iQJGBAABCAAwFiEEzS913cjjpNwuT1Fz8ww4vT8vvjwFAmDtmAUSHGxhdXJlbnRA
 dml2aWVyLmV1AAoJEPMMOL0/L748BFEP/jl0taLq3vqwUuAmsirJadf4K5+4iFiO
 /FhhAw5kzNasrlO2WrvTazDpJsAotuPeI2kQpnuEruDPOJulf21SUWRVeOUmzYBQ
 V/mKKkazY7FnmI8jhO8FhFrNGrOQWEGtU5adNX0bJwArbhfSWMfBTCfaoWePUmVO
 26X2gNchHwvymoLyqKG7fA1AahSQEqgE6wG9L3n6eXxPy8xBEj0WDQoISakTKJJ4
 J7Z2QchsYC51A2kpkA77TF+W2yLRrOwO8XNLQNOjSuPyvi6v0nsbe6mTSxVOmEcN
 3PYezsAWHg0rptIWuGwnVN2+juQt2pp8T3sADRWAMrgnkCrBkiwgWmBNzWuPXLX6
 xoXd1//lZcmGRwmJzvvWs1+1IZsVcRyWq9nAg/+CTbrTe+TC2q5a90jkEPup9bkx
 up/xKJJwFyl4j/eeUfmWBruAjX0PCCtgpcVTMafWSTD9maooHVTOJmy6ZETgCCXC
 KvRoWNK1Cx94KblHrGU/mWxBT9OhqPJxlvPswKDYidS0bMOoVt8eqFI7GRKxX7qK
 wldbNiqLBB0ZxHLZPP3TgE96qVr+PSP9UfShWC+X5Cl5o4/m0tvq+V0dq3PBsz4O
 Zq7NQ1cU7z7rBcyqDFiREVzucScCY3rch3E4xGJASulrqm3HFJHJXuaBcFUMJhUt
 61puHc6tOyBp
 =Stii
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/vivier2/tags/linux-user-for-6.1-pull-request' into staging

Linux-user pull request 20210713

Update headers to linux v5.13
cleanup errno target headers
Fix race condition on fd translation table

# gpg: Signature made Tue 13 Jul 2021 14:41:25 BST
# gpg:                using RSA key CD2F75DDC8E3A4DC2E4F5173F30C38BD3F2FBE3C
# gpg:                issuer "laurent@vivier.eu"
# gpg: Good signature from "Laurent Vivier <lvivier@redhat.com>" [full]
# gpg:                 aka "Laurent Vivier <laurent@vivier.eu>" [full]
# gpg:                 aka "Laurent Vivier (Red Hat) <lvivier@redhat.com>" [full]
# Primary key fingerprint: CD2F 75DD C8E3 A4DC 2E4F  5173 F30C 38BD 3F2F BE3C

* remotes/vivier2/tags/linux-user-for-6.1-pull-request:
  linux-user: update syscall.tbl to Linux v5.13
  linux-user, mips: update syscall-args-o32.c.inc to Linux v5.13
  linux-user: update syscall_nr.h to Linux v5.13
  fd-trans: Fix race condition on reallocation of the translation table.
  linux-user/syscall: Remove ERRNO_TABLE_SIZE check
  linux-user: Simplify host <-> target errno conversion using macros
  linux-user/mips: Move errno definitions to 'target_errno_defs.h'
  linux-user/hppa: Move errno definitions to 'target_errno_defs.h'
  linux-user/alpha: Move errno definitions to 'target_errno_defs.h'
  linux-user: Extract target errno to 'target_errno_defs.h'
  linux-user/sparc: Rename target_errno.h -> target_errno_defs.h
  linux-user/syscall: Fix RF-kill errno (typo in ERFKILL)

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-07-13 21:58:57 +01:00
Laurent Vivier
e2dcdcea23 linux-user, mips: update syscall-args-o32.c.inc to Linux v5.13
Updated running scripts/update-mips-syscall-args.sh

scripts/update-mips-syscall-args.sh has been updated to reflect
file directory changes in strace repository.

Signed-off-by: Laurent Vivier <laurent@vivier.eu>
Message-Id: <20210708215756.268805-3-laurent@vivier.eu>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
2021-07-13 13:59:59 +02:00
Gerd Hoffmann
2adf216491 qemu-trace-stap: changing SYSTEMTAP_TAPSET considered harmful.
Setting SYSTEMTAP_TAPSET to some value other than
/usr/share/systemtap/tapsets results in systemtap not finding the
standard tapset library any more, which in turn breaks tracing because
pid() and other standard systemtap functions are not available any more.

So using SYSTEMTAP_TAPSET to point systemtap to the qemu probes will
only work for the prefix=/usr installs because both qemu and system
tapsets in the same directory then.  All other prefixes are broken.

Fix that by using the "-I $tapsetdir" command line switch instead.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-id: 20210601132414.432430-2-kraxel@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2021-07-12 17:37:06 +01:00
Akihiko Odaki
411ad8dd80 meson: Use input/output for entitlements target
input/output parameters respect dependencies.

Signed-off-by: Akihiko Odaki <akihiko.odaki@gmail.com>
Message-Id: <20210709012533.58262-1-akihiko.odaki@gmail.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-07-09 18:21:34 +02:00
Jose R. Ziviani
af19eecf84 modules: check if all dependencies can be satisfied
Verifies if all dependencies are correctly listed in the modinfo.c too
and stop the builds if they're not.

Signed-off-by: Jose R. Ziviani <jziviani@suse.de>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Jose R. Ziviani <jziviani@suse.de>
Message-Id: <20210624103836.2382472-5-kraxel@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-07-09 18:20:27 +02:00
Gerd Hoffmann
5ebbfecc3e modules: generate modinfo.c
Add script to generate C source with a small
database containing the module meta-data.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Jose R. Ziviani <jziviani@suse.de>
Message-Id: <20210624103836.2382472-4-kraxel@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-07-09 18:20:27 +02:00
Gerd Hoffmann
f5723ab665 modules: collect module meta-data
Add script to collect the module meta-data from the source code,
store the results in *.modinfo files.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Jose R. Ziviani <jziviani@suse.de>
Message-Id: <20210624103836.2382472-3-kraxel@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-07-09 18:20:27 +02:00
John Arbuckle
3983a767e3 Set icon for QEMU binary on Mac OS
Before switching the build system over to Meson, an icon was
added to the QEMU binary on Mac OS. This patch adds back that
feature; it piggybacks on the existing scripts/entitlement.sh,
which already does in-place changes to the executable on Darwin.

Signed-off-by: John Arbuckle <programmingkidx@gmail.com>
Message-Id: <20210705195328.36442-1-programmingkidx@gmail.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-07-06 09:35:08 +02:00
Paolo Bonzini
dd52af17ec coverity-scan: switch to vpath build
This is the patch that has been running on the coverity cronjob
for a few weeks now.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-07-06 07:54:50 +02:00
Philippe Mathieu-Daudé
dff5f68224 coverity-scan: Remove lm32 / unicore32 targets
lm32 has been removed in commit 9d49bcf699 ("Drop the deprecated
lm32 target"), and unicore32 in 4369223902 ("Drop the deprecated
unicore32 target").

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20210619091342.3660495-2-f4bug@amsat.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-07-06 07:49:41 +02:00
Alex Bennée
66cf70149a scripts/checkpatch: roll diff tweaking into checkpatch itself
Rather than relying on external tweaks lets just do it inside
checkpatch's direct commitish handling which is QEMU specific code
anyway.

Suggested-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20210623102749.25686-4-alex.bennee@linaro.org>
2021-06-25 10:08:33 +01:00
Peter Maydell
0add99ea3e Python Pull request
Moves QMP-related tools not used for build or automatic testing from
 scripts/ to python/qemu/qmp/ where they will be protected from bitrot by
 the check-python-* CI jobs.
 
 stub forwarders are left in the old locations for now.
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEE+ber27ys35W+dsvQfe+BBqr8OQ4FAmDNJhAACgkQfe+BBqr8
 OQ5uZg/8CxwwDVskXya2BWa0WHrqQGmods2XMVupp9qI5HQAPE6AzlypUzvoA+6o
 jr5uenG/CGSNZgWUfGN3UxCw85knEGLekU9u3Rx/N+CmlHvNPZz8UUg0oOy9+QIV
 owFYXPq4Uc9siUjvFhFNmC6ZIUGf2y5vXOECqjTZL4SM9Qq0F3mdrFUdT78AR2jw
 HTUipCiaZQLFArlxqIFvIv/wR4i+zh9cXs4A31THjoLBz6kkgX89D+yb9cUDwcjp
 fDBYWuAN7dJzt1jqxuD1yUj3rqwKj0MZa6QXd0pKBQt3fHJCE4geY+mGzmGd9nTx
 ZZ8RI64CEjH946eNpRFCl+DPqLNE88pTbRpUHXy3YOvNj1ZWGjPtA6aDqcpQdySL
 UcQxc5K0zKROh9vUIMmmXW4oGlv9xgilO0BnMq5oMTvolaflaF1JdPpmjd+0pHJ2
 u9/I/+Bc5tJEmSYPu4ASh0UVllAVSSjnopLcjLJv9g8yApDxuodfyNafdctbsVmL
 YjiSedsPI2Hawm7O54cE1D6fkjopxwFaEuv1CyTqiDcULk6qWSDj3KtnrDOp0BNV
 qpN6ReK75UHBzhkdgoKWLm3WuZZk9s2r58e9GAPb7UxR3EiaPOBLOYrKawooqiI7
 +LDTqXNbVQJBpe1IgkygZiy2GuQ8unQCsxqhjy7nn6M01/SrPEc=
 =YTLz
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/jsnow-gitlab/tags/python-pull-request' into staging

Python Pull request

Moves QMP-related tools not used for build or automatic testing from
scripts/ to python/qemu/qmp/ where they will be protected from bitrot by
the check-python-* CI jobs.

stub forwarders are left in the old locations for now.

# gpg: Signature made Sat 19 Jun 2021 00:02:40 BST
# gpg:                using RSA key F9B7ABDBBCACDF95BE76CBD07DEF8106AAFC390E
# gpg: Good signature from "John Snow (John Huston) <jsnow@redhat.com>" [full]
# Primary key fingerprint: FAEB 9711 A12C F475 812F  18F2 88A9 064D 1835 61EB
#      Subkey fingerprint: F9B7 ABDB BCAC DF95 BE76  CBD0 7DEF 8106 AAFC 390E

* remotes/jsnow-gitlab/tags/python-pull-request: (72 commits)
  scripts/qmp-shell: add redirection shim
  python: add qmp-shell entry point
  scripts/qmp-shell: move to python/qemu/qmp/qmp_shell.py
  scripts/qmp-shell: add docstrings
  scripts/qmp-shell: make QMPShellError inherit QMPError
  scripts/qmp-shell: remove double-underscores
  scripts/qmp-shell: convert usage comment to docstring
  scripts/qmp-shell: Remove too-broad-exception
  scripts/qmp-shell: Fix empty-transaction invocation
  scripts/qmp-shell: remove TODO
  scripts/qmp-shell: use logging to show warnings
  scripts/qmp-shell: Use context manager instead of atexit
  python/qmp: return generic type from context manager
  scripts/qmp-shell: unprivatize 'pretty' property
  scripts/qmp-shell: Accept SocketAddrT instead of string
  scripts/qmp-shell: add mypy types
  python/qmp: add QMPObject type alias
  scripts/qmp-shell: initialize completer early
  scripts/qmp-shell: refactor QMPCompleter
  scripts/qmp-shell: Fix "FuzzyJSON" parser
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-06-21 16:11:33 +01:00
Peter Maydell
53f306f316 x86 queue, 2021-06-18
Features:
 * Add ratelimit for bus locks acquired in guest (Chenyi Qiang)
 
 Documentation:
 * SEV documentation updates (Tom Lendacky)
 * Add a table showing x86-64 ABI compatibility levels (Daniel P. Berrangé)
 
 Automated changes:
 * Update Linux headers to 5.13-rc4 (Eduardo Habkost)
 -----BEGIN PGP SIGNATURE-----
 
 iQJIBAABCAAyFiEEWjIv1avE09usz9GqKAeTb5hNxaYFAmDM+T4UHGVoYWJrb3N0
 QHJlZGhhdC5jb20ACgkQKAeTb5hNxabUrQ/+PtiJjd1cW9nhA0kWu8dVGq3xXJb4
 Nbma86tRPKBauTeQCLccXEvUjLqgFejeQlArhq4QKErLisXu4TDuQ+GeAfdR7h5P
 MTMSo0C665cT2/NbrwQizSPQdrNEgZAYRaDRafZLQTJ1TStzWDB1Vg79rzpWPcn0
 76XjIfSdGZUa4B1OvjNvUFq/SXf+0hW75soCwRhDNh5tfzfyct0XCSRF/wTXqyR/
 7yxDtfTzUAvT+6l3qb8ky+wqUTIY58BgjbdIGhyAUr5/N8y5YystF41TUVoy772k
 pmCXHniMmgmhH7HVwGujtc6mPe5y1VFJVaA08Pzb7KwSfdO9F/3Gk3DHpKW8/whi
 tCGluBqz0qlyhsnP9wDRJb6BzCBl2hVqu50DL+uSNsJOSIW60LLMJV4ANlDYdDM3
 s33S5NrM0DsRAjrtczPdvKPWwaVE4NB2bYX1I3yYGgflwzQYOjBmswM/UgymhlZk
 5dxtF9CX2p+Vre6UoLDKum1DJDCcWjHouJAAqZzxxEko56yWgTUSzTcK4GVOlsAc
 qX4gJbFpOzDlSdpDTG/fcnQlCnwc1jxCzsB8Wy2KJiBif3Sa3Wh1s00Cp7oGNQt+
 P/z2Fp1agl8u83bbvlIjZnsv0O2g5Ks4r5tBhXmqI36aiU26F/x39SUfp7/7OAUd
 CQBGBGXqpnOmUcE=
 =YWGX
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/ehabkost-gl/tags/x86-next-pull-request' into staging

x86 queue, 2021-06-18

Features:
* Add ratelimit for bus locks acquired in guest (Chenyi Qiang)

Documentation:
* SEV documentation updates (Tom Lendacky)
* Add a table showing x86-64 ABI compatibility levels (Daniel P. Berrangé)

Automated changes:
* Update Linux headers to 5.13-rc4 (Eduardo Habkost)

# gpg: Signature made Fri 18 Jun 2021 20:51:26 BST
# gpg:                using RSA key 5A322FD5ABC4D3DBACCFD1AA2807936F984DC5A6
# gpg:                issuer "ehabkost@redhat.com"
# gpg: Good signature from "Eduardo Habkost <ehabkost@redhat.com>" [full]
# Primary key fingerprint: 5A32 2FD5 ABC4 D3DB ACCF  D1AA 2807 936F 984D C5A6

* remotes/ehabkost-gl/tags/x86-next-pull-request:
  scripts: helper to generate x86_64 CPU ABI compat info
  docs: add a table showing x86-64 ABI compatibility levels
  docs/interop/firmware.json: Add SEV-ES support
  docs: Add SEV-ES documentation to amd-memory-encryption.txt
  doc: Fix some mistakes in the SEV documentation
  i386: Add ratelimit for bus locks acquired in guest
  Update Linux headers to 5.13-rc4

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-06-21 11:26:04 +01:00
John Snow
d08caefe66 scripts/qmp-shell: add redirection shim
qmp-shell has a new home, add a redirect for a little while as the dust
settles.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-43-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:07 -04:00
John Snow
6be7206efc scripts/qmp-shell: move to python/qemu/qmp/qmp_shell.py
The script will be unavailable for a commit or two, which will help
preserve development history attached to the new file. A forwarder will
be added shortly afterwards.

With qmp_shell in the python qemu.qmp package, now it is fully type
checked, linted, etc. via the Python CI. It will be quite a bit harder
to accidentally break it again in the future.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-41-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:07 -04:00
John Snow
e359c5a8b8 scripts/qmp-shell: add docstrings
Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-40-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:07 -04:00
John Snow
6a1105adba scripts/qmp-shell: make QMPShellError inherit QMPError
In preparation for moving qmp-shell into the qemu.qmp package, make
QMPShellError inherit from QMPError so that all custom errors in this
package all derive from QMPError.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-39-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:07 -04:00
John Snow
a64fe44d5a scripts/qmp-shell: remove double-underscores
They're not needed; single underscore is enough to express intent that
these methods are "internal". double underscore is used as a weak name
mangling, but that isn't beneficial for us here.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-38-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:07 -04:00
John Snow
7fc29896d2 scripts/qmp-shell: convert usage comment to docstring
The nice usage comment should be a docstring instead of a comment, so
that it's visible from other python tooling.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-37-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:07 -04:00
John Snow
26d3ce9e5e scripts/qmp-shell: Remove too-broad-exception
We are only anticipating QMPShellErrors here, for syntax we weren't able
to understand. Other errors, if any, should be allowed to percolate
upwards.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-36-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:07 -04:00
John Snow
c83055ef1d scripts/qmp-shell: Fix empty-transaction invocation
calling "transaction( )" is pointless, but valid. Rework the parser to
allow this kind of invocation. This helps clean up exception handling
later by removing accidental breakages of the parser that aren't
explicitly forbidden.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-35-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:07 -04:00
John Snow
74688377fe scripts/qmp-shell: remove TODO
We still want to revamp qmp-shell again, but there's much more to the
idea than the comment now intuits. Remove it.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-34-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:07 -04:00
John Snow
be19c6a712 scripts/qmp-shell: use logging to show warnings
A perfect candidate is non-fatal shell history messages.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-33-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:07 -04:00
John Snow
d1d14e5989 scripts/qmp-shell: Use context manager instead of atexit
We can invoke the shell history writing when we leave the QMPShell scope
instead of relying on atexit. Doing so may be preferable to avoid global
state being registered from within a class instead of from the
application logic directly.

Use QMP's context manager to hook this history saving at close time,
which gets invoked when we leave the context block.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-32-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:07 -04:00
John Snow
6e24a7edb8 scripts/qmp-shell: unprivatize 'pretty' property
Similar to verbose, there's no reason this needs to be hidden.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-30-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:07 -04:00
John Snow
b0b8ca17e5 scripts/qmp-shell: Accept SocketAddrT instead of string
Don't "extend" QEMUMonitorProtocol by changing the argument types. Move
the string parsing just outside of the class instead.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-29-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:07 -04:00
John Snow
1eab88723c scripts/qmp-shell: add mypy types
As per my usual, this patch is annotations only. Any changes with side
effects are done elsewhere.

Note: pylint does not understand the subscripts for Collection in Python 3.6,
so use the stronger Sequence type as a workaround.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-28-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:07 -04:00
John Snow
4157429582 scripts/qmp-shell: initialize completer early
Add an empty completer as a more type-safe placeholder instead of
'None'.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-26-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:07 -04:00
John Snow
db12abc208 scripts/qmp-shell: refactor QMPCompleter
list is a generic type, but we expect to use strings directly. We could
subclass list[str], but pylint does not presently understand that
invocation.

Change this class to envelop a list instead of *being* a list, for
simpler mypy typing.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-25-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:07 -04:00
John Snow
6faf2384ec scripts/qmp-shell: Fix "FuzzyJSON" parser
I'm not sure when this regressed (Or maybe if it was ever working right
to begin with?), but the Python AST requires you to change "Names" to
"Constants" in order to truly convert `false` to `False`.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-24-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:07 -04:00
John Snow
ad4eebee00 scripts/qmp-shell: move the REPL functionality into QMPShell
Instead of doing this in main, move it into the class itself. (This
makes it easier to put into the qemu.qmp package later by removing as
much as we can from the main() function.)

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-23-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:07 -04:00
John Snow
1215a1fbef scripts/qmp-shell: remove prompt argument from read_exec_command
It's only ever used by one caller, we can just absorb that logic.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-22-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:07 -04:00
John Snow
1caa505766 scripts/qmp-shell: move get_prompt() to prompt property
Small tidying; treat "prompt" like an immutable property instead of
function/method/routine.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-21-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:07 -04:00
John Snow
2ac3f3786e scripts/qmp-shell: Make verbose a public attribute
No real reason to hide this behind an underscore; make it part of the
initializer and make it a regular RW attribute.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-20-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:07 -04:00
John Snow
ad459132c0 scripts/qmp-shell: Add pretty attribute to HMP shell
It's less useful, but it makes the initialization methods LSP
consistent, which quiets a mypy complaint.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-19-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:07 -04:00
John Snow
17329be2fd scripts/qmp-shell: use argparse
Use argparse instead of an open-coded CLI parser, for consistency with
everything else.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-18-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:07 -04:00
John Snow
90bd8eb8dc scripts/qmp-shell: use isinstance() instead of type()
A bit more idiomatic, and quiets some linter warnings.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-17-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:07 -04:00
John Snow
73f699c903 scripts/qmp-shell: remove if-raise-else patterns
Shushes pylint. I don't always mind these patterns personally, but I'm
not as sure that I want to remove the warning from pylint's repertoire
entirely. Oh well.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-16-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:06 -04:00
John Snow
d962ec85ed scripts/qmp-shell: fix shell history exception handling
We want to remove exceptions that are too broad here; we only want to
catch IOErrors that get raised as a direct result of the open call.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-15-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:06 -04:00
John Snow
628b92dd67 scripts/qmp-shell: rename one and two-letter variables
A bit of churn and housekeeping for pylint, flake8 et al.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-14-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:06 -04:00
John Snow
2813dee053 scripts/qmp-shell: make QMPCompleter returns explicit
This function returns None when it doesn't find a match; do that
explicitly.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-13-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:06 -04:00
John Snow
50f6f1c308 scripts/qmp-shell: ignore visit_Name name
Not something I control, sorry, pylint.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-12-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:06 -04:00
John Snow
c6be2bf86e scripts/qmp-shell: use triple-double-quote docstring style
(2014 me had never written python before.)

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-11-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:06 -04:00
John Snow
31226369ab scripts/qmp-shell: declare verbose in __init__
Linters get angry when we don't define state at init time.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-10-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:06 -04:00
John Snow
5cb0233861 scripts/qmp-shell: Use python3-style super()
Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-9-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:06 -04:00
John Snow
c4a1447fc1 scripts/qmp-shell: use @classmethod where appropriate
Methods with no self-use should belong to the class.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-8-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:06 -04:00
John Snow
9669c8289c scripts/qmp-shell: remove shadowed variable from _print()
Don't use 'qmp' here, which shadows the qmp module.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-7-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:06 -04:00
John Snow
f2daa2d489 scripts/qmp-shell: fix connect method signature
It needs to match the parent's signature -- the negotiate parameter must
be optional.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-6-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:06 -04:00
John Snow
de14ba24f3 scripts/qmp-shell: fix exception handling
Fixes: 50d189c

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-5-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:06 -04:00
John Snow
70e5674018 scripts/qmp-shell: fix show_banner signature
The signatures need to match.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-4-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:06 -04:00
John Snow
169b43b367 scripts/qmp-shell: Apply flake8 rules
A lot of fiddling around to get us below 80 columns.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-3-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:06 -04:00
John Snow
badf462985 scripts/qmp-shell: apply isort rules
Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-2-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:06 -04:00
John Snow
88fb483fc5 scripts/qemu-ga-client: Add forwarder shim
Add a little forwarder shim until we are sure that everyone is
comfortable with how to use the tools in their new packaged location.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210604155532.1499282-12-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:06 -04:00
John Snow
1e129afc31 scripts/qemu-ga-client: move to python/qemu/qmp/qemu_ga_client.py
The script itself will be unavailable for a few commits before being
restored, with no way to run it right after this commit. This helps move
git history into the new file. To prevent linter regressions, though, we
do need to immediately touch up the filename to remove dashes (to make
the module importable), and remove the executable bit.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210604155532.1499282-10-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:06 -04:00
John Snow
ca683d4a2f scripts/qemu-ga-client: add mypy type hints
This script is in slightly rough shape, but it still works. A lot of
care went into its initial development. In good faith, I'm updating it
to the latest Python coding standards. If there is in interest in this
script, though, I'll be asking for a contributor to take care of it
further.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210604155532.1499282-9-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:06 -04:00
John Snow
f85d3252ef scripts/qemu-ga-client: apply (most) pylint rules
I'm only doing a very quick best-effort to preserve this script, to help
keep it from breaking further. I think there are pending ideas swirling
on the right way to implement better SDKs and better clients, and this
script might be a handy reference for those discussions. It presents
some interesting design problems, like static type safety when using a
dynamic RPC mechanism.

I believe it's worth preserving the effort and care that went into
making this script by updating it to work with our current
infrastructure. However, I am disabling the requirement for docstrings
in this file.

If you would like to help improve this script, please add docstrings
alongside any refactors or rejuvenations you might apply at that time.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210604155532.1499282-7-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:06 -04:00
John Snow
aff103b531 scripts/qemu-ga-client: add module docstring
Turn that nice usage comment into a docstring.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210604155532.1499282-6-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:06 -04:00
John Snow
0cf1a52d28 scripts/qemu-ga-client: replace deprecated optparse with argparse
optparse isn't supported anymore, it's from the python2 days. Replace it
with the mostly similar argparse.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210604155532.1499282-5-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:06 -04:00
John Snow
e6de9ce90a scripts/qemu-ga-client: Fix exception handling
Fixes: 50d189c

These error classes aren't available anymore. Fix the bitrot.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210604155532.1499282-4-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:06 -04:00
John Snow
e75f516ac1 scripts/qemu-ga-client: apply (most) flake8 rules
- Line length should be < 80
- You shouldn't perform unscoped imports except at the top of the module

Notably, the sys.path hack creates problems with the import rule. This
will be fixed later.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210604155532.1499282-3-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:06 -04:00
John Snow
9510e4fb69 scripts/qemu-ga-client: apply isort rules
Hint:
> ln -s scripts/qmp/qemu-ga-client python/qemu/qmp/qemu_ga_client.py
> cd python
> isort qemu

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210604155532.1499282-2-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:06 -04:00
John Snow
c89b38cd0a scripts/qom-fuse: add redirection shim to python/qemu/qmp/qom-fuse.py
By leaving the script absent for a commit, git-blame travels to the new
file instead of staying on the shim.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-id: 20210603003719.1321369-19-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:06 -04:00
John Snow
173d185de9 scripts/qom-fuse: move to python/qemu/qmp/qom_fuse.py
Move qom-fuse over to the python package now that it passes the
linter. Update the import paradigms so that it continues to pass in the
context of the Python package.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210603003719.1321369-18-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:06 -04:00
John Snow
30ec845c59 scripts/qom-fuse: add static type hints
Because fusepy does not have type hints, add some targeted warning
suppressions.

Namely, we need to allow subclassing something of an unknown type (in
qom_fuse.py), and we need to allow missing imports (recorded against
fuse itself) because mypy will be unable to import fusepy (even when
installed) as it has no types nor type stubs available.

Note: Until now, it was possible to run invocations like 'mypy qemu/'
from ./python and have that work. However, these targeted suppressions
require that you run 'mypy -p qemu/' instead. The correct, canonical
invocation is recorded in ./python/tests/mypy.sh and all of the various
CI invocations always use this correct form.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210603003719.1321369-16-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:06 -04:00
John Snow
2cea713462 scripts/qom-fuse: ensure QOMFuse.read always returns bytes
- Use FuseOSError to signal ENOENT instead of returning it
- Wrap qom-get in str(), as we don't always know its type
- The empty return should be b'', not ''.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-id: 20210603003719.1321369-15-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:06 -04:00
John Snow
9ec8a38694 scripts/qom-fuse: use QOMCommand.qom_list()
the qom_list method provides a type-safe object that's easier to type
check, so switch to using it.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-id: 20210603003719.1321369-14-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:06 -04:00
John Snow
2aa101799a scripts/qom-fuse: Convert to QOMCommand
Move qom-fuse onto the QOMCommand base established in
python/qemu/qmp/qom_common.py. The interface doesn't change
incompatibly, "qom-fuse mountpoint" still works as an invocation, and
QMP_SOCKET is still used as the environment variable.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210603003719.1321369-13-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:06 -04:00
John Snow
187be27c7b scripts/qom-fuse: Add docstrings
The methods inherited from fuse don't need docstrings; that's up to
fusepy to handle.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210603003719.1321369-12-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:06 -04:00
John Snow
7552823a36 scripts/qom-fuse: Apply pylint rules
- Catch specific exceptions from QMP
- Reraise errors with explicit context
- method parameters should match parent's names

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-id: 20210603003719.1321369-11-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:06 -04:00
John Snow
26c1ccadc4 scripts/qom-fuse: apply flake8 rules
flake8 still has one warning because of the sys.path hack, but that will
be going away by the end of this patch series.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-id: 20210603003719.1321369-9-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:06 -04:00
John Snow
c6b7eae9b6 scripts/qom-fuse: apply isort rules
Hint: you can use symlinks to create qom_fuse.py in python/qemu/qmp/ and
point to scripts/qom-fuse to apply the standard linting rules to this
script.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-id: 20210603003719.1321369-8-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:06 -04:00
John Snow
908ff4b29f scripts/qmp: redirect qom-xxx scripts to python/qemu/qmp/
Redirect to the new qom scripts. These forwarders can be deleted
eventually when there has been more time for the dust on the Python
packaging to settle and people understand how to find these commands.

Note: You can run these by setting $PYTHONPATH in your shell and then
running "python3 -m qemu.qmp.qom", or you can install the qemu namespace
package and use the "qom" or "qom-set" scripts.

I've written how to install the package elsewhere, but for the sake of
git-blame, cd to ./python, and then do:

- pip3 install [--user] [-e] .

--user will install to your local user install (will not work inside of
  a venv), omitting this flag installs to your system-wide packages
  (outside of a venv) or to your current virtual environment (inside the
  venv).

  When installing to a venv or to your system-wide packages, "qom"
  should be in your $PATH already. If you do a user install, you may
  need to add ~/.local/bin to your $PATH if you haven't already.

-e installs in editable mode: the installed package is effectively just
 a symlink to this folder; so changes to your git working tree are
 reflected in the installed package.

Note: installing these packages to an environment outside a venv can be
dangerous: Many QEMU scripts will begin to prefer the installed version
instead of the version directly in the tree. Use with caution. editable
mode is recommended when working outside of a venv.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-id: 20210603003719.1321369-7-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:06 -04:00
John Snow
587adaca55 python/qmp: add parse_address classmethod
This takes the place of qmp-shell's __get_address function. It also
allows other utilities to share the same parser and syntax for
specifying QMP locations.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210603003719.1321369-4-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-18 16:10:06 -04:00
Vladimir Sementsov-Ogievskiy
bb43694872 block-coroutine-wrapper: allow non bdrv_ prefix
We are going to reuse the script to generate a nbd_ function in
further commit. Prepare the script now.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210610100802.5888-28-vsementsov@virtuozzo.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
2021-06-18 12:21:21 -05:00
Daniel P. Berrangé
4e2f5f3a9d scripts: helper to generate x86_64 CPU ABI compat info
This script is what is used to generate the docs data table in:

  docs/system/cpu-models-x86-abi.csv

It can be useful to run if adding new CPU models / versions and
the csv needs updating.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Acked-by: Eduardo Habkost <ehabkost@redhat.com>
Message-Id: <20210607135843.196595-4-berrange@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2021-06-17 14:11:06 -04:00
David Hildenbrand
7f863cba4d softmmu/memory: Pass ram_flags to memory_region_init_ram_shared_nomigrate()
Let's forward ram_flags instead, renaming
memory_region_init_ram_shared_nomigrate() into
memory_region_init_ram_flags_nomigrate().

Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Acked-by: Eduardo Habkost <ehabkost@redhat.com> for memory backend and machine core
Signed-off-by: David Hildenbrand <david@redhat.com>
Message-Id: <20210510114328.21835-6-david@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-06-15 20:27:38 +02:00
Peter Maydell
33ba8b0adc Trivial branch pull request 20210607
-----BEGIN PGP SIGNATURE-----
 
 iQJGBAABCAAwFiEEzS913cjjpNwuT1Fz8ww4vT8vvjwFAmC+a+sSHGxhdXJlbnRA
 dml2aWVyLmV1AAoJEPMMOL0/L748SP0QAKogW9E/SFTKuHVAK+hgLP5UFT6oU4/a
 4/124nwf4iBXiwKpE5+M11EYdEpKJ6plohimq70TXUqGRMtDw/XSZNYwSHYVBZRi
 FB7haXwqeV9eioZZMvdSzo33CJXWzMdBWEohVLEStGMB7SZDOyRPG+E5tAUKt8zQ
 kn+nSWRq6GziyDV6OXpDey+1x7lRGpI7MqvJPAzsa59EiYwAyd5UFICX96DV69YN
 qA1+RKvBuI4zO7iqoCj1NR9tdQJnEY7f3TGFCG/JnR9Oveg0QRpnYPU3xe3Hrg6h
 BR5YScmQTGdDdDpeShECfwcUPYwpLH+OWGuo4QKRx2Dyv2/kIUjHxs+tHjY9XRXA
 FwCHM1zoKQPN4yf6XBdNHt4sDZZuGwc2aX58qtiJS3MMyzAuLd3ul7Jdt87et4zk
 BP4JWM2vkF6Mv3ethlpcSHSyLbKcs5h3iPl1w1oqjkM1tZKIdE0L8bsEJ7B6/E7z
 qXFdS4SdrG3RPkOJZyoPn6IH3t+5EAk9FULLmhzdsqSwNZTfRLkrGLXhTBDStnZH
 TElRLvHIZSiTgEftSJpxW0si+aIQtGpRF5F5lwjVMMZbRpttelXKTrfio5Zcy3op
 /uEJOJRB2FdDEvFaquoCEHg1sb2N1PTKb4MOPsu2lKBMi2X+POpFfnlK6skyhhA/
 NtPKb5zcHvdu
 =9AIJ
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/vivier2/tags/trivial-branch-for-6.1-pull-request' into staging

Trivial branch pull request 20210607

# gpg: Signature made Mon 07 Jun 2021 19:56:43 BST
# gpg:                using RSA key CD2F75DDC8E3A4DC2E4F5173F30C38BD3F2FBE3C
# gpg:                issuer "laurent@vivier.eu"
# gpg: Good signature from "Laurent Vivier <lvivier@redhat.com>" [full]
# gpg:                 aka "Laurent Vivier <laurent@vivier.eu>" [full]
# gpg:                 aka "Laurent Vivier (Red Hat) <lvivier@redhat.com>" [full]
# Primary key fingerprint: CD2F 75DD C8E3 A4DC 2E4F  5173 F30C 38BD 3F2F BE3C

* remotes/vivier2/tags/trivial-branch-for-6.1-pull-request:
  vhost-vdpa: Remove redundant declaration of address_space_memory
  scripts/oss-fuzz: Fix typo in documentation
  target/mips: Fix 'Uncoditional' typo
  target/hppa: Remove unused 'memory.h' header
  hw/display/macfb: Classify the "nubus-macfb" as display device
  target/nios2: fix page-fit instruction count
  docs: fix broken reference
  linux-user/syscall: Constify bitmask_transtbl fcntl/mmap flags_tlb[]
  misc: Correct relative include path
  i386/kvm: The value passed to strerror should be positive
  target/riscv: Do not include 'pmp.h' in user emulation

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-06-07 20:05:29 +01:00
Matheus Ferst
72205289a0 scripts/checkpatch.pl: process .c.inc and .h.inc files as C source
Change the regex used to determine whether a file should be processed as
C source to include .c.inc and .h.inc extensions.

Signed-off-by: Matheus Ferst <matheus.ferst@eldorado.org.br>
Reviewed-by: Luis Pires <luis.pires@eldorado.org.br>
Message-Id: <20210520195142.941261-1-matheus.ferst@eldorado.org.br>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2021-06-07 14:49:30 +01:00
Philippe Mathieu-Daudé
eeae5466c4 scripts/oss-fuzz: Fix typo in documentation
While we only use stdin, the chardev is named 'stdio'.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Alexander Bulekov <alxndr@bu.edu>
Message-Id: <20210602170759.2500248-4-f4bug@amsat.org>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
2021-06-05 21:29:39 +02:00
Stefano Garzarella
d0fb9657a3 docs: fix references to docs/devel/tracing.rst
Commit e50caf4a5c ("tracing: convert documentation to rST")
converted docs/devel/tracing.txt to docs/devel/tracing.rst.

We still have several references to the old file, so let's fix them
with the following command:

  sed -i s/tracing.txt/tracing.rst/ $(git grep -l docs/devel/tracing.txt)

Signed-off-by: Stefano Garzarella <sgarzare@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210517151702.109066-2-sgarzare@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-06-02 06:51:09 +02:00
Vivek Kasireddy
4d01086161 headers: Add udmabuf.h
This adds udmabuf header to standard headers so that the
relevant udmabuf objects can be accessed in subsequent
patches.

Based-on-patch-by: Gerd Hoffmann <kraxel@redhat.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Signed-off-by: Vivek Kasireddy <vivek.kasireddy@intel.com>
Message-Id: <20210526231429.1045476-3-vivek.kasireddy@intel.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
2021-05-27 12:06:37 +02:00
Peter Maydell
6c769690ac scripts/simplebench improvements for 2021-05-04
-----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEEi5wmzbL9FHyIDoahVh8kwfGfefsFAmCRCZsACgkQVh8kwfGf
 efuD8g//cALt2FiC5R0RwVqUhZ/RAOHM7LrENx8GxHgzZZMiWuF7hX9JvAtNzyUz
 dflFxkOv1mhBfbXpU6ymrsfR5AKzyXlKMWfi14tGJKxIfJODPmBPVwM5IZmQCuMG
 2laD5XPIj/sgIRUZXN5dXlyGlnTujyKpeubgN1dx8g+xJa2MvZ3lRJciEvGjJNbJ
 QZX3kEQaBAkjSeDcS8RH1S3VoEaaon3rUyOUap6WpbDiLM/ZcgYZYQEjBnkinJXm
 IJoUr5cR8gh9oxz82XFTJFLJ6ujUY/yMjHoHDRXCtmdlWU56oqJPQxfcqnzbOSk1
 OrdIOofN8qM4ks4W5SL4aquBy3DIgfSZui1/+UboKGfQBU5/vxAuzsv9LCgiG4PL
 Xkitpy09scxerKci9W9zyBHpMTIhIaf/ISxtYWaZlUSDLiuh1O6d28/TmMlRz3Xb
 HUW9TZqhzFkmw8koqe2HcgCoWW4MHaFoBvwe/GTuQySJC8hts7gO36znN4Y3yTzS
 9GcBybVnn6hKXn+N90iqgkYHNXJ3+CfF149Gfg30jg8Fm9oEJKFuAJnPNm5HQAoh
 3LOF3D/qq7tmU6imE8molU1uTgxcIpA5fYu5rAxFPNUF5QpfATh7/IIp2Y7+ZCOg
 9OE9uLJalUe3juZTccv/3KFRjqnewz0zT8UQBLrL3XNZBC52AM0=
 =Abdk
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/vsementsov/tags/pull-simplebench-2021-05-04' into staging

scripts/simplebench improvements for 2021-05-04

# gpg: Signature made Tue 04 May 2021 09:45:15 BST
# gpg:                using RSA key 8B9C26CDB2FD147C880E86A1561F24C1F19F79FB
# gpg: Good signature from "Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>" [unknown]
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg:          There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 8B9C 26CD B2FD 147C 880E  86A1 561F 24C1 F19F 79FB

* remotes/vsementsov/tags/pull-simplebench-2021-05-04:
  MAINTAINERS: update Benchmark util: add git tree
  simplebench/bench-backup: add --drop-caches argument
  simplebench/bench-backup: add --count and --no-initial-run
  simplebench/bench-backup: support qcow2 source files
  simplebench/bench_block_job: handle error in BLOCK_JOB_COMPLETED
  simplebench/bench-backup: add target-cache argument
  simplebench/bench-backup: add --compressed option
  simplebench: bench_one(): support count=1
  simplebench: bench_one(): add slow_limit argument

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-05-21 12:02:34 +01:00
John Snow
d4092ffa26 qapi/parser: add docstrings
Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210519183951.3946870-16-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Doc string spacing tweaked slightly]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-05-20 17:10:09 +02:00
John Snow
9b91e76b3a qapi/parser: allow 'ch' variable name
We can have a two-letter variable name, as a treat.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210519183951.3946870-15-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-05-20 17:09:44 +02:00
John Snow
013a3aceb5 qapi/parser: Remove superfluous list comprehension
A generator suffices (and quiets a pylint warning).

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210519183951.3946870-14-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-05-20 11:28:29 +02:00
John Snow
810aff8f29 qapi/parser: add type hint annotations
Annotations do not change runtime behavior.
This commit *only* adds annotations.

(Annotations for QAPIDoc are in a forthcoming commit.)

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210519183951.3946870-13-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-05-20 11:28:28 +02:00
John Snow
03386200b9 qapi/parser: Rework _check_pragma_list_of_str as a TypeGuard
TypeGuards wont exist in Python proper until 3.10. Ah well. We can hack
up our own by declaring this function to return the type we claim it
checks for and using this to safely downcast object -> List[str].

In so doing, I bring this function under _pragma so it can use the
'info' object in its closure. Having done this, _pragma also now no
longer needs to take a 'self' parameter, so drop it.

To help with line-length, and with the context evident from its new
scope, rename the function to the shorter check_list_str().

Signed-off-by: John Snow <jsnow@redhat.com>

Message-Id: <20210519183951.3946870-12-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-05-20 11:28:28 +02:00
John Snow
c256263f3d qapi/parser: Fix token membership tests when token can be None
When the token can be None (EOF), we can't use 'x in "abc"' style
membership tests to group types of tokens together, because 'None in
"abc"' is a TypeError.

Easy enough to fix. (Use a tuple: It's neither a static typing error nor
a runtime error to check for None in Tuple[str, ...])

Add tests to prevent a regression. (Note: they cannot be added prior to
this fix, as the unhandled stack trace will not match test output in the
CI system.)

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210519183951.3946870-11-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-05-20 11:28:28 +02:00
John Snow
e0e8a0ac2e qapi: add must_match helper
Mypy cannot generally understand that these regex functions cannot
possibly fail. Add a "must_match" helper that makes this clear for
mypy.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210519183951.3946870-10-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-05-20 11:28:28 +02:00
John Snow
43b1be65f0 qapi/parser: Use @staticmethod where appropriate
No self, no thank you!

(Quiets pylint warnings.)

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210519183951.3946870-9-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-05-20 11:28:28 +02:00
John Snow
234dce2c2d qapi/parser: assert object keys are strings
The single quote token implies the value is a string. Assert this to be
the case, to allow us to write an accurate return type for get_members.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210519183951.3946870-8-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-05-20 11:28:28 +02:00
John Snow
9cd0205d55 qapi/parser: enforce all top-level expressions must be dict in _parse()
Instead of using get_expr nested=False, allow get_expr to always return
any expression. In exchange, add a new error message to the top-level
parser that explains the semantic error: Top-level expressions must
always be JSON objects.

This helps mypy understand the rest of this function which assumes that
get_expr did indeed return a dict.

The exception type changes from QAPIParseError to QAPISemError as a
result, and the error message in two tests now changes.

Signed-off-by: John Snow <jsnow@redhat.com>

Message-Id: <20210519183951.3946870-7-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-05-20 11:28:28 +02:00
John Snow
7c610ce6a9 qapi/parser: Assert lexer value is a string
The type checker can't narrow the type of the token value to string,
because it's only loosely correlated with the return token.

We know that a token of '#' should always have a "str" value.
Add an assertion.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210519183951.3946870-6-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-05-20 11:28:27 +02:00
John Snow
16ff40acc9 qapi/parser: factor parsing routine into method
For the sake of keeping __init__ smaller (and treating it more like a
gallery of what state variables we can expect to see), put the actual
parsing action into a parse method. It remains invoked from the init
method to reduce churn.

To accomplish this, @previously_included becomes the private data
member ._included, and the filename is stashed as ._fname.

Add any missing declarations to the init method, and group them by
function so they can be understood quickly at a glance.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210519183951.3946870-5-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-05-20 11:28:27 +02:00
John Snow
b2b31fdf9b qapi/source: Remove line number from QAPISourceInfo initializer
With the QAPISourceInfo(None, None, None) construct gone, there's no
longer any reason to have to specify that a file starts on the first
line. Remove it from the initializer and default it to 1.

Remove the last vestiges where we check for 'line' being unset, that
can't happen, now.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210519183951.3946870-4-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-05-20 11:28:27 +02:00
John Snow
3404e57410 qapi/parser: Don't try to handle file errors
Fixes: f5d4361cda
Fixes: 52a474180a
Fixes: 46f49468c6

Remove the try/except block that handles file-opening errors in
QAPISchemaParser.__init__() and add one each to
QAPISchemaParser._include() and QAPISchema.__init__() respectively.

This simultaneously fixes the typing of info.fname (f5d4361cda), A
static typing violation in test-qapi (46f49468c6), and a regression of
an error message (52a474180a).

The short-ish version of what motivates this patch is:

- It's hard to write a good error message in the init method,
  because we need to determine the context of our caller to do so.
  It's easier to just let the caller write the message.
- We don't want to allow QAPISourceInfo(None, None, None) to exist. The
  typing introduced by commit f5d4361cda types the 'fname' field as
  (non-optional) str, which was premature until the removal of this
  construct.
- Errors made using such an object are currently incorrect (since
  52a474180a)
- It's not technically a semantic error if we cannot open the schema.
- There are various typing constraints that make mixing these two cases
  undesirable for a single special case.
- test-qapi's code handling an fname of 'None' is now dead, drop it.
  Additionally, Not all QAPIError objects have an 'info' field (since
  46f49468), so deleting this stanza corrects a typing oversight in
  test-qapi introduced by that commit.

Other considerations:

- open() is moved to a 'with' block to ensure file pointers are
  cleaned up deterministically.
- Python 3.3 deprecated IOError and made it a synonym for OSError.
  Avoid the misleading perception these exception handlers are
  narrower than they really are.

The long version:

The error message here is incorrect (since commit 52a474180a):

> python3 qapi-gen.py 'fake.json'
qapi-gen.py: qapi-gen.py: can't read schema file 'fake.json': No such file or directory

In pursuing it, we find that QAPISourceInfo has a special accommodation
for when there's no filename. Meanwhile, the intent when QAPISourceInfo
was typed (f5d4361cda) was non-optional 'str'. This usage was
overlooked.

To remove this, I'd want to avoid having a "fake" QAPISourceInfo
object. I also don't want to explicitly begin accommodating
QAPISourceInfo itself being None, because we actually want to eventually
prove that this can never happen -- We don't want to confuse "The file
isn't open yet" with "This error stems from a definition that wasn't
defined in any file".

(An earlier series tried to create a dummy info object, but it was tough
to prove in review that it worked correctly without creating new
regressions. This patch avoids that distraction. We would like to first
prove that we never raise QAPISemError for any built-in object before we
add "special" info objects. We aren't ready to do that yet.)

So, which way out of the labyrinth?

Here's one way: Don't try to handle errors at a level with "mixed"
semantic contexts; i.e. don't mix inclusion errors (should report a
source line where the include was triggered) and command line errors
(where we specified a file we couldn't read).

Remove the error handling from the initializer of the parser. Pythonic!
Now it's the caller's job to figure out what to do about it. Handle the
error in QAPISchemaParser._include() instead, where we can write a
targeted error message where we are guaranteed to have an 'info' context
to report with.

The root level error can similarly move to QAPISchema.__init__(), where
we know we'll never have an info context to report with, so we use a
more abstract error type.

Now the error looks sensible again:

> python3 qapi-gen.py 'fake.json'
qapi-gen.py: can't read schema file 'fake.json': No such file or directory

With these error cases separated, QAPISourceInfo can be solidified as
never having placeholder arguments that violate our desired types. Clean
up test-qapi along similar lines.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210519183951.3946870-2-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-05-20 11:28:27 +02:00
Peter Maydell
6005ee07c3 pc,pci,virtio: bugfixes, improvements
Fixes all over the place. Faster boot for virtio. ioeventfd support for
 mmio.
 
 Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
 -----BEGIN PGP SIGNATURE-----
 
 iQFDBAABCAAtFiEEXQn9CHHI+FuUyooNKB8NuNKNVGkFAmCeiMEPHG1zdEByZWRo
 YXQuY29tAAoJECgfDbjSjVRpqsIH/A49Av5Bv8huL75lf9GzCx3E1a/z2W9Fphik
 OcQ1ahR+7CRDARub+vTG40MBmZBVefIWjLAj3BwBWzFGPX0DZq0zeI102VzlEVKY
 OeUx8ixuiKOSLcS+QxE7ZXIBL2Pn7l+MFUi4nLMYKti7c/kola7zlB57qsmXh+VD
 AOQ7Utj6NWoi6QocWJsMSCyHCh3Fk9QzcStLlr6/MkSJa1zqv8l22+8oWH07Fk2M
 wZfhrm9k094on28iSejsFYL5e4ROeXUajbOdfyMIxWvAB7boC9Jxk/e0oAbuSB4y
 2f71Gfk3mU6irS7PvrxcKbk6BVD2zxM2WumOchZJgxFAujDO6yg=
 =fvkT
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/mst/tags/for_upstream' into staging

pc,pci,virtio: bugfixes, improvements

Fixes all over the place. Faster boot for virtio. ioeventfd support for
mmio.

Signed-off-by: Michael S. Tsirkin <mst@redhat.com>

# gpg: Signature made Fri 14 May 2021 15:27:13 BST
# gpg:                using RSA key 5D09FD0871C8F85B94CA8A0D281F0DB8D28D5469
# gpg:                issuer "mst@redhat.com"
# gpg: Good signature from "Michael S. Tsirkin <mst@kernel.org>" [full]
# gpg:                 aka "Michael S. Tsirkin <mst@redhat.com>" [full]
# Primary key fingerprint: 0270 606B 6F3C DF3D 0B17  0970 C350 3912 AFBE 8E67
#      Subkey fingerprint: 5D09 FD08 71C8 F85B 94CA  8A0D 281F 0DB8 D28D 5469

* remotes/mst/tags/for_upstream:
  Fix build with 64 bits time_t
  vhost-vdpa: Make vhost_vdpa_get_device_id() static
  hw/virtio: enable ioeventfd configuring for mmio
  hw/smbios: support for type 41 (onboard devices extended information)
  checkpatch: Fix use of uninitialized value
  virtio-scsi: Configure all host notifiers in a single MR transaction
  virtio-scsi: Set host notifiers and callbacks separately
  virtio-blk: Configure all host notifiers in a single MR transaction
  virtio-blk: Fix rollback path in virtio_blk_data_plane_start()
  pc-dimm: remove unnecessary get_vmstate_memory_region() method
  amd_iommu: fix wrong MMIO operations
  virtio-net: Constify VirtIOFeature feature_sizes[]
  virtio-blk: Constify VirtIOFeature feature_sizes[]
  hw/virtio: Pass virtio_feature_get_config_size() a const argument
  x86: acpi: use offset instead of pointer when using build_header()
  amd_iommu: Fix pte_override_page_mask()

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

# Conflicts:
#	hw/arm/virt.c
2021-05-16 17:22:46 +01:00
Greg Kurz
43bea44357 checkpatch: Fix use of uninitialized value
checkfilename() doesn't always set $acpi_testexpected. Fix the following
warning:

Use of uninitialized value $acpi_testexpected in string eq at
 ./scripts/checkpatch.pl line 1529.

Fixes: d2f1af0e41 ("checkpatch: don't emit warning on newly created acpi data files")
Cc: isaku.yamahata@intel.com
Signed-off-by: Greg Kurz <groug@kaod.org>
Message-Id: <161786467973.295167.5612704777283969903.stgit@bahia.lan>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2021-05-14 10:26:18 -04:00
Paolo Bonzini
e804f892b9 coverity-scan: list components, move model to scripts/coverity-scan
Place all files that can be useful to rebuild the Coverity
configuration in scripts/coverity-scan: the existing model
file, and the components setup.

The Markdown syntax was tested with Pandoc (but in any case
is meant more as a human-readable reference than as a part
of documentation).

Suggested-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-05-12 04:06:50 -04:00
Vladimir Sementsov-Ogievskiy
8c8407fe45 simplebench/bench-backup: add --drop-caches argument
Add an option to drop caches before each test run. It may probably
improve reliability of results when testing in cached mode.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2021-05-04 11:37:26 +03:00
Vladimir Sementsov-Ogievskiy
8fa4ee1b00 simplebench/bench-backup: add --count and --no-initial-run
Add arguments to set number of test runs per table cell and to disable
initial run that is not counted in results.

It's convenient to set --count 1 --no-initial-run to fast run test
onece, and to set --count to some large enough number for good
precision of the results.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2021-05-04 11:37:26 +03:00
Vladimir Sementsov-Ogievskiy
0dc9f816d4 simplebench/bench-backup: support qcow2 source files
Add support for qcow2 source. New option says to use test-source.qcow2
instead of test-source. Of course, test-source.qcow2 should be
precreated.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: John Snow <jsnow@redhat.com>
2021-05-04 11:37:26 +03:00
Vladimir Sementsov-Ogievskiy
4ad7a5c065 simplebench/bench_block_job: handle error in BLOCK_JOB_COMPLETED
We should not report success if there is an error in final event.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: John Snow <jsnow@redhat.com>
2021-05-04 11:37:26 +03:00
Vladimir Sementsov-Ogievskiy
684d18efd1 simplebench/bench-backup: add target-cache argument
Allow benchmark with different kinds of target cache.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: John Snow <jsnow@redhat.com>
2021-05-04 11:37:26 +03:00
Vladimir Sementsov-Ogievskiy
af2ac8514f simplebench/bench-backup: add --compressed option
Allow bench compressed backup.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2021-05-04 11:37:26 +03:00
Vladimir Sementsov-Ogievskiy
27eacb390e simplebench: bench_one(): support count=1
statistics.stdev raises if sequence length is less than two. Support
that case by hand.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: John Snow <jsnow@redhat.com>
2021-05-04 11:37:26 +03:00
Vladimir Sementsov-Ogievskiy
dab346986e simplebench: bench_one(): add slow_limit argument
Sometimes one of cells in a testing table runs too slow. And we really
don't want to wait so long. Limit number of runs in this case.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2021-05-04 11:37:26 +03:00
Silvano Cirujano Cuesta
e75941331e scripts: fix generation update-binfmts templates
This patch fixes the update-binfmts templates being used in the script
scripts/qemu-binfmt-conf.sh when the option --debian is used.

Fixed issues are:
- Typo in flag 'credentials' (previously 'credential').
- Missing flags 'preserve' and 'fix_binary'.

Reference: https://manpages.debian.org/buster/binfmt-support/update-binfmts.8.en.html#FORMAT_FILES

Signed-off-by: Silvano Cirujano Cuesta <silvano.cirujano-cuesta@siemens.com>
Reviewed-by: Laurent Vivier <laurent@vivier.eu>
Message-Id: <20210323123457.23747-1-silvano.cirujano-cuesta@siemens.com>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
2021-05-03 10:07:41 +02:00
Richard Henderson
af93ccacc7 decodetree: Extend argument set syntax to allow types
Rather than force all structure members to be 'int',
allow the type of the member to be specified.

Reviewed-by: Luis Pires <luis.pires@eldorado.org.br>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-05-01 11:45:35 -07:00
Luis Fernando Fujita Pires
60c425f328 decodetree: Add support for 64-bit instructions
Allow '64' to be specified for the instruction width command line params
and use the appropriate extract and deposit functions in that case.

This will be used to implement the new 64-bit Power ISA 3.1 instructions.

Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Luis Pires <luis.pires@eldorado.org.br>
Message-Id: <CP2PR80MB3668E123E2EFDB0ACD3A46F1DA759@CP2PR80MB3668.lamprd80.prod.outlook.com>
[rth: Drop the change to the field type; use bitop_width instead of separate
variables for extract/deposit; use "ull" for 64-bit constants.]
Reviewed-by: Luis Pires <luis.pires@eldorado.org.br>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-05-01 11:45:19 -07:00
Richard Henderson
9f6e2b4d34 decodetree: More use of f-strings
Reviewed-by: Luis Pires <luis.pires@eldorado.org.br>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-05-01 11:45:02 -07:00
Richard Henderson
c7cefe6c66 decodetree: Introduce whex and whexC helpers
Form a hex constant of the appropriate insnwidth.
Begin using f-strings on changed lines.

Reviewed-by: Luis Pires <luis.pires@eldorado.org.br>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-05-01 11:44:56 -07:00
John Snow
b54626e0b8 qapi/error.py: enable mypy checks
Signed-off-by: John Snow <jsnow@redhat.com>

Message-Id: <20210421192233.3542904-9-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-04-30 12:59:54 +02:00
John Snow
30d0a016e9 qapi/error: Add type hints
No functional change.

Note: QAPISourceError's info parameter is Optional[] because schema.py
treats the info property of its various classes as Optional to
accommodate built-in types, which have no source. See prior commit
'qapi/error: assert QAPISourceInfo is not None'.

Signed-off-by: John Snow <jsnow@redhat.com>

Message-Id: <20210421192233.3542904-8-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-04-30 12:59:54 +02:00
John Snow
92870cf3af qapi/error.py: enable pylint checks
Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210421192233.3542904-7-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-04-30 12:59:54 +02:00
John Snow
ac6a7d8884 qapi/error.py: move QAPIParseError to parser.py
Keeping it in error.py will create some cyclic import problems when we
add types to the QAPISchemaParser. Callers don't need to know the
details of QAPIParseError unless they are parsing or dealing directly
with the parser, so this won't create any harsh new requirements for
callers in the general case.

Update error.py with a little docstring that gives a nod to where the
error may now be found.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210421192233.3542904-6-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-04-30 12:59:54 +02:00
John Snow
ac89761179 qapi/error: assert QAPISourceInfo is not None
Built-in stuff is not parsed from a source file, and therefore have no
QAPISourceInfo. If such None info was used for reporting an error,
built-in stuff would be broken. Programming error. Instead of reporting
a confusing error with bogus source location then, we better crash.

We currently crash only if self.col was set. Assert that self.info is
not None in order to crash reliably.

We can not yet change the type of the initializer to prove this cannot
happen at static analysis time before the remainder of the code is fully
typed.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210421192233.3542904-5-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-04-30 12:59:54 +02:00
John Snow
86cc2ff65a qapi/error: Make QAPISourceError 'col' parameter optional
It's already treated as optional, with one direct caller and some
subclass callers passing 'None'. Make it officially optional, which
requires moving the position of the argument to come after all required
parameters.

QAPISemError becomes functionally identical to QAPISourceError. Keep the
name to preserve its semantic meaning and avoid code churn, but remove
the now-useless __init__ wrapper.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210421192233.3542904-4-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-04-30 12:59:54 +02:00
John Snow
b54e07cc46 qapi/error: Use Python3-style super()
Missed in commit 2cae67bcb5 "qapi: Use super() now we have Python 3".

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210421192233.3542904-3-jsnow@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-04-30 12:59:54 +02:00
John Snow
46f49468c6 qapi/error: Repurpose QAPIError as an abstract base exception class
Rename QAPIError to QAPISourceError, and then create a new QAPIError
class that serves as the basis for all of our other custom exceptions,
without specifying any class properties.

This leaves QAPIError as a package-wide error class that's suitable for
any current or future errors.

(Right now, we don't have any errors that DON'T also want to specify a
Source location, but this MAY change. In these cases, a common abstract
ancestor would be desired.)

Add docstrings to explain the intended function of each error class.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210421192233.3542904-2-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-04-30 12:59:54 +02:00
John Snow
e81718c698 qapi/expr: Update authorship and copyright information
Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210421182032.3521476-18-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-04-30 12:59:54 +02:00
John Snow
eab99939a7 qapi/expr.py: Use tuples instead of lists for static data
It is -- maybe -- possibly -- three nanoseconds faster.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20210421182032.3521476-17-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-04-30 12:59:54 +02:00
John Snow
a48653638f qapi/expr.py: Add docstrings
Now with more :words:!

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210421182032.3521476-16-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-04-30 12:59:54 +02:00
John Snow
79e4fd14fb qapi/expr: Only explicitly prohibit 'Kind' nor 'List' for type names
Per list review: qapi-code-gen.txt reserves suffixes Kind and
List only for type names, but the code rejects them for events and
commands, too.

It turns out we reject them earlier anyway: In check_name_upper() for
event names, and in check_name_lower() for command names.

Still, adjust the code for clarity over what precisely we are guarding
against.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210421182032.3521476-15-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-04-30 12:59:54 +02:00
John Snow
328e8ca71a qapi/expr.py: enable pylint checks
Signed-off-by: John Snow <jsnow@redhat.com>
Tested-by: Eduardo Habkost <ehabkost@redhat.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Tested-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20210421182032.3521476-14-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-04-30 12:59:54 +02:00
John Snow
e42648dccd qapi/expr.py: Remove single-letter variable
Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210421182032.3521476-13-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-04-30 12:59:54 +02:00
John Snow
210fd63104 qapi/expr.py: Consolidate check_if_str calls in check_if
This is a small rewrite to address some minor style nits.

Don't compare against the empty list to check for the empty condition, and
move the normalization forward to unify the check on the now-normalized
structure.

With the check unified, the local nested function isn't needed anymore
and can be brought down into the normal flow of the function. With the
nesting level changed, shuffle the error strings around a bit to get
them to fit in 79 columns.

Note: although ifcond is typed as Sequence[str] elsewhere, we *know* that
the parser will produce real, bona-fide lists. It's okay to check
isinstance(ifcond, list) here.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210421182032.3521476-12-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-04-30 12:59:54 +02:00
John Snow
b9ad358aa0 qapi/expr.py: add type hint annotations
Annotations do not change runtime behavior.
This commit *only* adds annotations.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210421182032.3521476-11-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-04-30 12:59:54 +02:00
John Snow
538cd41065 qapi/expr.py: Modify check_keys to accept any Collection
This is a minor adjustment that lets parameters @required and
@optional take tuple arguments, in particular ().  Later patches will
make use of that.

(Iterable would also have worked, but Iterable also includes things like
generator expressions which are consumed upon iteration, which would
require a rewrite to make sure that each input was only traversed
once. Collection implies the "can re-iterate" property.)

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210421182032.3521476-10-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-04-30 12:59:54 +02:00
John Snow
7a783ce5b5 qapi/expr.py: Add casts in a few select cases
Casts are instructions to the type checker only, they aren't "safe" and
should probably be avoided in general. In this case, when we perform
type checking on a nested structure, the type of each field does not
"stick".

(See PEP 647 for an example of "type narrowing" that does "stick".
 It is available in Python 3.10, so we can't use it yet.)

We don't need to assert that something is a str if we've already checked
or asserted that it is -- use a cast instead for these cases.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210421182032.3521476-9-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-04-30 12:59:54 +02:00
John Snow
4918bb7def qapi/expr.py: Check type of union and alternate 'data' member
Prior to this commit, specifying a non-object value here causes the QAPI
parser to crash in expr.py with a stack trace with (likely) an
AttributeError when we attempt to call that value's items() method.

This member needs to be an object (Dict), and not anything else. Add a
check for this with a nicer error message, and formalize that check with
new test cases that exercise that error.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210421182032.3521476-8-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-04-30 12:59:54 +02:00
John Snow
926bb8add7 qapi/expr.py: move string check upwards in check_type
For readability purposes only, shimmy the early return upwards to the
top of the function, so cases proceed in order from least to most
complex.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20210421182032.3521476-7-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-04-30 12:59:54 +02:00
John Snow
b66c62a2d3 qapi/expr.py: Add assertion for union type 'check_dict'
mypy isn't fond of allowing you to check for bool membership in a
collection of str elements. Guard this lookup for precisely when we were
given a name.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20210421182032.3521476-6-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-04-30 12:59:54 +02:00
John Snow
59b5556ce8 qapi/expr.py: constrain incoming expression types
mypy does not know the types of values stored in Dicts that masquerade
as objects. Help the type checker out by constraining the type.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210421182032.3521476-5-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-04-30 12:59:54 +02:00
John Snow
0f231dcf29 qapi/expr.py: Check for dict instead of OrderedDict
OrderedDict is a subtype of dict, so we can check for a more general
form. These functions do not themselves depend on it being any
particular type.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20210421182032.3521476-4-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-04-30 12:59:54 +02:00
John Snow
b7341b89c9 qapi/expr.py: Remove 'info' argument from nested check_if_str
The function can just use the argument from the scope above. Otherwise,
we get shadowed argument errors because the parameter name clashes with
the name of a variable already in-scope.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20210421182032.3521476-3-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-04-30 12:59:54 +02:00
John Snow
d0a263cdd0 qapi/expr: Comment cleanup
The linter yaps after 0825f62c84. Fix this trivial issue to restore the
linter baseline.

(Yes, ideally -- and soon -- the linter will be part of CI so we don't
clutter up the log with fixups. For now, though, the baseline is useful
for testing intermediate commits as types are added to the QAPI
library.)

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210421182032.3521476-2-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-04-30 12:59:54 +02:00
Thomas Huth
ca89d15f8e device-crash-test: Ignore errors about a bus not being available
Recent QEMU versions now sometimes exit cleanly with an error message
that a bus is not available for a specified device. Don't flag those
as an error in the device-crash-test script.

Message-Id: <20210323164718.1393792-1-thuth@redhat.com>
Acked-by: John Snow <jsnow@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-04-01 14:28:39 +02:00
Alex Bennée
b30df2751e scripts/kernel-doc: strip QEMU_ from function definitions
Some packaged versions of Sphinx (fedora33/alpine so far) have issues
with the annotated C code that kernel-doc spits out. Without knowing
about things like QEMU_PLUGIN_EXPORT it chokes trying to understand
the code. Evidently this is a problem for the kernel as well as the
long stream of regex substitutions we add to in this patch can attest.

Fortunately we have a fairly common format for all our compiler
shenanigans as applied to functions so lets just filter them all out.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20210323165308.15244-2-alex.bennee@linaro.org>
2021-03-24 14:24:40 +00:00
Markus Armbruster
d83b47646e qapi: Enforce union and alternate branch naming rules
Union branch names should use '-', not '_'.  Enforce this.  The only
offenders are in tests/.  Fix them.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210323094025.3569441-29-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[Commit message typo fixed]
2021-03-23 22:31:53 +01:00
Markus Armbruster
407efbf9e7 qapi: Enforce enum member naming rules
Enum members should use '-', not '_'.  Enforce this.  Fix the fixable
offenders (all in tests/), and add the remainder to pragma
member-name-exceptions.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210323094025.3569441-28-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2021-03-23 22:31:43 +01:00
Markus Armbruster
5aceeac04d qapi: Enforce struct member naming rules
Struct members, including command arguments, event data, and union
inline base members, should use '-', not '_'.  Enforce this.  Fix the
fixable offenders (all in tests/), and add the remainder to pragma
member-name-exceptions.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210323094025.3569441-27-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2021-03-23 22:31:43 +01:00
Markus Armbruster
05ebf841ef qapi: Enforce command naming rules
Command names should be lower-case.  Enforce this.  Fix the fixable
offenders (all in tests/), and add the remainder to pragma
command-name-exceptions.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210323094025.3569441-25-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2021-03-23 22:31:05 +01:00
Markus Armbruster
e744708a77 qapi: Enforce feature naming rules
Feature names should use '-', not '_'.  Enforce this.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210323094025.3569441-24-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2021-03-23 22:31:05 +01:00
Markus Armbruster
9af4b6b9e8 qapi: Prepare for rejecting underscore in command and member names
Command names and member names within a type should be all lower case
with words separated by a hyphen.  We also accept underscore.  Rework
check_name_lower() to optionally reject underscores, but don't use
that option, yet.

Update expected test output for the changed error message.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210323094025.3569441-23-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2021-03-23 22:31:05 +01:00
Markus Armbruster
b86df37478 qapi: Rename pragma *-whitelist to *-exceptions
Rename pragma returns-whitelist to command-returns-exceptions, and
name-case-whitelist to member-name-case-exceptions.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210323094025.3569441-20-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2021-03-23 22:31:05 +01:00
Markus Armbruster
4a67bd31a4 qapi: Factor out QAPISchemaParser._check_pragma_list_of_str()
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210323094025.3569441-17-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2021-03-23 22:31:05 +01:00
Markus Armbruster
3e6c8a6331 qapi: Enforce type naming rules
Type names should be CamelCase.  Enforce this.  The only offenders are
in tests/.  Fix them.  Add test type-case to cover the new error.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210323094025.3569441-15-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[Regexp simplified, new test made more robust]
2021-03-23 22:31:05 +01:00
Markus Armbruster
d4f4cae8de qapi: Enforce event naming rules
Event names should be ALL_CAPS with words separated by underscore.
Enforce this.  The only offenders are in tests/.  Fix them.  Existing
test event-case covers the new error.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210323094025.3569441-14-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2021-03-23 22:31:05 +01:00
Markus Armbruster
00ffe242d6 qapi: Consistently permit any case in downstream prefixes
We require lowercase __RFQDN_ downstream prefixes only where we
require the prefixed name to be lowercase.  Don't; permit any case in
__RFQDN_ prefixes anywhere.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210323094025.3569441-13-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2021-03-23 22:31:05 +01:00
Markus Armbruster
d224e0c092 qapi: Move uppercase rejection to check_name_lower()
check_name_lower() is the only user of check_name_str() using
permit_upper=False.  Move the associated code from check_name_str() to
check_name_lower(), and drop the parameter.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210323094025.3569441-12-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2021-03-23 22:31:05 +01:00
Markus Armbruster
eaab06faa5 qapi: Rework name checking in preparation of stricter checking
Naming rules differ for the various kinds of names.  To prepare
enforcing them, define functions to check them: check_name_upper(),
check_name_lower(), and check_name_camel().  For now, these merely
wrap around check_name_str(), but that will change shortly.  Replace
the other uses of check_name_str() by appropriate uses of the
wrappers.  No change in behavior just yet.

check_name_str() now returns the name without downstream and x-
prefix, for use by the wrappers in later patches.  Requires tweaking
regexp @valid_name.  It accepts the same strings as before.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210323094025.3569441-11-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[Commit message improved]
2021-03-23 22:30:51 +01:00
Markus Armbruster
0825f62c84 qapi: Lift enum-specific code out of check_name_str()
check_name_str() masks leading digits when passed enum_member=True.
Only check_enum() does.  Lift the masking into check_enum().

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210323094025.3569441-10-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2021-03-23 22:30:32 +01:00
Markus Armbruster
5fbc78dd36 qapi: Permit flat union members for any tag value
Flat union branch names match the tag enum's member names.  Omitted
branches default to "no members for this tag value".

Branch names starting with a digit get rejected like "'data' member
'0' has an invalid name".  However, omitting the branch works.

This is because flat union tag values get checked twice: as enum
member name, and as union branch name.  The former accepts leading
digits, the latter doesn't.

Branches whose names start with a digit therefore cannot have members.
Feels wrong.  Get rid of the restriction by skipping the latter check.

This can expose c_name() to input it can't handle: a name starting
with a digit.  Improve it to return a valid C identifier for any
input.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210323094025.3569441-9-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[Commit message rewritten]
2021-03-23 22:30:08 +01:00
Markus Armbruster
dbfe3c7c28 qapi: Fix to reject optional members with reserved names
check_type() fails to reject optional members with reserved names,
because it neglects to strip off the leading '*'.  Fix that.

The stripping in check_name_str() is now useless.  Drop.

Also drop the "no leading '*'" assertion, because valid_name.match()
ensures it can't fail.

Fixes: 9fb081e0b9
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210323094025.3569441-8-armbru@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
2021-03-23 22:29:37 +01:00
Markus Armbruster
db29164103 qapi: Implement deprecated-input=reject for QMP command arguments
This policy rejects deprecated input, and thus permits "testing the
future".  Implement it for QMP command arguments: reject commands with
deprecated ones.  Example: when QEMU is run with -compat
deprecated-input=reject, then

    {"execute": "eject", "arguments": {"device": "cd"}}

fails like this

    {"error": {"class": "GenericError", "desc": "Deprecated parameter 'device' disabled by policy"}}

When the deprecated parameter is removed, the error will change to

    {"error": {"class": "GenericError", "desc": "Parameter 'device' is unexpected"}}

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210318155519.1224118-11-armbru@redhat.com>
2021-03-19 16:05:11 +01:00
Markus Armbruster
d2032598c4 qapi: Implement deprecated-input=reject for QMP commands
This policy rejects deprecated input, and thus permits "testing the
future".  Implement it for QMP commands: make deprecated ones fail.
Example: when QEMU is run with -compat deprecated-input=reject, then

    {"execute": "query-cpus"}

fails like this

    {"error": {"class": "CommandNotFound", "desc": "Deprecated command query-cpus disabled by policy"}}

When the deprecated command is removed, the error will change to

    {"error": {"class": "CommandNotFound", "desc": "The command query-cpus has not been found"}}

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210318155519.1224118-10-armbru@redhat.com>
2021-03-19 16:05:11 +01:00
Markus Armbruster
a291a38fa1 qapi: Implement deprecated-output=hide for QMP event data
This policy suppresses deprecated bits in output, and thus permits
"testing the future".  Implement it for QMP event data: suppress
deprecated members.

No QMP event data is deprecated right now.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210318155519.1224118-6-armbru@redhat.com>
2021-03-19 15:43:33 +01:00
Markus Armbruster
278fc2f7d3 qapi: Implement deprecated-output=hide for QMP events
This policy suppresses deprecated bits in output, and thus permits
"testing the future".  Implement it for QMP events: suppress
deprecated ones.

No QMP event is deprecated right now.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210318155519.1224118-5-armbru@redhat.com>
2021-03-19 15:43:33 +01:00
Markus Armbruster
91fa93e516 qapi: Implement deprecated-output=hide for QMP command results
This policy suppresses deprecated bits in output, and thus permits
"testing the future".  Implement it for QMP command results.  Example:
when QEMU is run with -compat deprecated-output=hide, then

    {"execute": "query-cpus-fast"}

yields

    {"return": [{"thread-id": 9805, "props": {"core-id": 0, "thread-id": 0, "socket-id": 0}, "qom-path": "/machine/unattached/device[0]", "cpu-index": 0, "target": "x86_64"}]}

instead of

    {"return": [{"arch": "x86", "thread-id": 22436, "props": {"core-id": 0, "thread-id": 0, "socket-id": 0}, "qom-path": "/machine/unattached/device[0]", "cpu-index": 0, "target": "x86_64"}]}

Note the suppression of deprecated member "arch".

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210318155519.1224118-4-armbru@redhat.com>
2021-03-19 15:43:33 +01:00
Peter Maydell
1b507e55f8 Remove many old deprecated features
The following features have been deprecated for well over the 2
 release cycle we promise
 
   ``-drive file=json:{...{'driver':'file'}}`` (since 3.0)
   ``-vnc acl`` (since 4.0.0)
   ``-mon ...,control=readline,pretty=on|off`` (since 4.1)
   ``migrate_set_downtime`` and ``migrate_set_speed`` (since 2.8.0)
   ``query-named-block-nodes`` result ``encryption_key_missing`` (since 2.10.0)
   ``query-block`` result ``inserted.encryption_key_missing`` (since 2.10.0)
   ``migrate-set-cache-size`` and ``query-migrate-cache-size`` (since 2.11.0)
   ``query-named-block-nodes`` and ``query-block`` result dirty-bitmaps[i].status (since 4.0)
   ``query-cpus`` (since 2.12.0)
   ``query-cpus-fast`` ``arch`` output member (since 3.0.0)
   ``query-events`` (since 4.0)
   chardev client socket with ``wait`` option (since 4.0)
   ``acl_show``, ``acl_reset``, ``acl_policy``, ``acl_add``, ``acl_remove`` (since 4.0.0)
   ``ide-drive`` (since 4.2)
   ``scsi-disk`` (since 4.2)
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEE2vOm/bJrYpEtDo4/vobrtBUQT98FAmBTHBsACgkQvobrtBUQ
 T9/6cA//QVlWFvNJjmYuXMFOkYdokqde9W66iMO2vt+p3LSqlcxVa59+ZOcb5XWc
 7xYcZlDHEHRMe8DWrTwUyjftJ7Ild+/HY5Nz8NqDFvX08umIhC6qmzBSG64O3fU5
 KtSLWe+KfbUH2usmczQO1zGG8mSIAMnOdugpLln9W13v7X9NO4yC+/O36Mm8l4VK
 VEF+o9w7+s2l7QloLdhVe7RVOrqz6+9TFHx1+q1pPhMbzlv5vbfp9vIreiCgmqiO
 rPLXB4bjklktWSkJBe/2Jt7ixNQZrT0DfnCmshHJmmBxTCI1+b798Xy+7j9Dz8ER
 LzKHhlsilCv/jb8QKlIZSxvoDE02vAgwyVXlqxc7FdyaLmg/Emwr5LGUFHKCyhDR
 3hmJafNokdWPcHwBYwRTdbG1OlgKA+yHSBzQ4OnjsTsKrrzlBrm9B9DiDVABXadS
 5NnhvjZfaRmzjXZinf1th7fdMtLs4k+9XKn6szkrp2QhtlX9gyhkXuOhdwRVcyJB
 V/8dLZDdOB3Kpbu5DGlmmIzdHQx5bo/tQqZlvJJNmcOWe+TKwHg56WRUYQ3PFQZK
 M2A4XTd5kil6+PT7AnDHLoYAFdfScq6R32V38AI7S34kPrejftbSbDz8EVRXnrzj
 zzXGNWtSQpIDZGgNEg3Ta6aIf28TxLCA7nRhuPYIDcswlUsHn4c=
 =cFRH
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/berrange-gitlab/tags/dep-many-pull-request' into staging

Remove many old deprecated features

The following features have been deprecated for well over the 2
release cycle we promise

  ``-drive file=json:{...{'driver':'file'}}`` (since 3.0)
  ``-vnc acl`` (since 4.0.0)
  ``-mon ...,control=readline,pretty=on|off`` (since 4.1)
  ``migrate_set_downtime`` and ``migrate_set_speed`` (since 2.8.0)
  ``query-named-block-nodes`` result ``encryption_key_missing`` (since 2.10.0)
  ``query-block`` result ``inserted.encryption_key_missing`` (since 2.10.0)
  ``migrate-set-cache-size`` and ``query-migrate-cache-size`` (since 2.11.0)
  ``query-named-block-nodes`` and ``query-block`` result dirty-bitmaps[i].status (since 4.0)
  ``query-cpus`` (since 2.12.0)
  ``query-cpus-fast`` ``arch`` output member (since 3.0.0)
  ``query-events`` (since 4.0)
  chardev client socket with ``wait`` option (since 4.0)
  ``acl_show``, ``acl_reset``, ``acl_policy``, ``acl_add``, ``acl_remove`` (since 4.0.0)
  ``ide-drive`` (since 4.2)
  ``scsi-disk`` (since 4.2)

# gpg: Signature made Thu 18 Mar 2021 09:23:39 GMT
# gpg:                using RSA key DAF3A6FDB26B62912D0E8E3FBE86EBB415104FDF
# gpg: Good signature from "Daniel P. Berrange <dan@berrange.com>" [full]
# gpg:                 aka "Daniel P. Berrange <berrange@redhat.com>" [full]
# Primary key fingerprint: DAF3 A6FD B26B 6291 2D0E  8E3F BE86 EBB4 1510 4FDF

* remotes/berrange-gitlab/tags/dep-many-pull-request:
  block: remove support for using "file" driver with block/char devices
  block: remove 'dirty-bitmaps' field from 'BlockInfo' struct
  block: remove dirty bitmaps 'status' field
  block: remove 'encryption_key_missing' flag from QAPI
  hw/scsi: remove 'scsi-disk' device
  hw/ide: remove 'ide-drive' device
  chardev: reject use of 'wait' flag for socket client chardevs
  machine: remove 'arch' field from 'query-cpus-fast' QMP command
  machine: remove 'query-cpus' QMP command
  migrate: remove QMP/HMP commands for speed, downtime and cache size
  monitor: remove 'query-events' QMP command
  monitor: raise error when 'pretty' option is used with HMP
  ui, monitor: remove deprecated VNC ACL option and HMP commands

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-03-18 19:00:50 +00:00
Daniel P. Berrangé
879be3af49 hw/scsi: remove 'scsi-disk' device
The 'scsi-hd' and 'scsi-cd' devices provide suitable alternatives.

Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2021-03-18 09:22:55 +00:00
Daniel P. Berrangé
b501018339 hw/ide: remove 'ide-drive' device
The 'ide-hd' and 'ide-cd' devices provide suitable alternatives.

Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2021-03-18 09:22:55 +00:00
Alexander Bulekov
97ef5f8862 fuzz: add a script to build reproducers
Currently, bash and C crash reproducers are be built manually. This is a
problem, as we want to integrate reproducers into the tree, for
regression testing. This patch adds a script that converts a sequence of
QTest commands into a pasteable Bash reproducer, or a libqtest-based C
program. This will try to wrap pasteable reproducers to 72 chars, but
the generated C code will not have nice formatting. Therefore, the C
output of this script should be piped through an auto-formatter, such as
clang-format

Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
Reviewed-by: Darren Kenny <darren.kenny@oracle.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-03-16 14:19:54 -04:00
Cornelia Huck
3aa1b7af0f pvrdma: wean code off pvrdma_ring.h kernel header
The pvrdma code relies on the pvrdma_ring.h kernel header for some
basic ring buffer handling. The content of that header isn't very
exciting, but contains some (q)atomic_*() invocations that (a)
cause manual massaging when doing a headers update, and (b) are
an indication that we probably should not be importing that header
at all.

Let's reimplement the ring buffer handling directly in the pvrdma
code instead. This arguably also improves readability of the code.

Importing the header can now be dropped.

Signed-off-by: Cornelia Huck <cohuck@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Yuval Shaia <yuval.shaia.ml@gmail.com>
Tested-by: Yuval Shaia <yuval.shaia.ml@gmail.com>
Signed-off-by: Jason Wang <jasowang@redhat.com>
2021-03-15 16:41:22 +08:00
Laurent Vivier
6e1c0d7b95 linux-user: manage binfmt-misc preserve-arg[0] flag
Add --preserve-argv0 in qemu-binfmt-conf.sh to configure the preserve-argv0
flag.

This patch allows to use new flag in AT_FLAGS to detect if
preserve-argv0 is configured for this interpreter:
argv[0] (the full pathname provided by binfmt-misc) is removed and
replaced by argv[1] (the original argv[0] provided by binfmt-misc when
'P'/preserve-arg[0] is set)

For instance with this patch and kernel support for AT_FLAGS:

  $ sudo chroot m68k-chroot sh -c 'echo $0'
  sh

without this patch:

  $ sudo chroot m68k-chroot sh -c 'echo $0'
  /usr/bin/sh

The new flag is available in kernel (v5.12) since:
2347961b11d4 ("binfmt_misc: pass binfmt_misc flags to the interpreter")

This can be tested with something like:

  # cp ..../qemu-ppc /chroot/powerpc/jessie

  # qemu-binfmt-conf.sh --qemu-path / --systemd ppc --credential yes \
                        --persistent no --preserve-argv0 yes
  # systemctl restart systemd-binfmt.service
  # cat /proc/sys/fs/binfmt_misc/qemu-ppc
  enabled
  interpreter //qemu-ppc
  flags: POC
  offset 0
  magic 7f454c4601020100000000000000000000020014
  mask ffffffffffffff00fffffffffffffffffffeffff
  # chroot /chroot/powerpc/jessie  sh -c 'echo $0'
  sh

  # qemu-binfmt-conf.sh --qemu-path / --systemd ppc --credential yes \
                        --persistent no --preserve-argv0 no
  # systemctl restart systemd-binfmt.service
  # cat /proc/sys/fs/binfmt_misc/qemu-ppc
  enabled
  interpreter //qemu-ppc
  flags: OC
  offset 0
  magic 7f454c4601020100000000000000000000020014
  mask ffffffffffffff00fffffffffffffffffffeffff
  # chroot /chroot/powerpc/jessie  sh -c 'echo $0'
  /bin/sh

Signed-off-by: Laurent Vivier <laurent@vivier.eu>
Message-Id: <20210222105004.1642234-1-laurent@vivier.eu>
2021-03-13 10:45:11 +01:00
Peter Maydell
6f34661b6c Pull request
-----BEGIN PGP SIGNATURE-----
 
 iQJGBAABCAAwFiEEzS913cjjpNwuT1Fz8ww4vT8vvjwFAmBJQHkSHGxhdXJlbnRA
 dml2aWVyLmV1AAoJEPMMOL0/L748EdsP/2U2CGTM95tjDunTs9uZV/7zM6PWt85M
 vAPItNVU2jYPfzmaJN8twrzlj0PEDhvB9Q+OJjE4HEGxEbPcdblLg/R6Zs/EaWuY
 N6oKHPXnOnHb+e80UUJdiAq+Y5RUnJbb5L3ArycnVzBgws+Oj3DtqjB2VDccY4C/
 Gkt23tZ7ikU4958e5VBqW2NUUrr+BQO0mqsW+sbbeE3WPj75NQc6srvS3TWvsg7W
 OYEyVYwm52/q2W/1a3Knfv/YO6UU9NGMpGyDLD2kwQwKbgUWYLW2BiWVwOAUldo9
 De3nfKbKnFezLCZAZro20lfCa/aKwNGCOXWzlrKxqUQCmGYUx7gM1+3ahrSd5N0v
 zUgLdZm7O428ZHL6GujWGLA1UwwzpM9X3P3yo4c0S1J6fHypbI6a9jtewrUFvFgP
 TuQ7dp6cn2DTBYUcsrWilPHbTZMADYQNRD/xUtKqalYBEWy3FX5W75+OYBJKKh+X
 Qip68m6JBzgkszXhCcu6xlLb8ynZJr2VsHvtvIgf4NnLqNOIEgVLcMtoMZT8DPrp
 rIoRc5oUFz8zj5lHnJuLADBUvlCMqoCCoU3h2aqHwH8a7RGb180f+82BW9aBcb2u
 Jk+WgAhBUjWBBC97ReFgrINUD/qZRXVoOq8LthTuQSSyr/i1zq+oLM1F0EDXcMDm
 ssATku2IxL24
 =moUF
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/vivier2/tags/trivial-branch-for-6.0-pull-request' into staging

Pull request

# gpg: Signature made Wed 10 Mar 2021 21:56:09 GMT
# gpg:                using RSA key CD2F75DDC8E3A4DC2E4F5173F30C38BD3F2FBE3C
# gpg:                issuer "laurent@vivier.eu"
# gpg: Good signature from "Laurent Vivier <lvivier@redhat.com>" [full]
# gpg:                 aka "Laurent Vivier <laurent@vivier.eu>" [full]
# gpg:                 aka "Laurent Vivier (Red Hat) <lvivier@redhat.com>" [full]
# Primary key fingerprint: CD2F 75DD C8E3 A4DC 2E4F  5173 F30C 38BD 3F2F BE3C

* remotes/vivier2/tags/trivial-branch-for-6.0-pull-request: (22 commits)
  sysemu: Let VMChangeStateHandler take boolean 'running' argument
  sysemu/runstate: Let runstate_is_running() return bool
  hw/lm32/Kconfig: Have MILKYMIST select LM32_DEVICES
  hw/lm32/Kconfig: Rename CONFIG_LM32 -> CONFIG_LM32_DEVICES
  hw/lm32/Kconfig: Introduce CONFIG_LM32_EVR for lm32-evr/uclinux boards
  qemu-common.h: Update copyright string to 2021
  tests/fp/fp-test: Replace the word 'blacklist'
  qemu-options: Replace the word 'blacklist'
  seccomp: Replace the word 'blacklist'
  scripts/tracetool: Replace the word 'whitelist'
  ui: Replace the word 'whitelist'
  virtio-gpu: Adjust code space style
  exec/memory: Use struct Object typedef
  fuzz-test: remove unneccessary debugging flags
  net: Use id_generate() in the network subsystem, too
  MAINTAINERS: Fix the location of tools manuals
  vhost_user_gpu: Drop dead check for g_malloc() failure
  backends/dbus-vmstate: Fix short read error handling
  target/hexagon/gen_tcg_funcs: Fix a typo
  hw/elf_ops: Fix a typo
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-03-11 18:55:27 +00:00
Philippe Mathieu-Daudé
54fa79b793 scripts/tracetool: Replace the word 'whitelist'
Follow the inclusive terminology from the "Conscious Language in your
Open Source Projects" guidelines [*] and replace the words "whitelist"
appropriately.

[*] https://github.com/conscious-lang/conscious-lang-docs/blob/main/faq.md

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20210303184644.1639691-3-philmd@redhat.com>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
2021-03-09 22:14:26 +01:00
Cleber Rosa
6179f32eeb scripts/ci/gitlab-pipeline-status: give more info when pipeline not found
This includes both input parameters (project id and commit) in the
message so to make it easier to debug returned API calls.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20210222193240.921250-4-crosa@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-03-09 06:03:53 +01:00
Cleber Rosa
861d1d509b scripts/ci/gitlab-pipeline-status: give more information on failures
When an HTTP GET request fails, it's useful to go beyond the "not
successful" message, and show the code returned by the server.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20210222193240.921250-3-crosa@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-03-09 06:03:53 +01:00
Cleber Rosa
2faf56bd95 scripts/ci/gitlab-pipeline-status: split utlity function for HTTP GET
This simply splits out the code that does an HTTP GET so that it
can be used for other API requests.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20210222193240.921250-2-crosa@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-03-09 06:03:53 +01:00
Thomas Huth
c2f4c1a8ba meson: Re-enable the possibility to run "make check SPEED=slow"
"make check SPEED=slow" got lost in the conversion of the build
system to meson - the tests were always running in "quick" mode.
Fix it by passing the "-m" parameter to the test harness at the
right spot in scripts/mtest2make.py.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20210218172313.2217440-1-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-03-09 06:03:53 +01:00
Markus Armbruster
0e92a19b8c qapi: Fix parse errors for removal of null from schema language
Commit 9d55380b5a "qapi: Remove null from schema language" (v4.2.0)
neglected to update two error messages.  Do that now.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210224101442.1837475-1-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
2021-03-05 15:40:50 +01:00
Peter Maydell
07dbfdd290 * fix --enable-fuzzing linker failures (Alexander)
* target/i386: Add bus lock debug exception support (Chenyi)
 * update documentation for preferred boolean option syntax (Daniel)
 * make SCSI io_timeout configurable (Hannes)
 * fix handling of guest recoverable SCSI errors (myself)
 * misc fixes (Pavel, Zheng Zhan Liang, Zihao)
 * fix installation of binaries with entitlements (Akihiko)
 -----BEGIN PGP SIGNATURE-----
 
 iQFIBAABCAAyFiEE8TM4V0tmI4mGbHaCv/vSX3jHroMFAmA3tzQUHHBib256aW5p
 QHJlZGhhdC5jb20ACgkQv/vSX3jHroMsngf+OkbgTDouxq3qnn1K89n6j2wKlAj0
 qPP+57o1sv5u5MjX0BkawXCMJfvTKJaOt/Cs8S0FaPTehfR5H0T2tL76KZBEdlEP
 /+V3EC0Qmd1S47LKIOuDkV15szvnccyjpgwaL7Osjb0Eh0SxeXd4pBTc3yYtkacu
 zb/srZ63U2iJZr93QwGmEJjuW57WNym6QWXF3Mrg9PFXybkXFTGumB2vZMlQc+MN
 RWPTUKZKAiIr/HfdvpIETXY+HzJhylvIQ8OVWtM3XDorl48OWcOat/jyeNEVYgG3
 o+gxIRnfvIL+42GkmUFbOKLzT5SV6p9EYYGLTuE1kU7nc6HWoD0CdPxA3A==
 =UBVO
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/bonzini-gitlab/tags/for-upstream' into staging

* fix --enable-fuzzing linker failures (Alexander)
* target/i386: Add bus lock debug exception support (Chenyi)
* update documentation for preferred boolean option syntax (Daniel)
* make SCSI io_timeout configurable (Hannes)
* fix handling of guest recoverable SCSI errors (myself)
* misc fixes (Pavel, Zheng Zhan Liang, Zihao)
* fix installation of binaries with entitlements (Akihiko)

# gpg: Signature made Thu 25 Feb 2021 14:41:56 GMT
# gpg:                using RSA key F13338574B662389866C7682BFFBD25F78C7AE83
# gpg:                issuer "pbonzini@redhat.com"
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>" [full]
# gpg:                 aka "Paolo Bonzini <pbonzini@redhat.com>" [full]
# Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4  E2F7 7E15 100C CD36 69B1
#      Subkey fingerprint: F133 3857 4B66 2389 866C  7682 BFFB D25F 78C7 AE83

* remotes/bonzini-gitlab/tags/for-upstream: (29 commits)
  tcg/i386: rdpmc: fix the the condtions
  chardev: do not use short form boolean options in non-QemuOpts character device descriptions
  vl: deprecate -writeconfig
  target/i386: Add bus lock debug exception support
  qom/object.c: Fix typo
  target/i386: update to show preferred boolean syntax for -cpu
  docs: update to show preferred boolean syntax for -cpu
  docs: update to show preferred boolean syntax for -vnc
  docs: update to show preferred boolean syntax for -chardev
  qemu-options: update to show preferred boolean syntax for -vnc
  qemu-options: update to show preferred boolean syntax for -incoming
  qemu-options: update to show preferred boolean syntax for -netdev
  qemu-options: update to show preferred boolean syntax for -spice
  qemu-options: update to show preferred boolean syntax for -chardev
  gdbstub: use preferred boolean option syntax
  char: don't fail when client is not connected
  scsi: drop 'result' argument from command_complete callback
  scsi-disk: pass guest recoverable errors through even for rerror=stop
  scsi-disk: pass SCSI status to scsi_handle_rw_error
  scsi: introduce scsi_sense_from_errno()
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-03-02 15:17:45 +00:00
Peter Maydell
cbcf09872a Testing tweaks (build, docs, bumps)
- expose cross compiler info in meson pretty print
   - bump Fedora to 33
   - "graceful" handling of missing virgl config
   - updates to the container documentation
   - move CODING_STYLE.rst into developer manual
 -----BEGIN PGP SIGNATURE-----
 
 iQEzBAABCgAdFiEEZoWumedRZ7yvyN81+9DbCVqeKkQFAmA2M5MACgkQ+9DbCVqe
 KkT3FAf/VqJMFin2ON8GxD/0OvaPhX2zOMrPzzUioMcHZKTW2AFytY0y4JPp7bcg
 rCqbHHa2escODQk0p1Wf5VvNYSjC9Q2PA0tmGXIZPYt+8Xf2N5l9Ub3woDBO6uKD
 cEY590wWzrcLoaSEyTcqEUKFD5Ce9PO/WAf/dlZgYr3Mo1flX17wBNvrMi4iQrld
 mLajqx3x4vonnda4nrAiz7fD/CLwE/50+21+0xN2vY3YA7EjRp8UfkUBIwe+Fubn
 iu0FmGgHn7coNSHSD8sfYKHeuo/fMJl6jKiWJdFC7343WdAQtsnYF7Uhml+T6X7i
 cm8yHCaBnbpHEw0x7kTsxfP/1ixYVw==
 =hyyV
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/stsquad/tags/pull-testing-updates-240221-1' into staging

Testing tweaks (build, docs, bumps)

  - expose cross compiler info in meson pretty print
  - bump Fedora to 33
  - "graceful" handling of missing virgl config
  - updates to the container documentation
  - move CODING_STYLE.rst into developer manual

# gpg: Signature made Wed 24 Feb 2021 11:08:03 GMT
# gpg:                using RSA key 6685AE99E75167BCAFC8DF35FBD0DB095A9E2A44
# gpg: Good signature from "Alex Bennée (Master Work Key) <alex.bennee@linaro.org>" [full]
# Primary key fingerprint: 6685 AE99 E751 67BC AFC8  DF35 FBD0 DB09 5A9E 2A44

* remotes/stsquad/tags/pull-testing-updates-240221-1:
  docs: move CODING_STYLE into the developer documentation
  docs/devel: add forward reference to check-tcg
  docs/devel: update the container based tests
  docs/devel: expand on use of containers to build tests
  tests/acceptance: allow a "graceful" failing for virtio-gpu test
  docker: Bump Fedora images to release 33
  meson.build: expose TCG cross compiler information in summary

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-02-25 19:07:58 +00:00
Daniel P. Berrangé
c23874132b docs: update to show preferred boolean syntax for -chardev
The preferred syntax is to use "foo=on|off", rather than a bare
"foo" or "nofoo".

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20210216191027.595031-8-berrange@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-02-25 14:14:33 +01:00
Akihiko Odaki
237377ac72 hvf: Sign the code after installation
Before this change, the code signed during the build was installed
directly.

However, the signature gets invalidated because meson modifies the code
to fix dynamic library install names during the install process.

It also prevents meson to strip the code because the pre-signed file is
not marked as an executable (although it is somehow able to perform the
modification described above).

With this change, the unsigned code will be installed and modified by
meson first, and a script signs it later.

Signed-off-by: Akihiko Odaki <akihiko.odaki@gmail.com>
Message-Id: <20210225000614.46919-1-akihiko.odaki@gmail.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-02-25 13:57:34 +01:00
Alex Bennée
93a1100768 docs: move CODING_STYLE into the developer documentation
There is no particular reason to keep this on it's own in the root of
the tree. Move it into the rest of the fine developer manual and fixup
any links to it. The only tweak I've made is to fix the code-block
annotations to mention the language C.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Claudio Fontana <cfontana@suse.de>
Message-Id: <20210223095931.16908-1-alex.bennee@linaro.org>
2021-02-24 11:05:21 +00:00
Isaku Yamahata
d2f1af0e41 checkpatch: don't emit warning on newly created acpi data files
Newly created acpi data files(tests/data/acpi/) cause false positive
warning.
If file names are acpi expected file, don't emit warning.

Fixes: e625ba2a41 ("checkpatch: fix acpi check with multiple file name")
Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
Message-Id: <6899f9ad54cab8e7deca94ff0eeab641680e2b5e.1613615732.git.isaku.yamahata@intel.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2021-02-23 10:58:42 -05:00
Peter Maydell
e90ef02389 QAPI patches patches for 2021-02-18
-----BEGIN PGP SIGNATURE-----
 
 iQJGBAABCAAwFiEENUvIs9frKmtoZ05fOHC0AOuRhlMFAmAutzcSHGFybWJydUBy
 ZWRoYXQuY29tAAoJEDhwtADrkYZTvBsP/jI3Dh9j8HLLsvauU117LcIGnn5Qwog4
 8v0IrEieaOeMDNDzR8AgSqBLYm0ZQcBoZl7Qkv8XMxYa7iWiJv2uCShRS7MNl5NK
 M8IW0XWVjCJJ7TfFGj5iP/zlnhBMTzp66EDfkdiBWlig7NdZ+TBVp+WiZr8mT7EJ
 MdI2slzmgChDHJ2BLntLDt3w778GUIIXYxNFAhH4woAs6Xq7Z8tq3y+7c9JMXL/1
 B1esVN49h582mcldT5LcMK1AWOjxon+IaKS5KloMcr/uVCqaineSR/rPTXI4k9RL
 KfKeetniOz0ohyD+4dfLYW7OPhjJ3ZiUm6tSP6aVcg8D8qJAEvocw1uZS3iFpR5T
 lqdS6rcaumx3FaPix3MmhbNJLytohiAvvJyHpjLQ32ByhJ2FPzUOLFyRvoeoE+kD
 wJSTLn0iyjIlY3bz4sdA3STWK9TyKJ8enyRWGGZECL0ZJwCUe/Z7KwxZi7Ad4vLl
 j/os65T+XudTEJzk4iKpA03ThvAs82q34mpSkpACfDA9VZ584fRC79WbQoVAGmrk
 3CV4vfBS3pcPz7nCQqauTGWg3iqQT4iKYGdw9YTqA88kIk/swfPiucGrAwI1WPXz
 VReMbz/LBBa8gLb2Sx4MMlRjO/v/WlOyxJ1sfdETmiX+ZjUl8s7+94PwlWI2kT6k
 3r9bl3Tz+QlZ
 =pIh0
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/armbru/tags/pull-qapi-2021-02-18' into staging

QAPI patches patches for 2021-02-18

# gpg: Signature made Thu 18 Feb 2021 18:51:35 GMT
# gpg:                using RSA key 354BC8B3D7EB2A6B68674E5F3870B400EB918653
# gpg:                issuer "armbru@redhat.com"
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>" [full]
# gpg:                 aka "Markus Armbruster <armbru@pond.sub.org>" [full]
# Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867  4E5F 3870 B400 EB91 8653

* remotes/armbru/tags/pull-qapi-2021-02-18:
  qapi/introspect.py: set _gen_tree's default ifcond argument to ()
  qapi/introspect.py: Type _gen_tree variants as Sequence[str]
  qapi/introspect.py: Update copyright and authors list
  qapi/introspect.py: Add docstrings to _gen_tree and _tree_to_qlit
  qapi/introspect.py: add type hint annotations
  qapi/introspect.py: remove _gen_variants helper
  qapi/introspect.py: improve readability of _tree_to_qlit
  qapi/introspect.py: improve _tree_to_qlit error message
  qapi/introspect.py: create a typed 'Annotated' data strutcure
  qapi/introspect.py: Introduce preliminary tree typing
  qapi/introspect.py: Always define all 'extra' dict keys
  qapi/introspect.py: replace 'extra' dict with 'comment' argument
  qapi/introspect.py: Unify return type of _make_tree()
  qapi/introspect.py: guard against ifcond/comment misuse
  qapi/introspect.py: add _gen_features helper
  qapi/introspect.py: use _make_tree for features nodes
  qapi/introspect.py: assert schema is not None
  qapi: Replace List[str] with Sequence[str] for ifcond

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-02-19 17:22:42 +00:00
Peter Maydell
ce42fe17ad * Always build the container images in the gitlab-CI
* Some other small gitlab-CI improvements
 * Some qtest fixes
 -----BEGIN PGP SIGNATURE-----
 
 iQJFBAABCAAvFiEEJ7iIR+7gJQEY8+q5LtnXdP5wLbUFAmAvVkwRHHRodXRoQHJl
 ZGhhdC5jb20ACgkQLtnXdP5wLbX6JRAAmpDk8N3JP8XxG4ZVBbAquIRr87CgVykO
 Si4bQtoy6UGcht1aFM2QnfoU54lxVBArSjbqsV1soqHjC5mLl1t7FaHS98hOx7t4
 4x3oejxyiXhh9U+6AEzuSAlXLXYUs4eqpQ7f7AHrhqnj/pLea/dQFzal6TT2A8qf
 wYhUyPTVnVKrVPR5b0wlHtLJgpNa4A1McCDsD68NY/5uKTlgfteNln1XKBMJwokL
 LoZm8iFvLEdzPL0ikslrUmbbaYJvKiedSy9ZEYbuFllY2aVpkKPuN49ZWh35IJSA
 GkiXpbirPMIjydGrq68qWd/9p4i4ltX9LfKgpcDua1NptbK2LvYe7FaeyPBYoSFP
 wsXP497vwt9g3llTzbrEHX5ZeAivW/JqemX60pF61p0Lh1UnBA+30gHQVwYgvQZa
 HUCeNMrK3dMljfHOBrWLX84lfzBi3BUQhYVzyx1kTcOAEotgxnuKH7adWAvqXlpE
 x2Ff4hDQj+xL2mbbHWNGRHBS33SNNIKIvH+kMtqzzHteDp6jROw83+HlNMw38Zl5
 s0CwiA5Wk2vAW7gFZ0FTan+a9TxpotHVgRZQZ8TLk15UTFMWoqQfBz6OdsPz38MG
 5PjouX6K4ZR9uHZiOggsjN5XZkmJro0S3Lb9mWVCXLZp0XF0e9lCw/xXiUznsXu2
 QZXcSCmiMkw=
 =SDf2
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/huth-gitlab/tags/pull-request-2021-02-19' into staging

* Always build the container images in the gitlab-CI
* Some other small gitlab-CI improvements
* Some qtest fixes

# gpg: Signature made Fri 19 Feb 2021 06:10:20 GMT
# gpg:                using RSA key 27B88847EEE0250118F3EAB92ED9D774FE702DB5
# gpg:                issuer "thuth@redhat.com"
# gpg: Good signature from "Thomas Huth <th.huth@gmx.de>" [full]
# gpg:                 aka "Thomas Huth <thuth@redhat.com>" [full]
# gpg:                 aka "Thomas Huth <huth@tuxfamily.org>" [full]
# gpg:                 aka "Thomas Huth <th.huth@posteo.de>" [unknown]
# Primary key fingerprint: 27B8 8847 EEE0 2501 18F3  EAB9 2ED9 D774 FE70 2DB5

* remotes/huth-gitlab/tags/pull-request-2021-02-19:
  travis.yml: Limit simultaneous jobs to 3
  gitlab-ci.yml: Run check-tcg with TCI
  tests/qtest/boot-sector: Check that the guest did not panic
  gitlab-ci: Disable vhost-kernel in build-disable job
  scripts/checkpatch: Improve the check for authors mangled by the mailing list
  gitlab-ci: Display Avocado log content when tests timeout
  gitlab: fix inconsistent indentation
  gitlab: add fine grained job deps for all build jobs
  gitlab: always build container images
  tests/qtest/boot-serial-test: Test Virt machine with 'max'

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-02-19 14:54:52 +00:00
Thomas Huth
7b59d10e23 scripts/checkpatch: Improve the check for authors mangled by the mailing list
There were recently some patches on the list which had their "From:"
line mangled like this:

 From: qemu_oss--- via <qemu-devel@nongnu.org>

Since our test in the checkpatch.pl script did not trigger here, the
patches finally also ended up in a pull request, with the wrong author
set. So let's improve the regular expression to also complain on
these new patterns, too.

Message-Id: <20210216071512.1199827-1-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Christian Schoenebeck <qemu_oss@crudebyte.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-02-19 06:29:05 +01:00
Eduardo Habkost
1a14d4e16a device-crash-test: Remove problematic language
Replace "whitelist" in the device-crash-test script with
"rule list".

I'm using "rule list" instead of "allow list" or "pass list"
because the list is not used only for expected/allowed errors.
It also contain rules specifying which errors shouldn't be
ignored and/or should be fatal.

Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Acked-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20210202191207.4103973-1-ehabkost@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2021-02-18 18:34:47 -05:00
John Snow
9b77d94699 qapi/introspect.py: set _gen_tree's default ifcond argument to ()
We don't need to create an empty, mutable list to pass to _gen_tree;
since it is now typed as a Sequence, we can use the empty tuple as a
default and omit the argument.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-19-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:51:14 +01:00
John Snow
cea53c31b7 qapi/introspect.py: Type _gen_tree variants as Sequence[str]
Optional[List] is clunky; an empty sequence can more elegantly convey
"no variants". By downgrading "List" to "Sequence", we can also accept
tuples; this is useful for the empty tuple specifically, which we may
use as a default parameter because it is immutable.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-18-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Doc string touched up]
Reviewed-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:51:08 +01:00
John Snow
cf26906c4a qapi/introspect.py: Update copyright and authors list
To reflect the work that went into strictly typing introspect.py,
punish myself by claiming credit.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-17-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:50:26 +01:00
John Snow
5444dedfc7 qapi/introspect.py: Add docstrings to _gen_tree and _tree_to_qlit
Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-16-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Doc string improvements squashed in]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:50:23 +01:00
John Snow
82b52f6b84 qapi/introspect.py: add type hint annotations
NB: The type aliases (SchemaInfo et al) declare intent for some of the
"dictly-typed" objects we pass around in introspect.py. They do not
enforce the shape of those objects, and cannot, until Python 3.7 or
later. (And even then, it may not be "worth it".)

Annotations are also added to the QAPISchemaEntity __init__ method in
schema.py to allow mypy to statically prove the type of typ.name,
needed to prove the return type of
QAPISchemaGenIntrospectVisitor._use_type().

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-15-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Note on QAPISchemaEntity.__init__() squashed into commit message,
Comment wrapped to conform to PEP 8]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:45:20 +01:00
John Snow
cf5db2142b qapi/introspect.py: remove _gen_variants helper
It is easier to give a name to all of the dictly-typed objects we pass
around in introspect.py by removing this helper, as it does not return
an object that has any knowable type by itself.

Inline it into its only caller instead.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-14-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:45:20 +01:00
John Snow
c0e8d9f3c1 qapi/introspect.py: improve readability of _tree_to_qlit
Subjective, but I find getting rid of the comprehensions helps. Also,
divide the sections into scalar and non-scalar sections, and remove
old-style string formatting.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-13-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:45:20 +01:00
John Snow
2a6c161bbf qapi/introspect.py: improve _tree_to_qlit error message
Trivial; make the error message just a pinch more explicit in case we
trip this by accident in the future.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-12-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:45:20 +01:00
John Snow
4f7f97a7b3 qapi/introspect.py: create a typed 'Annotated' data strutcure
Presently, we use a tuple to attach a dict containing annotations
(comments and compile-time conditionals) to a tree node. This is
undesirable because dicts are difficult to strongly type; promoting it
to a real class allows us to name the values and types of the
annotations we are expecting.

In terms of typing, the Annotated<T> type serves as a generic container
where the annotated node's type is preserved, allowing for greater
specificity than we'd be able to provide without a generic.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-11-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:45:17 +01:00
John Snow
9db2734613 qapi/introspect.py: Introduce preliminary tree typing
The types will be used in forthcoming patches to add typing. These types
describe the layout and structure of the objects passed to
_tree_to_qlit, but lack the power to describe annotations until the next
commit.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-10-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:38:00 +01:00
John Snow
d4c5b429fd qapi/introspect.py: Always define all 'extra' dict keys
This mimics how a typed object works, where 'if' and 'comment' are
always set, regardless of if they have a value set or not.

It is safe to do this because of the way that _tree_to_qlit processes
these values (using dict.get with a default of None), resulting in no
change of output from _tree_to_qlit. There are no other users of this
data.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-9-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:37:54 +01:00
John Snow
5f50cede4a qapi/introspect.py: replace 'extra' dict with 'comment' argument
This is only used to pass in a dictionary with a comment already set, so
skip the runaround and just accept the (optional) comment.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-8-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:37:49 +01:00
John Snow
84cf09915f qapi/introspect.py: Unify return type of _make_tree()
Returning two different types conditionally can be complicated to
type. Return one type for consistency.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-7-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:37:41 +01:00
John Snow
055569603d qapi/introspect.py: guard against ifcond/comment misuse
_tree_to_qlit is called recursively on dict values (isolated from their
keys); at such a point in generating output it is too late to apply an
ifcond. Similarly, comments do not necessarily have a "tidy" place they
can be printed in such a circumstance.

Forbid this usage by renaming "suppress_first_indent" to "dict_value" to
emphasize that indents are suppressed only for the benefit of dict
values; then add an assertion assuring we do not pass ifcond/comments
in this case.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-6-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Comment wrapped to conform to PEP 8]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:37:28 +01:00
John Snow
84bece7dd4 qapi/introspect.py: add _gen_features helper
_make_tree might receive a dict (a SchemaInfo object) or some other type
(usually, a string) for its obj parameter. Adding features information
should arguably be performed by the caller at such a time when we know
the type of the object and don't have to re-interrogate it.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-5-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:37:20 +01:00
John Snow
d70f5130f6 qapi/introspect.py: use _make_tree for features nodes
At present, we open-code this in _make_tree itself; but if the structure
of the tree changes, this is brittle. Use an explicit recursive call to
_make_tree when appropriate to help keep the interior node typing
consistent.

A consequence of doing this is that the 'ifcond' key of the features
dict will be omitted when ifcond is false-ish, just like it is omitted
in top-level calls to _make_tree. This also increases consistency in our
handling of this property.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-4-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:37:13 +01:00
John Snow
6b67bcac0f qapi/introspect.py: assert schema is not None
The introspect visitor is stateful, but expects that it will have a
schema to refer to. Add assertions that state this.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-3-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:35:11 +01:00
Taylor Simpson
3e7a84eecc Hexagon build infrastructure
Add file to default-configs
Add hexagon to meson.build
Add hexagon to target/meson.build
Add target/hexagon/meson.build
Change scripts/qemu-binfmt-conf.sh

We can build a hexagon-linux-user target and run programs on
the Hexagon scalar core.  With hexagon-linux-clang installed,
"make check-tcg" will pass.

Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Taylor Simpson <tsimpson@quicinc.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <1612763186-18161-35-git-send-email-tsimpson@quicinc.com>
[rth: Use top-level python variable]
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-02-18 08:25:06 -08:00
John Snow
2184bca7b1 qapi: Replace List[str] with Sequence[str] for ifcond
It does happen to be a list (as of now), but we can describe it in more
general terms with no loss in accuracy to allow tuples and other
constructs.

In the future, we can write "ifcond: Sequence[str] = ()" as a default
parameter, which we could not do safely with a Mutable type like a List.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-2-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Commit message tweaked]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 17:10:29 +01:00
Taylor Simpson
d2a56bd242 Hexagon (linux-user/hexagon) Linux user emulation
Implementation of Linux user emulation for Hexagon
Some common files modified in addition to new files in linux-user/hexagon

Acked-by: Laurent Vivier <laurent@vivier.eu>
Signed-off-by: Taylor Simpson <tsimpson@quicinc.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <1612763186-18161-31-git-send-email-tsimpson@quicinc.com>
[rth: Fix termbits.h on review by Laurent]
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-02-18 07:48:22 -08:00
Philippe Mathieu-Daudé
04250c69bc travis.yml: Move gprof/gcov test across to gitlab
Similarly to commit 8cdb2cef3f, move the gprof/gcov test to GitLab.

The coverage-summary.sh script is not Travis-CI specific, make it
generic.

[thuth: Add gcovr and bsdmainutils which are required for the
        coverage-summary.sh script to the ubuntu docker file,
        and use 'check' as test target]

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20201108204535.2319870-10-philmd@redhat.com>
Message-Id: <20210211045455.456371-2-thuth@redhat.com>
Message-Id: <20210211122750.22645-2-alex.bennee@linaro.org>
2021-02-15 09:38:19 +00:00
Stefan Hajnoczi
7a53cd20e7 get_maintainer: update repo URL to GitLab
qemu.org is running out of bandwidth and the QEMU project is moving
towards a gating CI on GitLab. Use the GitLab repos instead of qemu.org
(they will become mirrors).

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-id: 20210111115017.156802-7-stefanha@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2021-02-09 20:53:56 +00:00
Peter Maydell
41d306ec7d * Fuzzing improvements (Qiuhao, Alexander)
* i386: Fix BMI decoding for instructions with the 0x66 prefix (David)
 * initial attempt at fixing event_notifier emulation (Maxim)
 * i386: PKS emulation, fix for "qemu-system-i386 -cpu host" (myself)
 * meson: RBD test fixes (myself)
 * meson: TCI warnings (Philippe)
 * Leaner build for --disable-guest-agent, --disable-system and
   --disable-tools (Philippe, Stefan)
 * --enable-tcg-interpreter fix (Richard)
 * i386: SVM feature bits (Wei)
 * KVM bugfix (Thomas H.)
 * Add missing MemoryRegionOps callbacks (PJP)
 -----BEGIN PGP SIGNATURE-----
 
 iQFIBAABCAAyFiEE8TM4V0tmI4mGbHaCv/vSX3jHroMFAmAhR4cUHHBib256aW5p
 QHJlZGhhdC5jb20ACgkQv/vSX3jHroOviAf/Ymk/KwHZKySVqHOjDZnvP5PXMAru
 p1zlMRLAorXK+CTbshkIliaQyD8ggzT4HCinJ2NisdfTWMmlWbgr8gahNqKyZ5UG
 HlL28va3dvGhelswh/CNso1ZhVb2Q+aAYn/c6LXQva2r0xi26ohJTkIkSCPP/bnI
 +73dGzwAilBOsBVbn4cCm/70XtwDpPkw41IZIDoy/4lhL8ZdpHMz8oOjNIlOdlcU
 aEDfM8vYE4C70OtUlRZ1OwVxzcjS1Bf6dQYcpg5gAKy/jAAqR+v2PStxXiUuj5D3
 cAzd03Goh78Wcre+CbWxDKGcGtiooUT+J09wmvDPYVUHcpQMbumf4MufrQ==
 =INB5
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/bonzini-gitlab/tags/for-upstream' into staging

* Fuzzing improvements (Qiuhao, Alexander)
* i386: Fix BMI decoding for instructions with the 0x66 prefix (David)
* initial attempt at fixing event_notifier emulation (Maxim)
* i386: PKS emulation, fix for "qemu-system-i386 -cpu host" (myself)
* meson: RBD test fixes (myself)
* meson: TCI warnings (Philippe)
* Leaner build for --disable-guest-agent, --disable-system and
  --disable-tools (Philippe, Stefan)
* --enable-tcg-interpreter fix (Richard)
* i386: SVM feature bits (Wei)
* KVM bugfix (Thomas H.)
* Add missing MemoryRegionOps callbacks (PJP)

# gpg: Signature made Mon 08 Feb 2021 14:15:35 GMT
# gpg:                using RSA key F13338574B662389866C7682BFFBD25F78C7AE83
# gpg:                issuer "pbonzini@redhat.com"
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>" [full]
# gpg:                 aka "Paolo Bonzini <pbonzini@redhat.com>" [full]
# Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4  E2F7 7E15 100C CD36 69B1
#      Subkey fingerprint: F133 3857 4B66 2389 866C  7682 BFFB D25F 78C7 AE83

* remotes/bonzini-gitlab/tags/for-upstream: (46 commits)
  target/i386: Expose VMX entry/exit load pkrs control bits
  target/i386: Add support for save/load IA32_PKRS MSR
  imx7-ccm: add digprog mmio write method
  tz-ppc: add dummy read/write methods
  spapr_pci: add spapr msi read method
  nvram: add nrf51_soc flash read method
  prep: add ppc-parity write method
  vfio: add quirk device write method
  pci-host: designware: add pcie-msi read method
  hw/pci-host: add pci-intack write method
  cpu-throttle: Remove timer_mod() from cpu_throttle_set()
  replay: rng-builtin support
  pc-bios/descriptors: fix paths in json files
  replay: fix replay of the interrupts
  accel/kvm/kvm-all: Fix wrong return code handling in dirty log code
  qapi/meson: Restrict UI module to system emulation and tools
  qapi/meson: Restrict system-mode specific modules
  qapi/meson: Remove QMP from user-mode emulation
  qapi/meson: Restrict qdev code to system-mode emulation
  meson: Restrict emulation code
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-02-09 10:04:51 +00:00
Peter Maydell
2436651b26 Migration pull 2021-02-08
v2
   Dropped vmstate: Fix memory leak in vmstate_handle_alloc
     Broke on Power
   Added migration: only check page size match if RAM postcopy is enabled
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEERfXHG0oMt/uXep+pBRYzHrxb/ecFAmAhIE4ACgkQBRYzHrxb
 /ecPuA/+Pgo++1ZSseJUgbLePwyTVc0jahdcvYEDmLUn8UM6ikBcBXBgUKHdkFW3
 bjSSVgB/xxvXSiafBK4xFNrCqSgqMSr3DJcHmvWgv2wVARcYf6Z26Da53LZq1Qru
 0tvRyb40Od1f9zb8Zj7e2Y3pjQ9ybLLbjfNhgnOBbQivqWkjZI31oV2KUCWY2+eV
 T1BEwr6mgYepqhmeB6OvQZtaQVC5toirS6NajNF4nt0vZEIGIvK6/A9erCVU8Tze
 5ch1J0MUqgc3q6ZSE/I9BHEy6MaL0X8G6H+ezjxdoRQtbt1iM/YqZJCSrXkAxiLC
 ROohryb6qVk26+UYuana79faLwrw359WlkwNEE6SEIRSENu+6p7bgN3LZuCILCO7
 xJEkeTgy6r40IGCkDC9aWa8pyLHpNX9gyLpGBHdIRD6zEOWaKNtzh7E2uo/T0ann
 BpcfgQOsYN25hIHiiXnxozUREbx71VDfMq7GqGB6eC3u2+a3U6jpSJb1nNq5NB89
 FJYLZy5Rbuy7OStMwfMsxRs7E63XvGgnwrN8FczU/pumCPX4lDYIpnocqinUmP8p
 XubRQQVaVDSKIq1mvzw7iR/1NsP9vfYvnrAIv941f38NBmDKqdPuMOXR/qB/Kp2Y
 jB7b1L5/JcXbWsQmK7fda9jmPzFwSO2cTeTiUonk9RfuuDEws0A=
 =4tbe
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/dgilbert/tags/pull-migration-20210208a' into staging

Migration pull 2021-02-08

v2
  Dropped vmstate: Fix memory leak in vmstate_handle_alloc
    Broke on Power
  Added migration: only check page size match if RAM postcopy is enabled

# gpg: Signature made Mon 08 Feb 2021 11:28:14 GMT
# gpg:                using RSA key 45F5C71B4A0CB7FB977A9FA90516331EBC5BFDE7
# gpg: Good signature from "Dr. David Alan Gilbert (RH2) <dgilbert@redhat.com>" [full]
# Primary key fingerprint: 45F5 C71B 4A0C B7FB 977A  9FA9 0516 331E BC5B FDE7

* remotes/dgilbert/tags/pull-migration-20210208a: (27 commits)
  migration: only check page size match if RAM postcopy is enabled
  migration: introduce snapshot-{save, load, delete} QMP commands
  iotests: fix loading of common.config from tests/ subdir
  iotests: add support for capturing and matching QMP events
  migration: introduce a delete_snapshot wrapper
  migration: wire up support for snapshot device selection
  migration: control whether snapshots are ovewritten
  block: rename and alter bdrv_all_find_snapshot semantics
  block: allow specifying name of block device for vmstate storage
  block: add ability to specify list of blockdevs during snapshot
  migration: stop returning errno from load_snapshot()
  migration: Make save_snapshot() return bool, not 0/-1
  block: push error reporting into bdrv_all_*_snapshot functions
  migration: Display the migration blockers
  migration: Add blocker information
  migration: Fix a few absurdly defective error messages
  migration: Fix cache_init()'s "Failed to allocate" error messages
  migration: Clean up signed vs. unsigned XBZRLE cache-size
  migration: Fix migrate-set-parameters argument validation
  migration: introduce 'userfaultfd-wrlat.py' script
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-02-08 18:23:47 +00:00
Peter Maydell
4f799257b3 QAPI patches patches for 2021-02-08
-----BEGIN PGP SIGNATURE-----
 
 iQJGBAABCAAwFiEENUvIs9frKmtoZ05fOHC0AOuRhlMFAmAhQpISHGFybWJydUBy
 ZWRoYXQuY29tAAoJEDhwtADrkYZTBFYP/jh1x1Isn35q3jwjzKy6N3EYbzmaRKhW
 rc5bFlZLQ95Nso+Q3b8izUSeoGz7t8E+WLZBBxzNoOWBS2uC8HS0jIpUmCJaB+Yu
 aVrG70+5/s/BPTX7iF6LIrcRj3GIQdFEWR5Zn1V4PFk5G7NZFwEiTnViq8pLw87H
 c2ktAorDyH6Zl4DA2YjV0XBLPMclt53sQ/M5b6ythUq6E05bVUkQFiUm3N8joIEX
 2djwumVi/dYpDwR/pVPBjPXcP5gc44y4D9sjJU6I16kyCyKi8Tr9mmEcSJFmR/Y/
 GdSilmtFhxUikAQogZwi/48BhR0GawhgcKZP5IkKNjvfwqE13Vhx8/p0pEubFPUE
 YexvFqkSmBRZ1mT0QpCcG4lp+u6PzhTyc/gbWwPm42/T+x9fG0cDgvDZCPjIRSsg
 LiUmmqrwOXr+Lw6GP1Q0f0KKv5QhCfeq1YcYrEXTQsa1PDT5T8ARBzxR5O1X2UNR
 Xw7j4u9R63p6P5nOk5+wwRLQkUkGl7N0SYqe4thyHUfW4r48V5J9RT7ONKD2CXO9
 Dw/Q3Ga1GmHaydpoZ4Az1k5kyr5dVBbAISRRWRpYA0sneg85o3RU3aTMt3r43pzC
 5pjeSpx6v7K4Y3NJPL1e/j/qJZq10WtEslkF/TKcBS2qLgiKCZ1oXQPORQxxX9uK
 zuK8oPQR7W42
 =uLm+
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/armbru/tags/pull-qapi-2021-02-08' into staging

QAPI patches patches for 2021-02-08

# gpg: Signature made Mon 08 Feb 2021 13:54:26 GMT
# gpg:                using RSA key 354BC8B3D7EB2A6B68674E5F3870B400EB918653
# gpg:                issuer "armbru@redhat.com"
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>" [full]
# gpg:                 aka "Markus Armbruster <armbru@pond.sub.org>" [full]
# Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867  4E5F 3870 B400 EB91 8653

* remotes/armbru/tags/pull-qapi-2021-02-08:
  qapi: enable strict-optional checks
  qapi: type 'info' as Optional[QAPISourceInfo]
  qapi/gen: Drop support for QAPIGen without a file name
  qapi/commands: Simplify command registry generation
  qapi/gen: Support switching to another module temporarily
  qapi/gen: write _genc/_genh access shims
  qapi: centralize the built-in module name definition
  qapi/gen: Combine ._add_[user|system]_module
  qapi: use './builtin' as the built-in module name
  qapi: use explicitly internal module names
  qapi/gen: Replace ._begin_system_module()
  qapi: centralize is_[user|system|builtin]_module methods
  qapi/gen: inline _wrap_ifcond into end_if()
  qapi/main: handle theoretical None-return from re.match()
  qapi/events: fix visit_event typing
  qapi/commands: assert arg_type is not None

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-02-08 16:12:21 +00:00
Qiuhao Li
487a1d13ba fuzz: fix wrong index in clear_bits
Signed-off-by: Qiuhao Li <Qiuhao.Li@outlook.com>
Message-Id: <SYCPR01MB3502E9F6EB06DEDCD484F738FCBA9@SYCPR01MB3502.ausprd01.prod.outlook.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-02-08 14:43:54 +01:00
John Snow
c51172667b qapi: enable strict-optional checks
In the modules that we are checking so far, we can be stricter about the
difference between Optional[T] and T types. Enable that check.

Enabling it now will assist review on further typing and cleanup work.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-17-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-08 14:15:58 +01:00
John Snow
4a82e468e7 qapi: type 'info' as Optional[QAPISourceInfo]
For everything typed so far, type this parameter as
Optional[QAPISourceInfo].

In the most generic case, QAPISchemaEntity's info field may be None to
represent types that come from built-in definitions. Although some
Entity types may not currently have any built-in definitions, it is not
easily possible to constrain the type except on an ad-hoc basis using
assertions.

It's easier and simpler, then, to just say it's always an Optional type.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-16-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-08 14:15:58 +01:00
Markus Armbruster
cc0747f6b7 qapi/gen: Drop support for QAPIGen without a file name
The previous commit removed the only user of QAPIGen(None).  Tighten
the type hint.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-15-jsnow@redhat.com>
2021-02-08 14:15:58 +01:00
Markus Armbruster
c6cd7e4151 qapi/commands: Simplify command registry generation
QAPISchemaGenCommandVisitor.visit_command() needs to generate the
marshalling function into the current module, and also generate its
registration into the ./init system module.  The latter is done
somewhat awkwardly: .__init__() creates a QAPIGenCCode that will not
be written out, each .visit_command() adds its registration to it, and
.visit_end() copies its contents into the ./init module it creates.

Instead provide the means to temporarily switch to another module.
Create the ./init module in .visit_begin(), and generate its initial
part.  Add registrations to it in .visit_command().  Finish it in
.visit_end().

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-14-jsnow@redhat.com>
2021-02-08 14:15:58 +01:00
Markus Armbruster
d921d27c1b qapi/gen: Support switching to another module temporarily
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-13-jsnow@redhat.com>
[Commit message tweaked]
2021-02-08 14:15:58 +01:00
John Snow
fd9b160384 qapi/gen: write _genc/_genh access shims
Many places assume they can access these fields without checking them
first to ensure they are defined. Eliminating the _genc and _genh fields
and replacing them with functional properties that check for correct
state can ease the typing overhead by eliminating the Optional[T] return
type.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-12-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-08 14:15:58 +01:00
John Snow
39b2d838f1 qapi: centralize the built-in module name definition
Use a constant to make it obvious we're referring to a very specific thing.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-11-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-08 14:15:58 +01:00
Markus Armbruster
4ab0ff6da0 qapi/gen: Combine ._add_[user|system]_module
With callers to _add_system_module now explicitly using the './' prefix
to indicate a system module, there is no longer any reason to have
separate interfaces for adding system vs user modules; use a unified
interface that differentiates based on the name.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-10-jsnow@redhat.com>
2021-02-08 14:15:58 +01:00
John Snow
e2bbc4eaa7 qapi: use './builtin' as the built-in module name
Use './builtin' as the built-in module name instead of
None. Clarify the typing that this is now always a string.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-9-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-08 14:15:58 +01:00
John Snow
12893a8ea7 qapi: use explicitly internal module names
QAPISchemaModularCVisitor._add_system_module() prefixes './' to its name
argument to make it a module name.  Pass the module name instead.  This
will allow us to coalesce the methods to add modules later on.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-8-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Commit message reworded]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-08 14:15:58 +01:00
Markus Armbruster
f3a705928a qapi/gen: Replace ._begin_system_module()
QAPISchemaModularCVisitor._begin_system_module() is actually just for
the builtin module.  Rename it to ._begin_builtin_module() and drop
its useless @name parameter.

Clarify conditionals in visit_module to make this clear.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-7-jsnow@redhat.com>
2021-02-08 14:15:58 +01:00
John Snow
98967c248c qapi: centralize is_[user|system|builtin]_module methods
Define what a module is and define what kind of a module it is once and
for all, in one place.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-6-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-08 14:15:58 +01:00
John Snow
a253b3eb9a qapi/gen: inline _wrap_ifcond into end_if()
We assert _start_if is not None in end_if, but that's opaque to mypy.
By inlining _wrap_ifcond, that constraint becomes provable to mypy.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-5-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-08 14:15:58 +01:00
John Snow
ad1218086e qapi/main: handle theoretical None-return from re.match()
Mypy cannot understand that this match can never be None, so help it
along.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-4-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-08 14:15:58 +01:00
John Snow
3cc01c546b qapi/events: fix visit_event typing
Actually, the arg_type can indeed be Optional.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-3-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-08 14:15:58 +01:00
John Snow
ec9697ab3f qapi/commands: assert arg_type is not None
When boxed is True, expr.py asserts that we must have
arguments. Ultimately, this should mean that if boxed is True that
arg_type should be defined. Mypy cannot infer this, and does not support
'stateful' type inference, e.g.:

```
if x:
    assert y is not None

...

if x:
    y.etc()
```

does not work, because mypy does not statefully remember the conditional
assertion in the second block. Help mypy out by creating a new local
that it can track more easily.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-2-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-08 14:15:58 +01:00
Andrey Gruzdev
c7243566d0 migration: introduce 'userfaultfd-wrlat.py' script
Add BCC/eBPF script to analyze userfaultfd write fault latency distribution.

Signed-off-by: Andrey Gruzdev <andrey.gruzdev@virtuozzo.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Message-Id: <20210129101407.103458-6-andrey.gruzdev@virtuozzo.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2021-02-08 11:19:51 +00:00
Alex Bennée
47e3424ac9 scripts/mtest2make.py: export all-%s-targets variable and use it
There are some places where the conditional makefile support is the
simplest solution. Now we don't expose CONFIG_TCG as a variable create
a new one that can be checked for the check-help output.

As check-tcg is a PHONY target we re-use check-softfloat to gate that
as well.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20210202134001.25738-13-alex.bennee@linaro.org>
2021-02-08 10:55:20 +00:00
Volker Rümelin
c6e93c9d62 simpletrace: build() missing 2 required positional arguments
Commit 4e66c9ef64 "tracetool: add input filename and line number to
Event" forgot to add a line number and a filename argument at one
build method call site.

Traceback (most recent call last):
  File "./scripts/simpletrace.py", line 261, in <module>
    run(Formatter())
  File "./scripts/simpletrace.py", line 236, in run
    process(events, sys.argv[2], analyzer, read_header=read_header)
  File "./scripts/simpletrace.py", line 177, in process
    dropped_event =
      Event.build("Dropped_Event(uint64_t num_events_dropped)")
TypeError: build() missing 2 required positional arguments:
  'lineno' and 'filename'

Add the missing arguments.

Signed-off-by: Volker Rümelin <vr_qemu@t-online.de>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-id: 20210131173415.3392-1-vr_qemu@t-online.de
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2021-02-01 10:51:00 +00:00
Stefan Hajnoczi
418ed14268 trace: make the 'log' backend timestamp configurable
Timestamps in tracing output can be distracting. Make it possible to
control tid/timestamp printing with -msg timestamp=on|off. The default
is no tid/timestamps. Previously they were always printed.

Suggested-by: BALATON Zoltan <balaton@eik.bme.hu>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: BALATON Zoltan <balaton@eik.bme.hu>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-id: 20210125113507.224287-3-stefanha@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2021-02-01 10:50:55 +00:00
Daniel P. Berrangé
09612de7e9 tracetool: also strip %l and %ll from systemtap format strings
All variables are 64-bit and so %l / %ll are not required, and the
latter is actually invalid:

  $ sudo stap -e 'probe begin{printf ("BEGIN")}'  -I .
  parse error: invalid or missing conversion specifier
          saw: operator ',' at ./qemu-system-x86_64-log.stp:15118:101
       source:     printf("%d@%d vhost_vdpa_set_log_base dev: %p base: 0x%x size: %llu
refcnt: %d fd: %d log: %p\n", pid(), gettimeofday_ns(), dev, base, size, refcnt, fd, log)

                       ^

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Laurent Vivier <lvivier@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Laurent Vivier <lvivier@redhat.com>
Message-id: 20210106130239.1004729-1-berrange@redhat.com

[Fixed "simiarly" typo found by Laurent Vivier <lvivier@redhat.com>
--Stefan]

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2021-02-01 10:50:55 +00:00
Laurent Vivier
9f4e519fd7 tracetool: fix "PRI" macro decoding
macro is not reset after use, so the format decoded is always the
one of the first "PRI" in the format string.

For instance:

  vhost_vdpa_set_config(void *dev, uint32_t offset, uint32_t size, \
                        uint32_t flags) "dev: %p offset: %"PRIu32" \
                        size: %"PRIu32" flags: 0x%"PRIx32

generates:

  printf("%d@%d vhost_vdpa_set_config dev: %p offset: %u size: %u \
          flags: 0x%u\n", pid(), gettimeofday_ns(), dev, offset, \
          size, flags)

for the "flags" parameter, we can see a "0x%u" rather than a "0x%x"
because the first macro was "PRIu32" (for offset).

In the loop, macro becomes "PRIu32PRIu32PRIx32", and c_macro_to_format()
returns always macro[3] ('u' in this case). This patch resets macro after
the format has been decoded.

Signed-off-by: Laurent Vivier <lvivier@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-id: 20210105191721.120463-3-lvivier@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2021-02-01 10:50:54 +00:00
Peter Maydell
74208cd252 * Replace --enable/disable-git-update with --with-git-submodules
to allow improved control over use of git submodules
 * Deprecate the -enable-fips option
 * Ensure docs use prefer format for bool options
 * Clarify platform support rules
 * Misc fixes to keymap conversions
 * Fix misc problems on macOS
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEE2vOm/bJrYpEtDo4/vobrtBUQT98FAmAUQXUACgkQvobrtBUQ
 T9/9Zw//VV3z7MI70HjWk17TUmkKLGFbmrc3xvtS6EMZbhfXpsR8qMfSjoRxP4Gn
 CgeftOa/lK5XjL7FcEUmcZeO8jZNE/+S5wgP2upP5Ae1zuDeyaSPTGR6UYs7WqSa
 P1kQsot2sAsZCP7Lko0jSv7rEDInK1bcdWr0a/xR5M2TvKiXoEdbElIZdwe3yLbC
 qWPWiv2pp6z2eGtJK/9VBxXjP0tlkNAR+jz8p/8RSZEEKf3aSq6HvVae+bHU86nX
 9t4baZk3ZuuwkmHJrHJP/72BjjitqEOiydtSsOCVYmY2GOclYeTnezG/vm75cGIK
 ej6hwFGElAEQk+KU9g7HOXi1eqXncSBjP0xWqc8PaZLW7wW/Uv8UV94fk+SCkIQ0
 GVBBLP8PzySL+w33NN7sv14cWeBpsjy9arMr4okZ80klFlS2hx/FqnUmhFhoN8ub
 ptIPUcSJoL+6pjxatV2yJ6bHkVga9PiwtLC8/rqETYPJz0b756xlPaEGd+0B1P0V
 ZMraE7lCgS0kJM6w0xF3R7JlGVtKouj4ZePCK5MGjpByBAK/vn/MbhbEmcyh+uJD
 r6sakKwqLjDLKdmQDd+biq3q6mvKE8ZKaUj8dXYdvxXSxxqXZ8DnrNpj/UkDTi3S
 chlTT1qUYrp2MVYqGmFjZ0XcEdEUR1I39rUs+wRxE3RgtE56M1A=
 =g6bC
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/berrange-gitlab/tags/misc-fixes-pull-request' into staging

* Replace --enable/disable-git-update with --with-git-submodules
  to allow improved control over use of git submodules
* Deprecate the -enable-fips option
* Ensure docs use prefer format for bool options
* Clarify platform support rules
* Misc fixes to keymap conversions
* Fix misc problems on macOS

# gpg: Signature made Fri 29 Jan 2021 17:10:13 GMT
# gpg:                using RSA key DAF3A6FDB26B62912D0E8E3FBE86EBB415104FDF
# gpg: Good signature from "Daniel P. Berrange <dan@berrange.com>" [full]
# gpg:                 aka "Daniel P. Berrange <berrange@redhat.com>" [full]
# Primary key fingerprint: DAF3 A6FD B26B 6291 2D0E  8E3F BE86 EBB4 1510 4FDF

* remotes/berrange-gitlab/tags/misc-fixes-pull-request:
  tests: Replace deprecated ASN1 code
  tests: Fix runtime error in test-authz-pam
  ui: update keycodemapdb submodule commit
  crypto: Add spaces around operator
  configure: replace --enable/disable-git-update with --with-git-submodules
  docs: fix missing backslash in certtool shell example
  docs: simplify and clarify the platform support rules
  Prefer 'on' | 'off' over 'yes' | 'no' for bool options
  os: deprecate the -enable-fips option and QEMU's FIPS enforcement
  crypto: Fix memory leaks in set_loaded for tls-*
  crypto: Forbid broken unloading of secrets
  crypto: Move USER_CREATABLE to secret_common base class
  crypto: Fix some code style problems, add spaces around operator

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-01-29 19:51:25 +00:00
Dan Streetman
7d7dbf9dc1 configure: replace --enable/disable-git-update with --with-git-submodules
Replace the --enable-git-update and --disable-git-update configure params
with the param --with-git-submodules=(update|validate|ignore) to
allow 3 options for building from a git repo.

This is needed because downstream packagers, e.g. Debian, Ubuntu, etc,
also keep the source code in git, but do not want to enable the
'git_update' mode; with the current code, that's not possible even
if the downstream package specifies --disable-git-update.

The previous parameters are deprecated but still available; the
--enable-git-update parameter maps to --with-git-submodules=update and
--disable-git-update parameter maps to --with-git-submodules=validate.

The configure script behavior is slightly modified, where previously
the dtc, capstone, and slirp submodules were not validated when
--disable-git-update was specified (but were updated with git-update
enabled), now they are validated when using --with-git-submodules=validate
and are only ignored when using --with-git-submodules=ignore.

Signed-off-by: Dan Streetman <ddstreet@canonical.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2021-01-29 17:07:53 +00:00
Alexander Graf
8a74ce618b hvf: Add hypervisor entitlement to output binaries
In macOS 11, QEMU only gets access to Hypervisor.framework if it has the
respective entitlement. Add an entitlement template and automatically self
sign and apply the entitlement in the build.

Signed-off-by: Alexander Graf <agraf@csgraf.de>
Reviewed-by: Roman Bolshakov <r.bolshakov@yadro.com>
Tested-by: Roman Bolshakov <r.bolshakov@yadro.com>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-01-29 10:47:28 +00:00
Vladimir Sementsov-Ogievskiy
c701f59253 simplebench: add bench-backup.py
Add script to benchmark new backup architecture.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20210116214705.822267-24-vsementsov@virtuozzo.com>
[mreitz: s/not unsupported/not supported/]
Signed-off-by: Max Reitz <mreitz@redhat.com>
2021-01-26 14:36:37 +01:00
Vladimir Sementsov-Ogievskiy
b2fcb0c575 simplebench: bench_block_job: add cmd_options argument
Add argument to allow additional block-job options.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-23-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2021-01-26 14:36:37 +01:00
Vladimir Sementsov-Ogievskiy
2096de521e simplebench/bench_block_job: use correct shebang line with python3
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-22-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2021-01-26 14:36:37 +01:00
Eric Farman
ab5ec23f9c update-linux-headers: Include const.h
Kernel commit a85cbe6159ff ("uapi: move constants from
<linux/kernel.h> to <linux/const.h>") breaks our script
because of the unrecognized include. Let's add that to
our processing.

Signed-off-by: Eric Farman <farman@linux.ibm.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210104202057.48048-2-farman@linux.ibm.com>
Signed-off-by: Cornelia Huck <cohuck@redhat.com>
2021-01-21 11:19:45 +01:00
Alex Bennée
9e5acb373d scripts/checkpatch.pl: fix git-show invocation to include diffstat
Without this checkpatch keeps complaining about new/changed files even
when MAINTAINERS has been updated. Normal invocations of checkpatch on
patch files rather than commit IDs are unaffected.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210114165730.31607-13-alex.bennee@linaro.org>
2021-01-18 10:04:31 +00:00
Philippe Mathieu-Daudé
4cacecaaa2 decodetree: Open files with encoding='utf-8'
When decodetree.py was added in commit 568ae7efae, QEMU was
using Python 2 which happily reads UTF-8 files in text mode.
Python 3 requires either UTF-8 locale or an explicit encoding
passed to open(). Now that Python 3 is required, explicit
UTF-8 encoding for decodetree source files.

To avoid further problems with the user locale, also explicit
UTF-8 encoding for the generated C files.

Explicit both input/output are plain text by using the 't' mode.

This fixes:

  $ /usr/bin/python3 scripts/decodetree.py test.decode
  Traceback (most recent call last):
    File "scripts/decodetree.py", line 1397, in <module>
      main()
    File "scripts/decodetree.py", line 1308, in main
      parse_file(f, toppat)
    File "scripts/decodetree.py", line 994, in parse_file
      for line in f:
    File "/usr/lib/python3.6/encodings/ascii.py", line 26, in decode
      return codecs.ascii_decode(input, self.errors)[0]
  UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 80:
  ordinal not in range(128)

Reported-by: Peter Maydell <peter.maydell@linaro.org>
Suggested-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20210110000240.761122-1-f4bug@amsat.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-01-13 08:39:08 -10:00
Maxim Levitsky
b9a0de3773 scripts/gdb: implement 'qemu bt'
This script first runs the regular gdb's 'bt' command, and then if we are in a
coroutine it prints the coroutines backtraces in the order in which they
were called.

Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
Message-Id: <20201217155436.927320-3-mlevitsk@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-01-12 12:38:03 +01:00
Maxim Levitsky
4cbf8efc5b scripts/gdb: fix 'qemu coroutine' when users selects a non topmost stack frame
The code that dumps the stack frame works like that:
* save current registers
* overwrite current registers (including rip/rsp) with coroutine snapshot
  in the jmpbuf
* print backtrace
* restore the saved registers.

If the user has currently selected a non topmost stack frame in gdb,
the above code will still restore the selected frame registers,
but the gdb will then lose the selected frame index, which makes it impossible
to switch back to frame 0, to continue debugging the executable.

Therefore switch temporarily to the topmost frame of the stack
for the above code.

Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
Message-Id: <20201217155436.927320-2-mlevitsk@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-01-12 12:38:03 +01:00
Qiuhao Li
4cc5752303 fuzz: heuristic split write based on past IOs
If previous write commands write the same length of data with the same step,
we view it as a hint.

Signed-off-by: Qiuhao Li <Qiuhao.Li@outlook.com>
Reviewed-by: Alexander Bulekov <alxndr@bu.edu>
Tested-by: Alexander Bulekov <alxndr@bu.edu>
Message-Id: <SYCPR01MB3502480AD07811A6A49B8FEAFCAB0@SYCPR01MB3502.ausprd01.prod.outlook.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-01-11 14:59:21 +01:00
Qiuhao Li
dd21ed0edf fuzz: add minimization options
-M1: remove IO commands iteratively
-M2: try setting bits in operand of write/out to zero

Signed-off-by: Qiuhao Li <Qiuhao.Li@outlook.com>
Reviewed-by: Alexander Bulekov <alxndr@bu.edu>
Tested-by: Alexander Bulekov <alxndr@bu.edu>
Message-Id: <SYCPR01MB350204C52E7A39E6B0EEC870FCAB0@SYCPR01MB3502.ausprd01.prod.outlook.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-01-11 14:59:21 +01:00
Qiuhao Li
9d20f2af53 fuzz: set bits in operand of write/out to zero
Simplifying the crash cases by opportunistically setting bits in operands of
out/write to zero may help to debug, since usually bit one means turn on or
trigger a function while zero is the default turn-off setting.

Tested bug https://bugs.launchpad.net/qemu/+bug/1908062

Signed-off-by: Qiuhao Li <Qiuhao.Li@outlook.com>
Reviewed-by: Alexander Bulekov <alxndr@bu.edu>
Tested-by: Alexander Bulekov <alxndr@bu.edu>
Message-Id: <SYCPR01MB3502C84B6346A3E3DE708C7BFCAB0@SYCPR01MB3502.ausprd01.prod.outlook.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-01-11 14:59:21 +01:00
Qiuhao Li
247ab240c2 fuzz: remove IO commands iteratively
Now we use a one-time scan and remove strategy in the minimizer,
which is not suitable for timing dependent instructions.

For example, instruction A will indicate an address where the config
chunk locates, and instruction B will make the configuration active.
If we have the following instruction sequence:

...
A1
B1
A2
B2
...

A2 and B2 are the actual instructions that trigger the bug.

If we scan from top to bottom, after we remove A1, the behavior of B1
might be unknowable, including not to crash the program. But we will
successfully remove B1 later cause A2 and B2 will crash the process
anyway:

...
A1
A2
B2
...

Now one more trimming will remove A1.

In the perfect case, we would need to be able to remove A and B (or C!) at
the same time. But for now, let's just add a loop around the minimizer.

Since we only remove instructions, this iterative algorithm is converging.

Tested with Bug 1908062.

Signed-off-by: Qiuhao Li <Qiuhao.Li@outlook.com>
Reviewed-by: Alexander Bulekov <alxndr@bu.edu>
Tested-by: Alexander Bulekov <alxndr@bu.edu>
Message-Id: <SYCPR01MB350263004448040ACCB9A9F1FCAB0@SYCPR01MB3502.ausprd01.prod.outlook.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-01-11 14:59:21 +01:00
Qiuhao Li
e72203abec fuzz: split write operand using binary approach
Currently, we split the write commands' data from the middle. If it does not
work, try to move the pivot left by one byte and retry until there is no
space.

But, this method has two flaws:

1. It may fail to trim all unnecessary bytes on the right side.

For example, there is an IO write command:

  write addr uuxxxxuu

u is the unnecessary byte for the crash. Unlike ram write commands, in most
case, a split IO write won't trigger the same crash, So if we split from the
middle, we will get:

  write addr uu (will be removed in next round)
  write addr xxxxuu

For xxxxuu, since split it from the middle and retry to the leftmost byte
won't get the same crash, we will be stopped from removing the last two
bytes.

2. The algorithm complexity is O(n) since we move the pivot byte by byte.

To solve the first issue, we can try a symmetrical position on the right if
we fail on the left. As for the second issue, instead moving by one byte, we
can approach the boundary exponentially, achieving O(log(n)).

Give an example:

                   xxxxuu len=6
                        +
                        |
                        +
                 xxx,xuu 6/2=3 fail
                        +
         +--------------+-------------+
         |                            |
         +                            +
  xx,xxuu 6/2^2=1 fail         xxxxu,u 6-1=5 success
                                 +   +
         +------------------+----+   |
         |                  |        +-------------+ u removed
         +                  +
   xx,xxu 5/2=2 fail  xxxx,u 6-2=4 success
                           +
                           |
                           +-----------+ u removed

In some rare cases, this algorithm will fail to trim all unnecessary bytes:

  xxxxxxxxxuxxxxxx
  xxxxxxxx-xuxxxxxx Fail
  xxxx-xxxxxuxxxxxx Fail
  xxxxxxxxxuxx-xxxx Fail
  ...

I think the trade-off is worth it.

Signed-off-by: Qiuhao Li <Qiuhao.Li@outlook.com>
Reviewed-by: Alexander Bulekov <alxndr@bu.edu>
Tested-by: Alexander Bulekov <alxndr@bu.edu>
Message-Id: <SYCPR01MB3502D26F1BEB680CBBC169E5FCAB0@SYCPR01MB3502.ausprd01.prod.outlook.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-01-11 14:59:21 +01:00
Qiuhao Li
7b339f287f fuzz: double the IOs to remove for every loop
Instead of removing IO instructions one by one, we can try deleting multiple
instructions at once. According to the locality of reference, we double the
number of instructions to remove for the next round and recover it to one
once we fail.

This patch is usually significant for large input.

Test with quadrupled trace input at:
  https://bugs.launchpad.net/qemu/+bug/1890333/comments/1

Patched 1/6 version:
  real  0m45.904s
  user  0m16.874s
  sys   0m10.042s

Refined version:
  real  0m11.412s
  user  0m6.888s
  sys   0m3.325s

Signed-off-by: Qiuhao Li <Qiuhao.Li@outlook.com>
Reviewed-by: Alexander Bulekov <alxndr@bu.edu>
Tested-by: Alexander Bulekov <alxndr@bu.edu>
Message-Id: <SYCPR01MB350280A67BB55C3FADF173E3FCAB0@SYCPR01MB3502.ausprd01.prod.outlook.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-01-11 14:59:21 +01:00
Qiuhao Li
22ec0c696f fuzz: accelerate non-crash detection
We spend much time waiting for the timeout program during the minimization
process until it passes a time limit. This patch hacks the CLOSED (indicates
the redirection file closed) notification in QTest's output if it doesn't
crash.

Test with quadrupled trace input at:
  https://bugs.launchpad.net/qemu/+bug/1890333/comments/1

Original version:
  real	1m37.246s
  user	0m13.069s
  sys	0m8.399s

Refined version:
  real	0m45.904s
  user	0m16.874s
  sys	0m10.042s

Note:

Sometimes the mutated or the same trace may trigger a different crash
summary (second-to-last line) but indicates the same bug. For example, Bug
1910826 [1], which will trigger a stack overflow, may output summaries
like:

SUMMARY: AddressSanitizer: stack-overflow
/home/qiuhao/hack/qemu/build/../softmmu/physmem.c:488 in
flatview_do_translate

or

SUMMARY: AddressSanitizer: stack-overflow
(/home/qiuhao/hack/qemu/build/qemu-system-i386+0x27ca049) in __asan_memcpy

Etc.

If we use the whole summary line as the token, we may be prevented from
further minimization. So in this patch, we only use the first three words
which indicate the type of crash:

SUMMARY: AddressSanitizer: stack-overflow

[1] https://bugs.launchpad.net/qemu/+bug/1910826

Signed-off-by: Qiuhao Li <Qiuhao.Li@outlook.com>
Reviewed-by: Alexander Bulekov <alxndr@bu.edu>
Tested-by: Alexander Bulekov <alxndr@bu.edu>
Message-Id: <SYCPR01MB350251DC04003450348FAF68FCAB0@SYCPR01MB3502.ausprd01.prod.outlook.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-01-11 14:59:21 +01:00
Peter Maydell
01b3e68bb1 scripts/coccinelle: New script to remove unnecessary timer_del() calls
Now that timer_free() implicitly calls timer_del(), sequences
  timer_del(mytimer);
  timer_free(mytimer);

can be simplified to just
  timer_free(mytimer);

Add a Coccinelle script to do this transformation.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20201215154107.3255-3-peter.maydell@linaro.org
2021-01-08 15:13:38 +00:00
Stefan Hajnoczi
7fb48c0ee1 tracetool: show trace-events filename/lineno in fmt string errors
The compiler encounters trace event format strings in generated code.
Format strings are error-prone and therefore clear compiler errors are
important.

Use the #line directive to show the trace-events filename and line
number in format string errors:
https://gcc.gnu.org/onlinedocs/gcc-10.2.0/cpp/Line-Control.html

For example, if the cpu_in trace event's %u is changed to %p the
following error is reported:

  trace-events:29:18: error: format ‘%p’ expects argument of type ‘void *’, but argument 7 has type ‘unsigned int’ [-Werror=format=]

Line 29 in trace-events is where cpu_in is defined. This works for any
trace-events file in the QEMU source tree and the correct path is
displayed.

Unfortunately there does not seem to be a way to set the column, so "18"
is not the right character on that line.

Suggested-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Message-Id: <20200827142915.108730-5-stefanha@redhat.com>
2021-01-04 14:24:58 +00:00
Stefan Hajnoczi
4e66c9ef64 tracetool: add input filename and line number to Event
Store the input filename and line number in Event.

A later patch will use this to improve error messages.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Message-Id: <20200827142915.108730-4-stefanha@redhat.com>
2021-01-04 14:24:58 +00:00
Stefan Hajnoczi
294170c1dd tracetool: add out_lineno and out_next_lineno to out()
Make the output file line number and next line number available to
out().

A later patch will use this to improve error messages.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Message-Id: <20200827142915.108730-3-stefanha@redhat.com>
2021-01-04 14:24:58 +00:00
Stefan Hajnoczi
c05012a365 tracetool: add output filename command-line argument
The tracetool.py script writes to stdout. This means the output filename
is not available to the script. Add the output filename to the
command-line so that the script has access to the filename.

This also simplifies the tracetool.py invocation. It's no longer
necessary to use meson's custom_build(capture : true) to save output.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Message-Id: <20200827142915.108730-2-stefanha@redhat.com>
2021-01-04 14:24:58 +00:00
Vladimir Sementsov-Ogievskiy
cff6d3ca43 scripts/simplebench: add bench_prealloc.py
Benchmark for new preallocate filter.

Example usage:
    ./bench_prealloc.py ../../build/qemu-img \
        ssd-ext4:/path/to/mount/point \
        ssd-xfs:/path2 hdd-ext4:/path3 hdd-xfs:/path4

The benchmark shows performance improvement (or degradation) when use
new preallocate filter with qcow2 image.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20201021145859.11201-22-vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-12-18 12:35:55 +01:00
Vladimir Sementsov-Ogievskiy
181f60c8c7 simplebench/results_to_text: make executable
Make results_to_text a tool to dump results saved in JSON file.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20201021145859.11201-21-vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-12-18 12:35:55 +01:00
Vladimir Sementsov-Ogievskiy
aa362403f4 simplebench/results_to_text: add difference line to the table
Performance improvements / degradations are usually discussed in
percentage. Let's make the script calculate it for us.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20201021145859.11201-20-vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
[mreitz: 'seconds' instead of 'secs']
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-12-18 12:35:55 +01:00
Vladimir Sementsov-Ogievskiy
96be1aeec7 simplebench/results_to_text: improve view of the table
Move to generic format for floats and percentage for error.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20201021145859.11201-19-vsementsov@virtuozzo.com>
Acked-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-12-18 12:35:55 +01:00
Vladimir Sementsov-Ogievskiy
8e979febb0 simplebench: move results_to_text() into separate file
Let's keep view part in separate: this way it's better to improve it in
the following commits.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20201021145859.11201-18-vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-12-18 12:35:55 +01:00
Vladimir Sementsov-Ogievskiy
bfccfa62ac simplebench: rename ascii() to results_to_text()
Next patch will use utf8 plus-minus symbol, let's use more generic (and
more readable) name.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20201021145859.11201-17-vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-12-18 12:35:55 +01:00
Vladimir Sementsov-Ogievskiy
f52e1af0b0 scripts/simplebench: use standard deviation for +- error
Standard deviation is more usual to see after +- than current maximum
of deviations.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20201021145859.11201-16-vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-12-18 12:35:55 +01:00
Vladimir Sementsov-Ogievskiy
4a44554a65 scripts/simplebench: support iops
Support benchmarks returning not seconds but iops. We'll use it for
further new test.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20201021145859.11201-15-vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-12-18 12:35:55 +01:00
Vladimir Sementsov-Ogievskiy
270124e7ef scripts/simplebench: fix grammar: s/successed/succeeded/
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20201021145859.11201-14-vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-12-18 12:35:55 +01:00
Philippe Mathieu-Daudé
a1bcbb485c scripts/git.orderfile: Keep files with .inc extension sorted
Sort .inc files along with the extension including them.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20201213205132.243628-1-f4bug@amsat.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-12-15 12:53:16 -05:00
Marc-André Lureau
07b35a23c3 compiler.h: remove QEMU_GNUC_PREREQ
When needed, the G_GNUC_CHECK_VERSION() glib macro can be used instead.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20201210134752.780923-14-marcandre.lureau@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-12-15 12:53:15 -05:00
Paolo Bonzini
953d0c333e scripts: kernel-doc: remove unnecessary change wrt Linux
A comment in kernel-doc mentions QEMU's qatomic_set macro, but since
this code originated in Linux we should just revert it and stay as close
to the kernel's copy of the script as possible.

The change was introduced (more or less unintentionally) in QEMU commit
commit d73415a315, which did a global search-and-replace of QEMU's
atomic access macros.

Suggested-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-12-10 12:15:25 -05:00
Mauro Carvalho Chehab
a1b8a57a0e scripts: kernel-doc: use :c:union when needed
Sphinx C domain code after 3.2.1 will start complaning if :c:struct
would be used for an union type:

	.../Documentation/gpu/drm-kms-helpers:352: ../drivers/video/hdmi.c:851: WARNING: C 'identifier' cross-reference uses wrong tag: reference name is 'union hdmi_infoframe' but found name is 'struct hdmi_infoframe'. Full reference name is 'union hdmi_infoframe'. Full found name is 'struct hdmi_infoframe'.

So, let's address this issue too in advance, in order to
avoid future issues.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/6e4ec3eec914df62389a299797a3880ae4490f35.1603791716.git.mchehab+huawei@kernel.org
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201117165312.118257-30-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-12-10 12:15:25 -05:00
Mauro Carvalho Chehab
e495a1b26c scripts: kernel-doc: split typedef complex regex
The typedef regex for function prototypes are very complex.
Split them into 3 separate regex and then join them using
qr.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/3a4af999a0d62d4ab9dfae1cdefdfcad93383356.1603792384.git.mchehab+huawei@kernel.org
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201117165312.118257-29-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-12-10 12:15:24 -05:00
Mauro Carvalho Chehab
01a7917d9b scripts: kernel-doc: fix typedef parsing
The include/linux/genalloc.h file defined this typedef:

	typedef unsigned long (*genpool_algo_t)(unsigned long *map,unsigned long size,unsigned long start,unsigned int nr,void *data, struct gen_pool *pool, unsigned long start_addr);

Because it has a type composite of two words (unsigned long),
the parser gets the typedef name wrong:

.. c:macro:: long

   **Typedef**: Allocation callback function type definition

Fix the regex in order to accept composite types when
defining a typedef for a function pointer.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/328e8018041cc44f7a1684e57f8d111230761c4f.1603792384.git.mchehab+huawei@kernel.org
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201117165312.118257-28-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-12-10 12:15:24 -05:00