Commit Graph

1951 Commits

Author SHA1 Message Date
Markus Armbruster
0e92a19b8c qapi: Fix parse errors for removal of null from schema language
Commit 9d55380b5a "qapi: Remove null from schema language" (v4.2.0)
neglected to update two error messages.  Do that now.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210224101442.1837475-1-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
2021-03-05 15:40:50 +01:00
Peter Maydell
07dbfdd290 * fix --enable-fuzzing linker failures (Alexander)
* target/i386: Add bus lock debug exception support (Chenyi)
 * update documentation for preferred boolean option syntax (Daniel)
 * make SCSI io_timeout configurable (Hannes)
 * fix handling of guest recoverable SCSI errors (myself)
 * misc fixes (Pavel, Zheng Zhan Liang, Zihao)
 * fix installation of binaries with entitlements (Akihiko)
 -----BEGIN PGP SIGNATURE-----
 
 iQFIBAABCAAyFiEE8TM4V0tmI4mGbHaCv/vSX3jHroMFAmA3tzQUHHBib256aW5p
 QHJlZGhhdC5jb20ACgkQv/vSX3jHroMsngf+OkbgTDouxq3qnn1K89n6j2wKlAj0
 qPP+57o1sv5u5MjX0BkawXCMJfvTKJaOt/Cs8S0FaPTehfR5H0T2tL76KZBEdlEP
 /+V3EC0Qmd1S47LKIOuDkV15szvnccyjpgwaL7Osjb0Eh0SxeXd4pBTc3yYtkacu
 zb/srZ63U2iJZr93QwGmEJjuW57WNym6QWXF3Mrg9PFXybkXFTGumB2vZMlQc+MN
 RWPTUKZKAiIr/HfdvpIETXY+HzJhylvIQ8OVWtM3XDorl48OWcOat/jyeNEVYgG3
 o+gxIRnfvIL+42GkmUFbOKLzT5SV6p9EYYGLTuE1kU7nc6HWoD0CdPxA3A==
 =UBVO
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/bonzini-gitlab/tags/for-upstream' into staging

* fix --enable-fuzzing linker failures (Alexander)
* target/i386: Add bus lock debug exception support (Chenyi)
* update documentation for preferred boolean option syntax (Daniel)
* make SCSI io_timeout configurable (Hannes)
* fix handling of guest recoverable SCSI errors (myself)
* misc fixes (Pavel, Zheng Zhan Liang, Zihao)
* fix installation of binaries with entitlements (Akihiko)

# gpg: Signature made Thu 25 Feb 2021 14:41:56 GMT
# gpg:                using RSA key F13338574B662389866C7682BFFBD25F78C7AE83
# gpg:                issuer "pbonzini@redhat.com"
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>" [full]
# gpg:                 aka "Paolo Bonzini <pbonzini@redhat.com>" [full]
# Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4  E2F7 7E15 100C CD36 69B1
#      Subkey fingerprint: F133 3857 4B66 2389 866C  7682 BFFB D25F 78C7 AE83

* remotes/bonzini-gitlab/tags/for-upstream: (29 commits)
  tcg/i386: rdpmc: fix the the condtions
  chardev: do not use short form boolean options in non-QemuOpts character device descriptions
  vl: deprecate -writeconfig
  target/i386: Add bus lock debug exception support
  qom/object.c: Fix typo
  target/i386: update to show preferred boolean syntax for -cpu
  docs: update to show preferred boolean syntax for -cpu
  docs: update to show preferred boolean syntax for -vnc
  docs: update to show preferred boolean syntax for -chardev
  qemu-options: update to show preferred boolean syntax for -vnc
  qemu-options: update to show preferred boolean syntax for -incoming
  qemu-options: update to show preferred boolean syntax for -netdev
  qemu-options: update to show preferred boolean syntax for -spice
  qemu-options: update to show preferred boolean syntax for -chardev
  gdbstub: use preferred boolean option syntax
  char: don't fail when client is not connected
  scsi: drop 'result' argument from command_complete callback
  scsi-disk: pass guest recoverable errors through even for rerror=stop
  scsi-disk: pass SCSI status to scsi_handle_rw_error
  scsi: introduce scsi_sense_from_errno()
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-03-02 15:17:45 +00:00
Peter Maydell
cbcf09872a Testing tweaks (build, docs, bumps)
- expose cross compiler info in meson pretty print
   - bump Fedora to 33
   - "graceful" handling of missing virgl config
   - updates to the container documentation
   - move CODING_STYLE.rst into developer manual
 -----BEGIN PGP SIGNATURE-----
 
 iQEzBAABCgAdFiEEZoWumedRZ7yvyN81+9DbCVqeKkQFAmA2M5MACgkQ+9DbCVqe
 KkT3FAf/VqJMFin2ON8GxD/0OvaPhX2zOMrPzzUioMcHZKTW2AFytY0y4JPp7bcg
 rCqbHHa2escODQk0p1Wf5VvNYSjC9Q2PA0tmGXIZPYt+8Xf2N5l9Ub3woDBO6uKD
 cEY590wWzrcLoaSEyTcqEUKFD5Ce9PO/WAf/dlZgYr3Mo1flX17wBNvrMi4iQrld
 mLajqx3x4vonnda4nrAiz7fD/CLwE/50+21+0xN2vY3YA7EjRp8UfkUBIwe+Fubn
 iu0FmGgHn7coNSHSD8sfYKHeuo/fMJl6jKiWJdFC7343WdAQtsnYF7Uhml+T6X7i
 cm8yHCaBnbpHEw0x7kTsxfP/1ixYVw==
 =hyyV
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/stsquad/tags/pull-testing-updates-240221-1' into staging

Testing tweaks (build, docs, bumps)

  - expose cross compiler info in meson pretty print
  - bump Fedora to 33
  - "graceful" handling of missing virgl config
  - updates to the container documentation
  - move CODING_STYLE.rst into developer manual

# gpg: Signature made Wed 24 Feb 2021 11:08:03 GMT
# gpg:                using RSA key 6685AE99E75167BCAFC8DF35FBD0DB095A9E2A44
# gpg: Good signature from "Alex Bennée (Master Work Key) <alex.bennee@linaro.org>" [full]
# Primary key fingerprint: 6685 AE99 E751 67BC AFC8  DF35 FBD0 DB09 5A9E 2A44

* remotes/stsquad/tags/pull-testing-updates-240221-1:
  docs: move CODING_STYLE into the developer documentation
  docs/devel: add forward reference to check-tcg
  docs/devel: update the container based tests
  docs/devel: expand on use of containers to build tests
  tests/acceptance: allow a "graceful" failing for virtio-gpu test
  docker: Bump Fedora images to release 33
  meson.build: expose TCG cross compiler information in summary

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-02-25 19:07:58 +00:00
Daniel P. Berrangé
c23874132b docs: update to show preferred boolean syntax for -chardev
The preferred syntax is to use "foo=on|off", rather than a bare
"foo" or "nofoo".

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20210216191027.595031-8-berrange@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-02-25 14:14:33 +01:00
Akihiko Odaki
237377ac72 hvf: Sign the code after installation
Before this change, the code signed during the build was installed
directly.

However, the signature gets invalidated because meson modifies the code
to fix dynamic library install names during the install process.

It also prevents meson to strip the code because the pre-signed file is
not marked as an executable (although it is somehow able to perform the
modification described above).

With this change, the unsigned code will be installed and modified by
meson first, and a script signs it later.

Signed-off-by: Akihiko Odaki <akihiko.odaki@gmail.com>
Message-Id: <20210225000614.46919-1-akihiko.odaki@gmail.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-02-25 13:57:34 +01:00
Alex Bennée
93a1100768 docs: move CODING_STYLE into the developer documentation
There is no particular reason to keep this on it's own in the root of
the tree. Move it into the rest of the fine developer manual and fixup
any links to it. The only tweak I've made is to fix the code-block
annotations to mention the language C.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Claudio Fontana <cfontana@suse.de>
Message-Id: <20210223095931.16908-1-alex.bennee@linaro.org>
2021-02-24 11:05:21 +00:00
Isaku Yamahata
d2f1af0e41 checkpatch: don't emit warning on newly created acpi data files
Newly created acpi data files(tests/data/acpi/) cause false positive
warning.
If file names are acpi expected file, don't emit warning.

Fixes: e625ba2a41 ("checkpatch: fix acpi check with multiple file name")
Signed-off-by: Isaku Yamahata <isaku.yamahata@intel.com>
Message-Id: <6899f9ad54cab8e7deca94ff0eeab641680e2b5e.1613615732.git.isaku.yamahata@intel.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2021-02-23 10:58:42 -05:00
Peter Maydell
e90ef02389 QAPI patches patches for 2021-02-18
-----BEGIN PGP SIGNATURE-----
 
 iQJGBAABCAAwFiEENUvIs9frKmtoZ05fOHC0AOuRhlMFAmAutzcSHGFybWJydUBy
 ZWRoYXQuY29tAAoJEDhwtADrkYZTvBsP/jI3Dh9j8HLLsvauU117LcIGnn5Qwog4
 8v0IrEieaOeMDNDzR8AgSqBLYm0ZQcBoZl7Qkv8XMxYa7iWiJv2uCShRS7MNl5NK
 M8IW0XWVjCJJ7TfFGj5iP/zlnhBMTzp66EDfkdiBWlig7NdZ+TBVp+WiZr8mT7EJ
 MdI2slzmgChDHJ2BLntLDt3w778GUIIXYxNFAhH4woAs6Xq7Z8tq3y+7c9JMXL/1
 B1esVN49h582mcldT5LcMK1AWOjxon+IaKS5KloMcr/uVCqaineSR/rPTXI4k9RL
 KfKeetniOz0ohyD+4dfLYW7OPhjJ3ZiUm6tSP6aVcg8D8qJAEvocw1uZS3iFpR5T
 lqdS6rcaumx3FaPix3MmhbNJLytohiAvvJyHpjLQ32ByhJ2FPzUOLFyRvoeoE+kD
 wJSTLn0iyjIlY3bz4sdA3STWK9TyKJ8enyRWGGZECL0ZJwCUe/Z7KwxZi7Ad4vLl
 j/os65T+XudTEJzk4iKpA03ThvAs82q34mpSkpACfDA9VZ584fRC79WbQoVAGmrk
 3CV4vfBS3pcPz7nCQqauTGWg3iqQT4iKYGdw9YTqA88kIk/swfPiucGrAwI1WPXz
 VReMbz/LBBa8gLb2Sx4MMlRjO/v/WlOyxJ1sfdETmiX+ZjUl8s7+94PwlWI2kT6k
 3r9bl3Tz+QlZ
 =pIh0
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/armbru/tags/pull-qapi-2021-02-18' into staging

QAPI patches patches for 2021-02-18

# gpg: Signature made Thu 18 Feb 2021 18:51:35 GMT
# gpg:                using RSA key 354BC8B3D7EB2A6B68674E5F3870B400EB918653
# gpg:                issuer "armbru@redhat.com"
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>" [full]
# gpg:                 aka "Markus Armbruster <armbru@pond.sub.org>" [full]
# Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867  4E5F 3870 B400 EB91 8653

* remotes/armbru/tags/pull-qapi-2021-02-18:
  qapi/introspect.py: set _gen_tree's default ifcond argument to ()
  qapi/introspect.py: Type _gen_tree variants as Sequence[str]
  qapi/introspect.py: Update copyright and authors list
  qapi/introspect.py: Add docstrings to _gen_tree and _tree_to_qlit
  qapi/introspect.py: add type hint annotations
  qapi/introspect.py: remove _gen_variants helper
  qapi/introspect.py: improve readability of _tree_to_qlit
  qapi/introspect.py: improve _tree_to_qlit error message
  qapi/introspect.py: create a typed 'Annotated' data strutcure
  qapi/introspect.py: Introduce preliminary tree typing
  qapi/introspect.py: Always define all 'extra' dict keys
  qapi/introspect.py: replace 'extra' dict with 'comment' argument
  qapi/introspect.py: Unify return type of _make_tree()
  qapi/introspect.py: guard against ifcond/comment misuse
  qapi/introspect.py: add _gen_features helper
  qapi/introspect.py: use _make_tree for features nodes
  qapi/introspect.py: assert schema is not None
  qapi: Replace List[str] with Sequence[str] for ifcond

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-02-19 17:22:42 +00:00
Peter Maydell
ce42fe17ad * Always build the container images in the gitlab-CI
* Some other small gitlab-CI improvements
 * Some qtest fixes
 -----BEGIN PGP SIGNATURE-----
 
 iQJFBAABCAAvFiEEJ7iIR+7gJQEY8+q5LtnXdP5wLbUFAmAvVkwRHHRodXRoQHJl
 ZGhhdC5jb20ACgkQLtnXdP5wLbX6JRAAmpDk8N3JP8XxG4ZVBbAquIRr87CgVykO
 Si4bQtoy6UGcht1aFM2QnfoU54lxVBArSjbqsV1soqHjC5mLl1t7FaHS98hOx7t4
 4x3oejxyiXhh9U+6AEzuSAlXLXYUs4eqpQ7f7AHrhqnj/pLea/dQFzal6TT2A8qf
 wYhUyPTVnVKrVPR5b0wlHtLJgpNa4A1McCDsD68NY/5uKTlgfteNln1XKBMJwokL
 LoZm8iFvLEdzPL0ikslrUmbbaYJvKiedSy9ZEYbuFllY2aVpkKPuN49ZWh35IJSA
 GkiXpbirPMIjydGrq68qWd/9p4i4ltX9LfKgpcDua1NptbK2LvYe7FaeyPBYoSFP
 wsXP497vwt9g3llTzbrEHX5ZeAivW/JqemX60pF61p0Lh1UnBA+30gHQVwYgvQZa
 HUCeNMrK3dMljfHOBrWLX84lfzBi3BUQhYVzyx1kTcOAEotgxnuKH7adWAvqXlpE
 x2Ff4hDQj+xL2mbbHWNGRHBS33SNNIKIvH+kMtqzzHteDp6jROw83+HlNMw38Zl5
 s0CwiA5Wk2vAW7gFZ0FTan+a9TxpotHVgRZQZ8TLk15UTFMWoqQfBz6OdsPz38MG
 5PjouX6K4ZR9uHZiOggsjN5XZkmJro0S3Lb9mWVCXLZp0XF0e9lCw/xXiUznsXu2
 QZXcSCmiMkw=
 =SDf2
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/huth-gitlab/tags/pull-request-2021-02-19' into staging

* Always build the container images in the gitlab-CI
* Some other small gitlab-CI improvements
* Some qtest fixes

# gpg: Signature made Fri 19 Feb 2021 06:10:20 GMT
# gpg:                using RSA key 27B88847EEE0250118F3EAB92ED9D774FE702DB5
# gpg:                issuer "thuth@redhat.com"
# gpg: Good signature from "Thomas Huth <th.huth@gmx.de>" [full]
# gpg:                 aka "Thomas Huth <thuth@redhat.com>" [full]
# gpg:                 aka "Thomas Huth <huth@tuxfamily.org>" [full]
# gpg:                 aka "Thomas Huth <th.huth@posteo.de>" [unknown]
# Primary key fingerprint: 27B8 8847 EEE0 2501 18F3  EAB9 2ED9 D774 FE70 2DB5

* remotes/huth-gitlab/tags/pull-request-2021-02-19:
  travis.yml: Limit simultaneous jobs to 3
  gitlab-ci.yml: Run check-tcg with TCI
  tests/qtest/boot-sector: Check that the guest did not panic
  gitlab-ci: Disable vhost-kernel in build-disable job
  scripts/checkpatch: Improve the check for authors mangled by the mailing list
  gitlab-ci: Display Avocado log content when tests timeout
  gitlab: fix inconsistent indentation
  gitlab: add fine grained job deps for all build jobs
  gitlab: always build container images
  tests/qtest/boot-serial-test: Test Virt machine with 'max'

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-02-19 14:54:52 +00:00
Thomas Huth
7b59d10e23 scripts/checkpatch: Improve the check for authors mangled by the mailing list
There were recently some patches on the list which had their "From:"
line mangled like this:

 From: qemu_oss--- via <qemu-devel@nongnu.org>

Since our test in the checkpatch.pl script did not trigger here, the
patches finally also ended up in a pull request, with the wrong author
set. So let's improve the regular expression to also complain on
these new patterns, too.

Message-Id: <20210216071512.1199827-1-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Christian Schoenebeck <qemu_oss@crudebyte.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-02-19 06:29:05 +01:00
Eduardo Habkost
1a14d4e16a device-crash-test: Remove problematic language
Replace "whitelist" in the device-crash-test script with
"rule list".

I'm using "rule list" instead of "allow list" or "pass list"
because the list is not used only for expected/allowed errors.
It also contain rules specifying which errors shouldn't be
ignored and/or should be fatal.

Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Acked-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20210202191207.4103973-1-ehabkost@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2021-02-18 18:34:47 -05:00
John Snow
9b77d94699 qapi/introspect.py: set _gen_tree's default ifcond argument to ()
We don't need to create an empty, mutable list to pass to _gen_tree;
since it is now typed as a Sequence, we can use the empty tuple as a
default and omit the argument.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-19-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:51:14 +01:00
John Snow
cea53c31b7 qapi/introspect.py: Type _gen_tree variants as Sequence[str]
Optional[List] is clunky; an empty sequence can more elegantly convey
"no variants". By downgrading "List" to "Sequence", we can also accept
tuples; this is useful for the empty tuple specifically, which we may
use as a default parameter because it is immutable.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-18-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Doc string touched up]
Reviewed-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:51:08 +01:00
John Snow
cf26906c4a qapi/introspect.py: Update copyright and authors list
To reflect the work that went into strictly typing introspect.py,
punish myself by claiming credit.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-17-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:50:26 +01:00
John Snow
5444dedfc7 qapi/introspect.py: Add docstrings to _gen_tree and _tree_to_qlit
Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-16-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Doc string improvements squashed in]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:50:23 +01:00
John Snow
82b52f6b84 qapi/introspect.py: add type hint annotations
NB: The type aliases (SchemaInfo et al) declare intent for some of the
"dictly-typed" objects we pass around in introspect.py. They do not
enforce the shape of those objects, and cannot, until Python 3.7 or
later. (And even then, it may not be "worth it".)

Annotations are also added to the QAPISchemaEntity __init__ method in
schema.py to allow mypy to statically prove the type of typ.name,
needed to prove the return type of
QAPISchemaGenIntrospectVisitor._use_type().

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-15-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Note on QAPISchemaEntity.__init__() squashed into commit message,
Comment wrapped to conform to PEP 8]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:45:20 +01:00
John Snow
cf5db2142b qapi/introspect.py: remove _gen_variants helper
It is easier to give a name to all of the dictly-typed objects we pass
around in introspect.py by removing this helper, as it does not return
an object that has any knowable type by itself.

Inline it into its only caller instead.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-14-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:45:20 +01:00
John Snow
c0e8d9f3c1 qapi/introspect.py: improve readability of _tree_to_qlit
Subjective, but I find getting rid of the comprehensions helps. Also,
divide the sections into scalar and non-scalar sections, and remove
old-style string formatting.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-13-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:45:20 +01:00
John Snow
2a6c161bbf qapi/introspect.py: improve _tree_to_qlit error message
Trivial; make the error message just a pinch more explicit in case we
trip this by accident in the future.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-12-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:45:20 +01:00
John Snow
4f7f97a7b3 qapi/introspect.py: create a typed 'Annotated' data strutcure
Presently, we use a tuple to attach a dict containing annotations
(comments and compile-time conditionals) to a tree node. This is
undesirable because dicts are difficult to strongly type; promoting it
to a real class allows us to name the values and types of the
annotations we are expecting.

In terms of typing, the Annotated<T> type serves as a generic container
where the annotated node's type is preserved, allowing for greater
specificity than we'd be able to provide without a generic.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-11-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:45:17 +01:00
John Snow
9db2734613 qapi/introspect.py: Introduce preliminary tree typing
The types will be used in forthcoming patches to add typing. These types
describe the layout and structure of the objects passed to
_tree_to_qlit, but lack the power to describe annotations until the next
commit.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-10-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:38:00 +01:00
John Snow
d4c5b429fd qapi/introspect.py: Always define all 'extra' dict keys
This mimics how a typed object works, where 'if' and 'comment' are
always set, regardless of if they have a value set or not.

It is safe to do this because of the way that _tree_to_qlit processes
these values (using dict.get with a default of None), resulting in no
change of output from _tree_to_qlit. There are no other users of this
data.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-9-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:37:54 +01:00
John Snow
5f50cede4a qapi/introspect.py: replace 'extra' dict with 'comment' argument
This is only used to pass in a dictionary with a comment already set, so
skip the runaround and just accept the (optional) comment.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-8-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:37:49 +01:00
John Snow
84cf09915f qapi/introspect.py: Unify return type of _make_tree()
Returning two different types conditionally can be complicated to
type. Return one type for consistency.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-7-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:37:41 +01:00
John Snow
055569603d qapi/introspect.py: guard against ifcond/comment misuse
_tree_to_qlit is called recursively on dict values (isolated from their
keys); at such a point in generating output it is too late to apply an
ifcond. Similarly, comments do not necessarily have a "tidy" place they
can be printed in such a circumstance.

Forbid this usage by renaming "suppress_first_indent" to "dict_value" to
emphasize that indents are suppressed only for the benefit of dict
values; then add an assertion assuring we do not pass ifcond/comments
in this case.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-6-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Comment wrapped to conform to PEP 8]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:37:28 +01:00
John Snow
84bece7dd4 qapi/introspect.py: add _gen_features helper
_make_tree might receive a dict (a SchemaInfo object) or some other type
(usually, a string) for its obj parameter. Adding features information
should arguably be performed by the caller at such a time when we know
the type of the object and don't have to re-interrogate it.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-5-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:37:20 +01:00
John Snow
d70f5130f6 qapi/introspect.py: use _make_tree for features nodes
At present, we open-code this in _make_tree itself; but if the structure
of the tree changes, this is brittle. Use an explicit recursive call to
_make_tree when appropriate to help keep the interior node typing
consistent.

A consequence of doing this is that the 'ifcond' key of the features
dict will be omitted when ifcond is false-ish, just like it is omitted
in top-level calls to _make_tree. This also increases consistency in our
handling of this property.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-4-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:37:13 +01:00
John Snow
6b67bcac0f qapi/introspect.py: assert schema is not None
The introspect visitor is stateful, but expects that it will have a
schema to refer to. Add assertions that state this.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-3-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:35:11 +01:00
Taylor Simpson
3e7a84eecc Hexagon build infrastructure
Add file to default-configs
Add hexagon to meson.build
Add hexagon to target/meson.build
Add target/hexagon/meson.build
Change scripts/qemu-binfmt-conf.sh

We can build a hexagon-linux-user target and run programs on
the Hexagon scalar core.  With hexagon-linux-clang installed,
"make check-tcg" will pass.

Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Taylor Simpson <tsimpson@quicinc.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <1612763186-18161-35-git-send-email-tsimpson@quicinc.com>
[rth: Use top-level python variable]
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-02-18 08:25:06 -08:00
John Snow
2184bca7b1 qapi: Replace List[str] with Sequence[str] for ifcond
It does happen to be a list (as of now), but we can describe it in more
general terms with no loss in accuracy to allow tuples and other
constructs.

In the future, we can write "ifcond: Sequence[str] = ()" as a default
parameter, which we could not do safely with a Mutable type like a List.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-2-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Commit message tweaked]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 17:10:29 +01:00
Taylor Simpson
d2a56bd242 Hexagon (linux-user/hexagon) Linux user emulation
Implementation of Linux user emulation for Hexagon
Some common files modified in addition to new files in linux-user/hexagon

Acked-by: Laurent Vivier <laurent@vivier.eu>
Signed-off-by: Taylor Simpson <tsimpson@quicinc.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <1612763186-18161-31-git-send-email-tsimpson@quicinc.com>
[rth: Fix termbits.h on review by Laurent]
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-02-18 07:48:22 -08:00
Philippe Mathieu-Daudé
04250c69bc travis.yml: Move gprof/gcov test across to gitlab
Similarly to commit 8cdb2cef3f, move the gprof/gcov test to GitLab.

The coverage-summary.sh script is not Travis-CI specific, make it
generic.

[thuth: Add gcovr and bsdmainutils which are required for the
        coverage-summary.sh script to the ubuntu docker file,
        and use 'check' as test target]

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20201108204535.2319870-10-philmd@redhat.com>
Message-Id: <20210211045455.456371-2-thuth@redhat.com>
Message-Id: <20210211122750.22645-2-alex.bennee@linaro.org>
2021-02-15 09:38:19 +00:00
Stefan Hajnoczi
7a53cd20e7 get_maintainer: update repo URL to GitLab
qemu.org is running out of bandwidth and the QEMU project is moving
towards a gating CI on GitLab. Use the GitLab repos instead of qemu.org
(they will become mirrors).

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-id: 20210111115017.156802-7-stefanha@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2021-02-09 20:53:56 +00:00
Peter Maydell
41d306ec7d * Fuzzing improvements (Qiuhao, Alexander)
* i386: Fix BMI decoding for instructions with the 0x66 prefix (David)
 * initial attempt at fixing event_notifier emulation (Maxim)
 * i386: PKS emulation, fix for "qemu-system-i386 -cpu host" (myself)
 * meson: RBD test fixes (myself)
 * meson: TCI warnings (Philippe)
 * Leaner build for --disable-guest-agent, --disable-system and
   --disable-tools (Philippe, Stefan)
 * --enable-tcg-interpreter fix (Richard)
 * i386: SVM feature bits (Wei)
 * KVM bugfix (Thomas H.)
 * Add missing MemoryRegionOps callbacks (PJP)
 -----BEGIN PGP SIGNATURE-----
 
 iQFIBAABCAAyFiEE8TM4V0tmI4mGbHaCv/vSX3jHroMFAmAhR4cUHHBib256aW5p
 QHJlZGhhdC5jb20ACgkQv/vSX3jHroOviAf/Ymk/KwHZKySVqHOjDZnvP5PXMAru
 p1zlMRLAorXK+CTbshkIliaQyD8ggzT4HCinJ2NisdfTWMmlWbgr8gahNqKyZ5UG
 HlL28va3dvGhelswh/CNso1ZhVb2Q+aAYn/c6LXQva2r0xi26ohJTkIkSCPP/bnI
 +73dGzwAilBOsBVbn4cCm/70XtwDpPkw41IZIDoy/4lhL8ZdpHMz8oOjNIlOdlcU
 aEDfM8vYE4C70OtUlRZ1OwVxzcjS1Bf6dQYcpg5gAKy/jAAqR+v2PStxXiUuj5D3
 cAzd03Goh78Wcre+CbWxDKGcGtiooUT+J09wmvDPYVUHcpQMbumf4MufrQ==
 =INB5
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/bonzini-gitlab/tags/for-upstream' into staging

* Fuzzing improvements (Qiuhao, Alexander)
* i386: Fix BMI decoding for instructions with the 0x66 prefix (David)
* initial attempt at fixing event_notifier emulation (Maxim)
* i386: PKS emulation, fix for "qemu-system-i386 -cpu host" (myself)
* meson: RBD test fixes (myself)
* meson: TCI warnings (Philippe)
* Leaner build for --disable-guest-agent, --disable-system and
  --disable-tools (Philippe, Stefan)
* --enable-tcg-interpreter fix (Richard)
* i386: SVM feature bits (Wei)
* KVM bugfix (Thomas H.)
* Add missing MemoryRegionOps callbacks (PJP)

# gpg: Signature made Mon 08 Feb 2021 14:15:35 GMT
# gpg:                using RSA key F13338574B662389866C7682BFFBD25F78C7AE83
# gpg:                issuer "pbonzini@redhat.com"
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>" [full]
# gpg:                 aka "Paolo Bonzini <pbonzini@redhat.com>" [full]
# Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4  E2F7 7E15 100C CD36 69B1
#      Subkey fingerprint: F133 3857 4B66 2389 866C  7682 BFFB D25F 78C7 AE83

* remotes/bonzini-gitlab/tags/for-upstream: (46 commits)
  target/i386: Expose VMX entry/exit load pkrs control bits
  target/i386: Add support for save/load IA32_PKRS MSR
  imx7-ccm: add digprog mmio write method
  tz-ppc: add dummy read/write methods
  spapr_pci: add spapr msi read method
  nvram: add nrf51_soc flash read method
  prep: add ppc-parity write method
  vfio: add quirk device write method
  pci-host: designware: add pcie-msi read method
  hw/pci-host: add pci-intack write method
  cpu-throttle: Remove timer_mod() from cpu_throttle_set()
  replay: rng-builtin support
  pc-bios/descriptors: fix paths in json files
  replay: fix replay of the interrupts
  accel/kvm/kvm-all: Fix wrong return code handling in dirty log code
  qapi/meson: Restrict UI module to system emulation and tools
  qapi/meson: Restrict system-mode specific modules
  qapi/meson: Remove QMP from user-mode emulation
  qapi/meson: Restrict qdev code to system-mode emulation
  meson: Restrict emulation code
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-02-09 10:04:51 +00:00
Peter Maydell
2436651b26 Migration pull 2021-02-08
v2
   Dropped vmstate: Fix memory leak in vmstate_handle_alloc
     Broke on Power
   Added migration: only check page size match if RAM postcopy is enabled
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEERfXHG0oMt/uXep+pBRYzHrxb/ecFAmAhIE4ACgkQBRYzHrxb
 /ecPuA/+Pgo++1ZSseJUgbLePwyTVc0jahdcvYEDmLUn8UM6ikBcBXBgUKHdkFW3
 bjSSVgB/xxvXSiafBK4xFNrCqSgqMSr3DJcHmvWgv2wVARcYf6Z26Da53LZq1Qru
 0tvRyb40Od1f9zb8Zj7e2Y3pjQ9ybLLbjfNhgnOBbQivqWkjZI31oV2KUCWY2+eV
 T1BEwr6mgYepqhmeB6OvQZtaQVC5toirS6NajNF4nt0vZEIGIvK6/A9erCVU8Tze
 5ch1J0MUqgc3q6ZSE/I9BHEy6MaL0X8G6H+ezjxdoRQtbt1iM/YqZJCSrXkAxiLC
 ROohryb6qVk26+UYuana79faLwrw359WlkwNEE6SEIRSENu+6p7bgN3LZuCILCO7
 xJEkeTgy6r40IGCkDC9aWa8pyLHpNX9gyLpGBHdIRD6zEOWaKNtzh7E2uo/T0ann
 BpcfgQOsYN25hIHiiXnxozUREbx71VDfMq7GqGB6eC3u2+a3U6jpSJb1nNq5NB89
 FJYLZy5Rbuy7OStMwfMsxRs7E63XvGgnwrN8FczU/pumCPX4lDYIpnocqinUmP8p
 XubRQQVaVDSKIq1mvzw7iR/1NsP9vfYvnrAIv941f38NBmDKqdPuMOXR/qB/Kp2Y
 jB7b1L5/JcXbWsQmK7fda9jmPzFwSO2cTeTiUonk9RfuuDEws0A=
 =4tbe
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/dgilbert/tags/pull-migration-20210208a' into staging

Migration pull 2021-02-08

v2
  Dropped vmstate: Fix memory leak in vmstate_handle_alloc
    Broke on Power
  Added migration: only check page size match if RAM postcopy is enabled

# gpg: Signature made Mon 08 Feb 2021 11:28:14 GMT
# gpg:                using RSA key 45F5C71B4A0CB7FB977A9FA90516331EBC5BFDE7
# gpg: Good signature from "Dr. David Alan Gilbert (RH2) <dgilbert@redhat.com>" [full]
# Primary key fingerprint: 45F5 C71B 4A0C B7FB 977A  9FA9 0516 331E BC5B FDE7

* remotes/dgilbert/tags/pull-migration-20210208a: (27 commits)
  migration: only check page size match if RAM postcopy is enabled
  migration: introduce snapshot-{save, load, delete} QMP commands
  iotests: fix loading of common.config from tests/ subdir
  iotests: add support for capturing and matching QMP events
  migration: introduce a delete_snapshot wrapper
  migration: wire up support for snapshot device selection
  migration: control whether snapshots are ovewritten
  block: rename and alter bdrv_all_find_snapshot semantics
  block: allow specifying name of block device for vmstate storage
  block: add ability to specify list of blockdevs during snapshot
  migration: stop returning errno from load_snapshot()
  migration: Make save_snapshot() return bool, not 0/-1
  block: push error reporting into bdrv_all_*_snapshot functions
  migration: Display the migration blockers
  migration: Add blocker information
  migration: Fix a few absurdly defective error messages
  migration: Fix cache_init()'s "Failed to allocate" error messages
  migration: Clean up signed vs. unsigned XBZRLE cache-size
  migration: Fix migrate-set-parameters argument validation
  migration: introduce 'userfaultfd-wrlat.py' script
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-02-08 18:23:47 +00:00
Peter Maydell
4f799257b3 QAPI patches patches for 2021-02-08
-----BEGIN PGP SIGNATURE-----
 
 iQJGBAABCAAwFiEENUvIs9frKmtoZ05fOHC0AOuRhlMFAmAhQpISHGFybWJydUBy
 ZWRoYXQuY29tAAoJEDhwtADrkYZTBFYP/jh1x1Isn35q3jwjzKy6N3EYbzmaRKhW
 rc5bFlZLQ95Nso+Q3b8izUSeoGz7t8E+WLZBBxzNoOWBS2uC8HS0jIpUmCJaB+Yu
 aVrG70+5/s/BPTX7iF6LIrcRj3GIQdFEWR5Zn1V4PFk5G7NZFwEiTnViq8pLw87H
 c2ktAorDyH6Zl4DA2YjV0XBLPMclt53sQ/M5b6ythUq6E05bVUkQFiUm3N8joIEX
 2djwumVi/dYpDwR/pVPBjPXcP5gc44y4D9sjJU6I16kyCyKi8Tr9mmEcSJFmR/Y/
 GdSilmtFhxUikAQogZwi/48BhR0GawhgcKZP5IkKNjvfwqE13Vhx8/p0pEubFPUE
 YexvFqkSmBRZ1mT0QpCcG4lp+u6PzhTyc/gbWwPm42/T+x9fG0cDgvDZCPjIRSsg
 LiUmmqrwOXr+Lw6GP1Q0f0KKv5QhCfeq1YcYrEXTQsa1PDT5T8ARBzxR5O1X2UNR
 Xw7j4u9R63p6P5nOk5+wwRLQkUkGl7N0SYqe4thyHUfW4r48V5J9RT7ONKD2CXO9
 Dw/Q3Ga1GmHaydpoZ4Az1k5kyr5dVBbAISRRWRpYA0sneg85o3RU3aTMt3r43pzC
 5pjeSpx6v7K4Y3NJPL1e/j/qJZq10WtEslkF/TKcBS2qLgiKCZ1oXQPORQxxX9uK
 zuK8oPQR7W42
 =uLm+
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/armbru/tags/pull-qapi-2021-02-08' into staging

QAPI patches patches for 2021-02-08

# gpg: Signature made Mon 08 Feb 2021 13:54:26 GMT
# gpg:                using RSA key 354BC8B3D7EB2A6B68674E5F3870B400EB918653
# gpg:                issuer "armbru@redhat.com"
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>" [full]
# gpg:                 aka "Markus Armbruster <armbru@pond.sub.org>" [full]
# Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867  4E5F 3870 B400 EB91 8653

* remotes/armbru/tags/pull-qapi-2021-02-08:
  qapi: enable strict-optional checks
  qapi: type 'info' as Optional[QAPISourceInfo]
  qapi/gen: Drop support for QAPIGen without a file name
  qapi/commands: Simplify command registry generation
  qapi/gen: Support switching to another module temporarily
  qapi/gen: write _genc/_genh access shims
  qapi: centralize the built-in module name definition
  qapi/gen: Combine ._add_[user|system]_module
  qapi: use './builtin' as the built-in module name
  qapi: use explicitly internal module names
  qapi/gen: Replace ._begin_system_module()
  qapi: centralize is_[user|system|builtin]_module methods
  qapi/gen: inline _wrap_ifcond into end_if()
  qapi/main: handle theoretical None-return from re.match()
  qapi/events: fix visit_event typing
  qapi/commands: assert arg_type is not None

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-02-08 16:12:21 +00:00
Qiuhao Li
487a1d13ba fuzz: fix wrong index in clear_bits
Signed-off-by: Qiuhao Li <Qiuhao.Li@outlook.com>
Message-Id: <SYCPR01MB3502E9F6EB06DEDCD484F738FCBA9@SYCPR01MB3502.ausprd01.prod.outlook.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-02-08 14:43:54 +01:00
John Snow
c51172667b qapi: enable strict-optional checks
In the modules that we are checking so far, we can be stricter about the
difference between Optional[T] and T types. Enable that check.

Enabling it now will assist review on further typing and cleanup work.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-17-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-08 14:15:58 +01:00
John Snow
4a82e468e7 qapi: type 'info' as Optional[QAPISourceInfo]
For everything typed so far, type this parameter as
Optional[QAPISourceInfo].

In the most generic case, QAPISchemaEntity's info field may be None to
represent types that come from built-in definitions. Although some
Entity types may not currently have any built-in definitions, it is not
easily possible to constrain the type except on an ad-hoc basis using
assertions.

It's easier and simpler, then, to just say it's always an Optional type.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-16-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-08 14:15:58 +01:00
Markus Armbruster
cc0747f6b7 qapi/gen: Drop support for QAPIGen without a file name
The previous commit removed the only user of QAPIGen(None).  Tighten
the type hint.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-15-jsnow@redhat.com>
2021-02-08 14:15:58 +01:00
Markus Armbruster
c6cd7e4151 qapi/commands: Simplify command registry generation
QAPISchemaGenCommandVisitor.visit_command() needs to generate the
marshalling function into the current module, and also generate its
registration into the ./init system module.  The latter is done
somewhat awkwardly: .__init__() creates a QAPIGenCCode that will not
be written out, each .visit_command() adds its registration to it, and
.visit_end() copies its contents into the ./init module it creates.

Instead provide the means to temporarily switch to another module.
Create the ./init module in .visit_begin(), and generate its initial
part.  Add registrations to it in .visit_command().  Finish it in
.visit_end().

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-14-jsnow@redhat.com>
2021-02-08 14:15:58 +01:00
Markus Armbruster
d921d27c1b qapi/gen: Support switching to another module temporarily
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-13-jsnow@redhat.com>
[Commit message tweaked]
2021-02-08 14:15:58 +01:00
John Snow
fd9b160384 qapi/gen: write _genc/_genh access shims
Many places assume they can access these fields without checking them
first to ensure they are defined. Eliminating the _genc and _genh fields
and replacing them with functional properties that check for correct
state can ease the typing overhead by eliminating the Optional[T] return
type.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-12-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-08 14:15:58 +01:00
John Snow
39b2d838f1 qapi: centralize the built-in module name definition
Use a constant to make it obvious we're referring to a very specific thing.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-11-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-08 14:15:58 +01:00
Markus Armbruster
4ab0ff6da0 qapi/gen: Combine ._add_[user|system]_module
With callers to _add_system_module now explicitly using the './' prefix
to indicate a system module, there is no longer any reason to have
separate interfaces for adding system vs user modules; use a unified
interface that differentiates based on the name.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-10-jsnow@redhat.com>
2021-02-08 14:15:58 +01:00
John Snow
e2bbc4eaa7 qapi: use './builtin' as the built-in module name
Use './builtin' as the built-in module name instead of
None. Clarify the typing that this is now always a string.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-9-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-08 14:15:58 +01:00
John Snow
12893a8ea7 qapi: use explicitly internal module names
QAPISchemaModularCVisitor._add_system_module() prefixes './' to its name
argument to make it a module name.  Pass the module name instead.  This
will allow us to coalesce the methods to add modules later on.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-8-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Commit message reworded]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-08 14:15:58 +01:00
Markus Armbruster
f3a705928a qapi/gen: Replace ._begin_system_module()
QAPISchemaModularCVisitor._begin_system_module() is actually just for
the builtin module.  Rename it to ._begin_builtin_module() and drop
its useless @name parameter.

Clarify conditionals in visit_module to make this clear.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-7-jsnow@redhat.com>
2021-02-08 14:15:58 +01:00
John Snow
98967c248c qapi: centralize is_[user|system|builtin]_module methods
Define what a module is and define what kind of a module it is once and
for all, in one place.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-6-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-08 14:15:58 +01:00
John Snow
a253b3eb9a qapi/gen: inline _wrap_ifcond into end_if()
We assert _start_if is not None in end_if, but that's opaque to mypy.
By inlining _wrap_ifcond, that constraint becomes provable to mypy.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-5-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-08 14:15:58 +01:00
John Snow
ad1218086e qapi/main: handle theoretical None-return from re.match()
Mypy cannot understand that this match can never be None, so help it
along.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-4-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-08 14:15:58 +01:00
John Snow
3cc01c546b qapi/events: fix visit_event typing
Actually, the arg_type can indeed be Optional.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-3-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-08 14:15:58 +01:00
John Snow
ec9697ab3f qapi/commands: assert arg_type is not None
When boxed is True, expr.py asserts that we must have
arguments. Ultimately, this should mean that if boxed is True that
arg_type should be defined. Mypy cannot infer this, and does not support
'stateful' type inference, e.g.:

```
if x:
    assert y is not None

...

if x:
    y.etc()
```

does not work, because mypy does not statefully remember the conditional
assertion in the second block. Help mypy out by creating a new local
that it can track more easily.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-2-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-08 14:15:58 +01:00
Andrey Gruzdev
c7243566d0 migration: introduce 'userfaultfd-wrlat.py' script
Add BCC/eBPF script to analyze userfaultfd write fault latency distribution.

Signed-off-by: Andrey Gruzdev <andrey.gruzdev@virtuozzo.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Message-Id: <20210129101407.103458-6-andrey.gruzdev@virtuozzo.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2021-02-08 11:19:51 +00:00
Alex Bennée
47e3424ac9 scripts/mtest2make.py: export all-%s-targets variable and use it
There are some places where the conditional makefile support is the
simplest solution. Now we don't expose CONFIG_TCG as a variable create
a new one that can be checked for the check-help output.

As check-tcg is a PHONY target we re-use check-softfloat to gate that
as well.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20210202134001.25738-13-alex.bennee@linaro.org>
2021-02-08 10:55:20 +00:00
Volker Rümelin
c6e93c9d62 simpletrace: build() missing 2 required positional arguments
Commit 4e66c9ef64 "tracetool: add input filename and line number to
Event" forgot to add a line number and a filename argument at one
build method call site.

Traceback (most recent call last):
  File "./scripts/simpletrace.py", line 261, in <module>
    run(Formatter())
  File "./scripts/simpletrace.py", line 236, in run
    process(events, sys.argv[2], analyzer, read_header=read_header)
  File "./scripts/simpletrace.py", line 177, in process
    dropped_event =
      Event.build("Dropped_Event(uint64_t num_events_dropped)")
TypeError: build() missing 2 required positional arguments:
  'lineno' and 'filename'

Add the missing arguments.

Signed-off-by: Volker Rümelin <vr_qemu@t-online.de>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-id: 20210131173415.3392-1-vr_qemu@t-online.de
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2021-02-01 10:51:00 +00:00
Stefan Hajnoczi
418ed14268 trace: make the 'log' backend timestamp configurable
Timestamps in tracing output can be distracting. Make it possible to
control tid/timestamp printing with -msg timestamp=on|off. The default
is no tid/timestamps. Previously they were always printed.

Suggested-by: BALATON Zoltan <balaton@eik.bme.hu>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: BALATON Zoltan <balaton@eik.bme.hu>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-id: 20210125113507.224287-3-stefanha@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2021-02-01 10:50:55 +00:00
Daniel P. Berrangé
09612de7e9 tracetool: also strip %l and %ll from systemtap format strings
All variables are 64-bit and so %l / %ll are not required, and the
latter is actually invalid:

  $ sudo stap -e 'probe begin{printf ("BEGIN")}'  -I .
  parse error: invalid or missing conversion specifier
          saw: operator ',' at ./qemu-system-x86_64-log.stp:15118:101
       source:     printf("%d@%d vhost_vdpa_set_log_base dev: %p base: 0x%x size: %llu
refcnt: %d fd: %d log: %p\n", pid(), gettimeofday_ns(), dev, base, size, refcnt, fd, log)

                       ^

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Laurent Vivier <lvivier@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Laurent Vivier <lvivier@redhat.com>
Message-id: 20210106130239.1004729-1-berrange@redhat.com

[Fixed "simiarly" typo found by Laurent Vivier <lvivier@redhat.com>
--Stefan]

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2021-02-01 10:50:55 +00:00
Laurent Vivier
9f4e519fd7 tracetool: fix "PRI" macro decoding
macro is not reset after use, so the format decoded is always the
one of the first "PRI" in the format string.

For instance:

  vhost_vdpa_set_config(void *dev, uint32_t offset, uint32_t size, \
                        uint32_t flags) "dev: %p offset: %"PRIu32" \
                        size: %"PRIu32" flags: 0x%"PRIx32

generates:

  printf("%d@%d vhost_vdpa_set_config dev: %p offset: %u size: %u \
          flags: 0x%u\n", pid(), gettimeofday_ns(), dev, offset, \
          size, flags)

for the "flags" parameter, we can see a "0x%u" rather than a "0x%x"
because the first macro was "PRIu32" (for offset).

In the loop, macro becomes "PRIu32PRIu32PRIx32", and c_macro_to_format()
returns always macro[3] ('u' in this case). This patch resets macro after
the format has been decoded.

Signed-off-by: Laurent Vivier <lvivier@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-id: 20210105191721.120463-3-lvivier@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2021-02-01 10:50:54 +00:00
Peter Maydell
74208cd252 * Replace --enable/disable-git-update with --with-git-submodules
to allow improved control over use of git submodules
 * Deprecate the -enable-fips option
 * Ensure docs use prefer format for bool options
 * Clarify platform support rules
 * Misc fixes to keymap conversions
 * Fix misc problems on macOS
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEE2vOm/bJrYpEtDo4/vobrtBUQT98FAmAUQXUACgkQvobrtBUQ
 T9/9Zw//VV3z7MI70HjWk17TUmkKLGFbmrc3xvtS6EMZbhfXpsR8qMfSjoRxP4Gn
 CgeftOa/lK5XjL7FcEUmcZeO8jZNE/+S5wgP2upP5Ae1zuDeyaSPTGR6UYs7WqSa
 P1kQsot2sAsZCP7Lko0jSv7rEDInK1bcdWr0a/xR5M2TvKiXoEdbElIZdwe3yLbC
 qWPWiv2pp6z2eGtJK/9VBxXjP0tlkNAR+jz8p/8RSZEEKf3aSq6HvVae+bHU86nX
 9t4baZk3ZuuwkmHJrHJP/72BjjitqEOiydtSsOCVYmY2GOclYeTnezG/vm75cGIK
 ej6hwFGElAEQk+KU9g7HOXi1eqXncSBjP0xWqc8PaZLW7wW/Uv8UV94fk+SCkIQ0
 GVBBLP8PzySL+w33NN7sv14cWeBpsjy9arMr4okZ80klFlS2hx/FqnUmhFhoN8ub
 ptIPUcSJoL+6pjxatV2yJ6bHkVga9PiwtLC8/rqETYPJz0b756xlPaEGd+0B1P0V
 ZMraE7lCgS0kJM6w0xF3R7JlGVtKouj4ZePCK5MGjpByBAK/vn/MbhbEmcyh+uJD
 r6sakKwqLjDLKdmQDd+biq3q6mvKE8ZKaUj8dXYdvxXSxxqXZ8DnrNpj/UkDTi3S
 chlTT1qUYrp2MVYqGmFjZ0XcEdEUR1I39rUs+wRxE3RgtE56M1A=
 =g6bC
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/berrange-gitlab/tags/misc-fixes-pull-request' into staging

* Replace --enable/disable-git-update with --with-git-submodules
  to allow improved control over use of git submodules
* Deprecate the -enable-fips option
* Ensure docs use prefer format for bool options
* Clarify platform support rules
* Misc fixes to keymap conversions
* Fix misc problems on macOS

# gpg: Signature made Fri 29 Jan 2021 17:10:13 GMT
# gpg:                using RSA key DAF3A6FDB26B62912D0E8E3FBE86EBB415104FDF
# gpg: Good signature from "Daniel P. Berrange <dan@berrange.com>" [full]
# gpg:                 aka "Daniel P. Berrange <berrange@redhat.com>" [full]
# Primary key fingerprint: DAF3 A6FD B26B 6291 2D0E  8E3F BE86 EBB4 1510 4FDF

* remotes/berrange-gitlab/tags/misc-fixes-pull-request:
  tests: Replace deprecated ASN1 code
  tests: Fix runtime error in test-authz-pam
  ui: update keycodemapdb submodule commit
  crypto: Add spaces around operator
  configure: replace --enable/disable-git-update with --with-git-submodules
  docs: fix missing backslash in certtool shell example
  docs: simplify and clarify the platform support rules
  Prefer 'on' | 'off' over 'yes' | 'no' for bool options
  os: deprecate the -enable-fips option and QEMU's FIPS enforcement
  crypto: Fix memory leaks in set_loaded for tls-*
  crypto: Forbid broken unloading of secrets
  crypto: Move USER_CREATABLE to secret_common base class
  crypto: Fix some code style problems, add spaces around operator

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-01-29 19:51:25 +00:00
Dan Streetman
7d7dbf9dc1 configure: replace --enable/disable-git-update with --with-git-submodules
Replace the --enable-git-update and --disable-git-update configure params
with the param --with-git-submodules=(update|validate|ignore) to
allow 3 options for building from a git repo.

This is needed because downstream packagers, e.g. Debian, Ubuntu, etc,
also keep the source code in git, but do not want to enable the
'git_update' mode; with the current code, that's not possible even
if the downstream package specifies --disable-git-update.

The previous parameters are deprecated but still available; the
--enable-git-update parameter maps to --with-git-submodules=update and
--disable-git-update parameter maps to --with-git-submodules=validate.

The configure script behavior is slightly modified, where previously
the dtc, capstone, and slirp submodules were not validated when
--disable-git-update was specified (but were updated with git-update
enabled), now they are validated when using --with-git-submodules=validate
and are only ignored when using --with-git-submodules=ignore.

Signed-off-by: Dan Streetman <ddstreet@canonical.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2021-01-29 17:07:53 +00:00
Alexander Graf
8a74ce618b hvf: Add hypervisor entitlement to output binaries
In macOS 11, QEMU only gets access to Hypervisor.framework if it has the
respective entitlement. Add an entitlement template and automatically self
sign and apply the entitlement in the build.

Signed-off-by: Alexander Graf <agraf@csgraf.de>
Reviewed-by: Roman Bolshakov <r.bolshakov@yadro.com>
Tested-by: Roman Bolshakov <r.bolshakov@yadro.com>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-01-29 10:47:28 +00:00
Vladimir Sementsov-Ogievskiy
c701f59253 simplebench: add bench-backup.py
Add script to benchmark new backup architecture.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20210116214705.822267-24-vsementsov@virtuozzo.com>
[mreitz: s/not unsupported/not supported/]
Signed-off-by: Max Reitz <mreitz@redhat.com>
2021-01-26 14:36:37 +01:00
Vladimir Sementsov-Ogievskiy
b2fcb0c575 simplebench: bench_block_job: add cmd_options argument
Add argument to allow additional block-job options.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-23-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2021-01-26 14:36:37 +01:00
Vladimir Sementsov-Ogievskiy
2096de521e simplebench/bench_block_job: use correct shebang line with python3
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210116214705.822267-22-vsementsov@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2021-01-26 14:36:37 +01:00
Eric Farman
ab5ec23f9c update-linux-headers: Include const.h
Kernel commit a85cbe6159ff ("uapi: move constants from
<linux/kernel.h> to <linux/const.h>") breaks our script
because of the unrecognized include. Let's add that to
our processing.

Signed-off-by: Eric Farman <farman@linux.ibm.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210104202057.48048-2-farman@linux.ibm.com>
Signed-off-by: Cornelia Huck <cohuck@redhat.com>
2021-01-21 11:19:45 +01:00
Alex Bennée
9e5acb373d scripts/checkpatch.pl: fix git-show invocation to include diffstat
Without this checkpatch keeps complaining about new/changed files even
when MAINTAINERS has been updated. Normal invocations of checkpatch on
patch files rather than commit IDs are unaffected.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210114165730.31607-13-alex.bennee@linaro.org>
2021-01-18 10:04:31 +00:00
Philippe Mathieu-Daudé
4cacecaaa2 decodetree: Open files with encoding='utf-8'
When decodetree.py was added in commit 568ae7efae, QEMU was
using Python 2 which happily reads UTF-8 files in text mode.
Python 3 requires either UTF-8 locale or an explicit encoding
passed to open(). Now that Python 3 is required, explicit
UTF-8 encoding for decodetree source files.

To avoid further problems with the user locale, also explicit
UTF-8 encoding for the generated C files.

Explicit both input/output are plain text by using the 't' mode.

This fixes:

  $ /usr/bin/python3 scripts/decodetree.py test.decode
  Traceback (most recent call last):
    File "scripts/decodetree.py", line 1397, in <module>
      main()
    File "scripts/decodetree.py", line 1308, in main
      parse_file(f, toppat)
    File "scripts/decodetree.py", line 994, in parse_file
      for line in f:
    File "/usr/lib/python3.6/encodings/ascii.py", line 26, in decode
      return codecs.ascii_decode(input, self.errors)[0]
  UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 80:
  ordinal not in range(128)

Reported-by: Peter Maydell <peter.maydell@linaro.org>
Suggested-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20210110000240.761122-1-f4bug@amsat.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-01-13 08:39:08 -10:00
Maxim Levitsky
b9a0de3773 scripts/gdb: implement 'qemu bt'
This script first runs the regular gdb's 'bt' command, and then if we are in a
coroutine it prints the coroutines backtraces in the order in which they
were called.

Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
Message-Id: <20201217155436.927320-3-mlevitsk@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-01-12 12:38:03 +01:00
Maxim Levitsky
4cbf8efc5b scripts/gdb: fix 'qemu coroutine' when users selects a non topmost stack frame
The code that dumps the stack frame works like that:
* save current registers
* overwrite current registers (including rip/rsp) with coroutine snapshot
  in the jmpbuf
* print backtrace
* restore the saved registers.

If the user has currently selected a non topmost stack frame in gdb,
the above code will still restore the selected frame registers,
but the gdb will then lose the selected frame index, which makes it impossible
to switch back to frame 0, to continue debugging the executable.

Therefore switch temporarily to the topmost frame of the stack
for the above code.

Signed-off-by: Maxim Levitsky <mlevitsk@redhat.com>
Message-Id: <20201217155436.927320-2-mlevitsk@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-01-12 12:38:03 +01:00
Qiuhao Li
4cc5752303 fuzz: heuristic split write based on past IOs
If previous write commands write the same length of data with the same step,
we view it as a hint.

Signed-off-by: Qiuhao Li <Qiuhao.Li@outlook.com>
Reviewed-by: Alexander Bulekov <alxndr@bu.edu>
Tested-by: Alexander Bulekov <alxndr@bu.edu>
Message-Id: <SYCPR01MB3502480AD07811A6A49B8FEAFCAB0@SYCPR01MB3502.ausprd01.prod.outlook.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-01-11 14:59:21 +01:00
Qiuhao Li
dd21ed0edf fuzz: add minimization options
-M1: remove IO commands iteratively
-M2: try setting bits in operand of write/out to zero

Signed-off-by: Qiuhao Li <Qiuhao.Li@outlook.com>
Reviewed-by: Alexander Bulekov <alxndr@bu.edu>
Tested-by: Alexander Bulekov <alxndr@bu.edu>
Message-Id: <SYCPR01MB350204C52E7A39E6B0EEC870FCAB0@SYCPR01MB3502.ausprd01.prod.outlook.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-01-11 14:59:21 +01:00
Qiuhao Li
9d20f2af53 fuzz: set bits in operand of write/out to zero
Simplifying the crash cases by opportunistically setting bits in operands of
out/write to zero may help to debug, since usually bit one means turn on or
trigger a function while zero is the default turn-off setting.

Tested bug https://bugs.launchpad.net/qemu/+bug/1908062

Signed-off-by: Qiuhao Li <Qiuhao.Li@outlook.com>
Reviewed-by: Alexander Bulekov <alxndr@bu.edu>
Tested-by: Alexander Bulekov <alxndr@bu.edu>
Message-Id: <SYCPR01MB3502C84B6346A3E3DE708C7BFCAB0@SYCPR01MB3502.ausprd01.prod.outlook.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-01-11 14:59:21 +01:00
Qiuhao Li
247ab240c2 fuzz: remove IO commands iteratively
Now we use a one-time scan and remove strategy in the minimizer,
which is not suitable for timing dependent instructions.

For example, instruction A will indicate an address where the config
chunk locates, and instruction B will make the configuration active.
If we have the following instruction sequence:

...
A1
B1
A2
B2
...

A2 and B2 are the actual instructions that trigger the bug.

If we scan from top to bottom, after we remove A1, the behavior of B1
might be unknowable, including not to crash the program. But we will
successfully remove B1 later cause A2 and B2 will crash the process
anyway:

...
A1
A2
B2
...

Now one more trimming will remove A1.

In the perfect case, we would need to be able to remove A and B (or C!) at
the same time. But for now, let's just add a loop around the minimizer.

Since we only remove instructions, this iterative algorithm is converging.

Tested with Bug 1908062.

Signed-off-by: Qiuhao Li <Qiuhao.Li@outlook.com>
Reviewed-by: Alexander Bulekov <alxndr@bu.edu>
Tested-by: Alexander Bulekov <alxndr@bu.edu>
Message-Id: <SYCPR01MB350263004448040ACCB9A9F1FCAB0@SYCPR01MB3502.ausprd01.prod.outlook.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-01-11 14:59:21 +01:00
Qiuhao Li
e72203abec fuzz: split write operand using binary approach
Currently, we split the write commands' data from the middle. If it does not
work, try to move the pivot left by one byte and retry until there is no
space.

But, this method has two flaws:

1. It may fail to trim all unnecessary bytes on the right side.

For example, there is an IO write command:

  write addr uuxxxxuu

u is the unnecessary byte for the crash. Unlike ram write commands, in most
case, a split IO write won't trigger the same crash, So if we split from the
middle, we will get:

  write addr uu (will be removed in next round)
  write addr xxxxuu

For xxxxuu, since split it from the middle and retry to the leftmost byte
won't get the same crash, we will be stopped from removing the last two
bytes.

2. The algorithm complexity is O(n) since we move the pivot byte by byte.

To solve the first issue, we can try a symmetrical position on the right if
we fail on the left. As for the second issue, instead moving by one byte, we
can approach the boundary exponentially, achieving O(log(n)).

Give an example:

                   xxxxuu len=6
                        +
                        |
                        +
                 xxx,xuu 6/2=3 fail
                        +
         +--------------+-------------+
         |                            |
         +                            +
  xx,xxuu 6/2^2=1 fail         xxxxu,u 6-1=5 success
                                 +   +
         +------------------+----+   |
         |                  |        +-------------+ u removed
         +                  +
   xx,xxu 5/2=2 fail  xxxx,u 6-2=4 success
                           +
                           |
                           +-----------+ u removed

In some rare cases, this algorithm will fail to trim all unnecessary bytes:

  xxxxxxxxxuxxxxxx
  xxxxxxxx-xuxxxxxx Fail
  xxxx-xxxxxuxxxxxx Fail
  xxxxxxxxxuxx-xxxx Fail
  ...

I think the trade-off is worth it.

Signed-off-by: Qiuhao Li <Qiuhao.Li@outlook.com>
Reviewed-by: Alexander Bulekov <alxndr@bu.edu>
Tested-by: Alexander Bulekov <alxndr@bu.edu>
Message-Id: <SYCPR01MB3502D26F1BEB680CBBC169E5FCAB0@SYCPR01MB3502.ausprd01.prod.outlook.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-01-11 14:59:21 +01:00
Qiuhao Li
7b339f287f fuzz: double the IOs to remove for every loop
Instead of removing IO instructions one by one, we can try deleting multiple
instructions at once. According to the locality of reference, we double the
number of instructions to remove for the next round and recover it to one
once we fail.

This patch is usually significant for large input.

Test with quadrupled trace input at:
  https://bugs.launchpad.net/qemu/+bug/1890333/comments/1

Patched 1/6 version:
  real  0m45.904s
  user  0m16.874s
  sys   0m10.042s

Refined version:
  real  0m11.412s
  user  0m6.888s
  sys   0m3.325s

Signed-off-by: Qiuhao Li <Qiuhao.Li@outlook.com>
Reviewed-by: Alexander Bulekov <alxndr@bu.edu>
Tested-by: Alexander Bulekov <alxndr@bu.edu>
Message-Id: <SYCPR01MB350280A67BB55C3FADF173E3FCAB0@SYCPR01MB3502.ausprd01.prod.outlook.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-01-11 14:59:21 +01:00
Qiuhao Li
22ec0c696f fuzz: accelerate non-crash detection
We spend much time waiting for the timeout program during the minimization
process until it passes a time limit. This patch hacks the CLOSED (indicates
the redirection file closed) notification in QTest's output if it doesn't
crash.

Test with quadrupled trace input at:
  https://bugs.launchpad.net/qemu/+bug/1890333/comments/1

Original version:
  real	1m37.246s
  user	0m13.069s
  sys	0m8.399s

Refined version:
  real	0m45.904s
  user	0m16.874s
  sys	0m10.042s

Note:

Sometimes the mutated or the same trace may trigger a different crash
summary (second-to-last line) but indicates the same bug. For example, Bug
1910826 [1], which will trigger a stack overflow, may output summaries
like:

SUMMARY: AddressSanitizer: stack-overflow
/home/qiuhao/hack/qemu/build/../softmmu/physmem.c:488 in
flatview_do_translate

or

SUMMARY: AddressSanitizer: stack-overflow
(/home/qiuhao/hack/qemu/build/qemu-system-i386+0x27ca049) in __asan_memcpy

Etc.

If we use the whole summary line as the token, we may be prevented from
further minimization. So in this patch, we only use the first three words
which indicate the type of crash:

SUMMARY: AddressSanitizer: stack-overflow

[1] https://bugs.launchpad.net/qemu/+bug/1910826

Signed-off-by: Qiuhao Li <Qiuhao.Li@outlook.com>
Reviewed-by: Alexander Bulekov <alxndr@bu.edu>
Tested-by: Alexander Bulekov <alxndr@bu.edu>
Message-Id: <SYCPR01MB350251DC04003450348FAF68FCAB0@SYCPR01MB3502.ausprd01.prod.outlook.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-01-11 14:59:21 +01:00
Peter Maydell
01b3e68bb1 scripts/coccinelle: New script to remove unnecessary timer_del() calls
Now that timer_free() implicitly calls timer_del(), sequences
  timer_del(mytimer);
  timer_free(mytimer);

can be simplified to just
  timer_free(mytimer);

Add a Coccinelle script to do this transformation.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20201215154107.3255-3-peter.maydell@linaro.org
2021-01-08 15:13:38 +00:00
Stefan Hajnoczi
7fb48c0ee1 tracetool: show trace-events filename/lineno in fmt string errors
The compiler encounters trace event format strings in generated code.
Format strings are error-prone and therefore clear compiler errors are
important.

Use the #line directive to show the trace-events filename and line
number in format string errors:
https://gcc.gnu.org/onlinedocs/gcc-10.2.0/cpp/Line-Control.html

For example, if the cpu_in trace event's %u is changed to %p the
following error is reported:

  trace-events:29:18: error: format ‘%p’ expects argument of type ‘void *’, but argument 7 has type ‘unsigned int’ [-Werror=format=]

Line 29 in trace-events is where cpu_in is defined. This works for any
trace-events file in the QEMU source tree and the correct path is
displayed.

Unfortunately there does not seem to be a way to set the column, so "18"
is not the right character on that line.

Suggested-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Message-Id: <20200827142915.108730-5-stefanha@redhat.com>
2021-01-04 14:24:58 +00:00
Stefan Hajnoczi
4e66c9ef64 tracetool: add input filename and line number to Event
Store the input filename and line number in Event.

A later patch will use this to improve error messages.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Message-Id: <20200827142915.108730-4-stefanha@redhat.com>
2021-01-04 14:24:58 +00:00
Stefan Hajnoczi
294170c1dd tracetool: add out_lineno and out_next_lineno to out()
Make the output file line number and next line number available to
out().

A later patch will use this to improve error messages.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Message-Id: <20200827142915.108730-3-stefanha@redhat.com>
2021-01-04 14:24:58 +00:00
Stefan Hajnoczi
c05012a365 tracetool: add output filename command-line argument
The tracetool.py script writes to stdout. This means the output filename
is not available to the script. Add the output filename to the
command-line so that the script has access to the filename.

This also simplifies the tracetool.py invocation. It's no longer
necessary to use meson's custom_build(capture : true) to save output.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Message-Id: <20200827142915.108730-2-stefanha@redhat.com>
2021-01-04 14:24:58 +00:00
Vladimir Sementsov-Ogievskiy
cff6d3ca43 scripts/simplebench: add bench_prealloc.py
Benchmark for new preallocate filter.

Example usage:
    ./bench_prealloc.py ../../build/qemu-img \
        ssd-ext4:/path/to/mount/point \
        ssd-xfs:/path2 hdd-ext4:/path3 hdd-xfs:/path4

The benchmark shows performance improvement (or degradation) when use
new preallocate filter with qcow2 image.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20201021145859.11201-22-vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-12-18 12:35:55 +01:00
Vladimir Sementsov-Ogievskiy
181f60c8c7 simplebench/results_to_text: make executable
Make results_to_text a tool to dump results saved in JSON file.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20201021145859.11201-21-vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-12-18 12:35:55 +01:00
Vladimir Sementsov-Ogievskiy
aa362403f4 simplebench/results_to_text: add difference line to the table
Performance improvements / degradations are usually discussed in
percentage. Let's make the script calculate it for us.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20201021145859.11201-20-vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
[mreitz: 'seconds' instead of 'secs']
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-12-18 12:35:55 +01:00
Vladimir Sementsov-Ogievskiy
96be1aeec7 simplebench/results_to_text: improve view of the table
Move to generic format for floats and percentage for error.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20201021145859.11201-19-vsementsov@virtuozzo.com>
Acked-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-12-18 12:35:55 +01:00
Vladimir Sementsov-Ogievskiy
8e979febb0 simplebench: move results_to_text() into separate file
Let's keep view part in separate: this way it's better to improve it in
the following commits.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20201021145859.11201-18-vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-12-18 12:35:55 +01:00
Vladimir Sementsov-Ogievskiy
bfccfa62ac simplebench: rename ascii() to results_to_text()
Next patch will use utf8 plus-minus symbol, let's use more generic (and
more readable) name.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20201021145859.11201-17-vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-12-18 12:35:55 +01:00
Vladimir Sementsov-Ogievskiy
f52e1af0b0 scripts/simplebench: use standard deviation for +- error
Standard deviation is more usual to see after +- than current maximum
of deviations.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20201021145859.11201-16-vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-12-18 12:35:55 +01:00
Vladimir Sementsov-Ogievskiy
4a44554a65 scripts/simplebench: support iops
Support benchmarks returning not seconds but iops. We'll use it for
further new test.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20201021145859.11201-15-vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-12-18 12:35:55 +01:00
Vladimir Sementsov-Ogievskiy
270124e7ef scripts/simplebench: fix grammar: s/successed/succeeded/
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20201021145859.11201-14-vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-12-18 12:35:55 +01:00
Philippe Mathieu-Daudé
a1bcbb485c scripts/git.orderfile: Keep files with .inc extension sorted
Sort .inc files along with the extension including them.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20201213205132.243628-1-f4bug@amsat.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-12-15 12:53:16 -05:00
Marc-André Lureau
07b35a23c3 compiler.h: remove QEMU_GNUC_PREREQ
When needed, the G_GNUC_CHECK_VERSION() glib macro can be used instead.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20201210134752.780923-14-marcandre.lureau@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-12-15 12:53:15 -05:00
Paolo Bonzini
953d0c333e scripts: kernel-doc: remove unnecessary change wrt Linux
A comment in kernel-doc mentions QEMU's qatomic_set macro, but since
this code originated in Linux we should just revert it and stay as close
to the kernel's copy of the script as possible.

The change was introduced (more or less unintentionally) in QEMU commit
commit d73415a315, which did a global search-and-replace of QEMU's
atomic access macros.

Suggested-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-12-10 12:15:25 -05:00
Mauro Carvalho Chehab
a1b8a57a0e scripts: kernel-doc: use :c:union when needed
Sphinx C domain code after 3.2.1 will start complaning if :c:struct
would be used for an union type:

	.../Documentation/gpu/drm-kms-helpers:352: ../drivers/video/hdmi.c:851: WARNING: C 'identifier' cross-reference uses wrong tag: reference name is 'union hdmi_infoframe' but found name is 'struct hdmi_infoframe'. Full reference name is 'union hdmi_infoframe'. Full found name is 'struct hdmi_infoframe'.

So, let's address this issue too in advance, in order to
avoid future issues.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/6e4ec3eec914df62389a299797a3880ae4490f35.1603791716.git.mchehab+huawei@kernel.org
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201117165312.118257-30-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-12-10 12:15:25 -05:00
Mauro Carvalho Chehab
e495a1b26c scripts: kernel-doc: split typedef complex regex
The typedef regex for function prototypes are very complex.
Split them into 3 separate regex and then join them using
qr.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/3a4af999a0d62d4ab9dfae1cdefdfcad93383356.1603792384.git.mchehab+huawei@kernel.org
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201117165312.118257-29-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-12-10 12:15:24 -05:00
Mauro Carvalho Chehab
01a7917d9b scripts: kernel-doc: fix typedef parsing
The include/linux/genalloc.h file defined this typedef:

	typedef unsigned long (*genpool_algo_t)(unsigned long *map,unsigned long size,unsigned long start,unsigned int nr,void *data, struct gen_pool *pool, unsigned long start_addr);

Because it has a type composite of two words (unsigned long),
the parser gets the typedef name wrong:

.. c:macro:: long

   **Typedef**: Allocation callback function type definition

Fix the regex in order to accept composite types when
defining a typedef for a function pointer.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/328e8018041cc44f7a1684e57f8d111230761c4f.1603792384.git.mchehab+huawei@kernel.org
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201117165312.118257-28-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-12-10 12:15:24 -05:00
Paolo Bonzini
3e72dc0135 Revert "kernel-doc: Handle function typedefs that return pointers"
This reverts commit 19ab6044be.
We will replace the commit with the fix from Linux.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201117165312.118257-27-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-12-10 12:15:24 -05:00
Paolo Bonzini
47da500a7e Revert "kernel-doc: Handle function typedefs without asterisks"
This reverts commit 3cd3c5193c.
We will replace the commit with the fix from Linux.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201117165312.118257-26-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-12-10 12:15:24 -05:00
Mauro Carvalho Chehab
ac3617d90c scripts: kernel-doc: try to use c:function if possible
There are a few namespace clashes by using c:macro everywhere:

basically, when using it, we can't have something like:

	.. c:struct:: pwm_capture

	.. c:macro:: pwm_capture

So, we need to use, instead:

	.. c:function:: int pwm_capture (struct pwm_device * pwm, struct pwm_capture * result, unsigned long timeout)

for the function declaration.

The kernel-doc change was proposed by Jakob Lykke Andersen here:

	6fd2076ec0

Although I did a different implementation.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201117165312.118257-25-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-12-10 12:15:24 -05:00
Mauro Carvalho Chehab
4f5f16a193 scripts: kernel-doc: fix line number handling
Address several issues related to pointing to the wrong line
number:

1) ensure that line numbers will always be initialized

   When section is the default (Description), the line number
   is not initializing, producing this:

	$ ./scripts/kernel-doc --enable-lineno ./drivers/media/v4l2-core/v4l2-mem2mem.c|less

	**Description**

	#define LINENO 0
	In case of streamoff or release called on any context,
	1] If the context is currently running, then abort job will be called
	2] If the context is queued, then the context will be removed from
	   the job_queue

  Which is not right. Ensure that the line number will always
  be there. After applied, the result now points to the right location:

	**Description**

	#define LINENO 410
	In case of streamoff or release called on any context,
	1] If the context is currently running, then abort job will be called
	2] If the context is queued, then the context will be removed from
	   the job_queue

2) The line numbers for function prototypes are always + 1,
   because it is taken at the line after handling the prototype.
   Change the logic to point to the next line after the /** */
   block;

3) The "DOC:" line number should point to the same line as this
   markup is found, and not to the next one.

Probably part of the issues were due to a but that was causing
the line number offset to be incremented by one, if --export
were used.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201117165312.118257-24-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-12-10 12:15:24 -05:00
Mauro Carvalho Chehab
486966e4a4 scripts: kernel-doc: allow passing desired Sphinx C domain dialect
When kernel-doc is called via kerneldoc.py, there's no need to
auto-detect the Sphinx version, as the Sphinx module already
knows it. So, add an optional parameter to allow changing the
Sphinx dialect.

As kernel-doc can also be manually called, keep the auto-detection
logic if the parameter was not specified. On such case, emit
a warning if sphinx-build can't be found at PATH.

I ended using a suggestion from Joe for using a more readable
regex, instead of using a complex one with a hidden group like:

	m/^(\d+)\.(\d+)(?:\.?(\d+)?)/

in order to get the optional <patch> argument.

Thanks-to: Joe Perches <joe@perches.com>
Suggested-by: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201117165312.118257-23-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-12-10 12:15:23 -05:00
Mauro Carvalho Chehab
0c77185233 scripts: kernel-doc: don't mangle with parameter list
While kernel-doc needs to parse parameters in order to
identify its name, it shouldn't be touching the type,
as parsing it is very difficult, and errors happen.

One current error is when parsing this parameter:

	const u32 (*tab)[256]

Found at ./lib/crc32.c, on this function:

	u32 __pure crc32_be_generic (u32 crc, unsigned char const *p, size_t len, const u32 (*tab)[256], u32 polynomial);

The current logic mangles it, producing this output:

	const u32 ( *tab

That's something that it is not recognizeable.

So, instead, let's push the argument as-is, and use it
when printing the function prototype and when describing
each argument.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201117165312.118257-22-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-12-10 12:15:23 -05:00
Mauro Carvalho Chehab
3999ffcf13 scripts: kernel-doc: fix typedef identification
Some typedef expressions are output as normal functions.

As we need to be clearer about the type with Sphinx 3.x,
detect such cases.

While here, fix a wrongly-indented block.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201117165312.118257-21-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-12-10 12:15:23 -05:00
Mauro Carvalho Chehab
78c8c92c5d scripts: kernel-doc: reimplement -nofunction argument
Right now, the build system doesn't use -nofunction, as
it is pretty much useless, because it doesn't consider
the other output modes (extern, internal), working only
with all.

Also, it is limited to exclude functions.

Re-implement it in order to allow excluding any symbols from
the document output, no matter what mode is used.

The parameter was also renamed to "-nosymbol", as it express
better its meaning.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201117165312.118257-20-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-12-10 12:15:23 -05:00
Mauro Carvalho Chehab
5abfaa6a7f scripts: kernel-doc: fix troubles with line counts
There's currently a bug with the way kernel-doc script
counts line numbers that can be seen with:

	$ ./scripts/kernel-doc -rst  -enable-lineno include/linux/math64.h >all && ./scripts/kernel-doc -rst -internal -enable-lineno include/linux/math64.h >int && diff -U0 int all

	--- int	2020-09-28 12:58:08.927486808 +0200
	+++ all	2020-09-28 12:58:08.905486845 +0200
	@@ -1 +1 @@
	-#define LINENO 27
	+#define LINENO 26
	@@ -3 +3 @@
	-#define LINENO 16
	+#define LINENO 15
	@@ -9 +9 @@
	-#define LINENO 17
	+#define LINENO 16
	...

This is happening with perl version 5.30.3, but I'm not
so sure if this is a perl bug, or if this is due to something
else.

In any case, fixing it is easy. Basically, when "-internal"
parameter is used, the process_export_file() function opens the
handle "IN". This makes the line number to be incremented, as the
handler for the main open is also "IN".

Fix the problem by using a different handler for the
main open().

While here, add a missing close for it.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201117165312.118257-19-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-12-10 12:15:23 -05:00
Mauro Carvalho Chehab
60ef7c1801 scripts: kernel-doc: use a less pedantic markup for funcs on Sphinx 3.x
Unfortunately, Sphinx 3.x parser for c functions is too pedantic:

	https://github.com/sphinx-doc/sphinx/issues/8241

While it could be relaxed with some configurations, there are
several corner cases that it would make it hard to maintain,
and will require teaching conf.py about several macros.

So, let's instead use the :c:macro notation. This will
produce an output that it is not as nice as currently, but it
should still be acceptable, and will provide cross-references,
removing thousands of warnings when building with newer
versions of Sphinx.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201117165312.118257-18-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-12-10 12:15:22 -05:00
Mauro Carvalho Chehab
6d3a3cfc2f scripts: kernel-doc: make it more compatible with Sphinx 3.x
With Sphinx 3.x, the ".. c:type:" tag was changed to accept either:

	.. c:type:: typedef-like declaration
	.. c:type:: name

Using it for other types (including functions) don't work anymore.

So, there are newer tags for macro, enum, struct, union, and others,
which doesn't exist on older versions.

Add a check for the Sphinx version and change the produced tags
accordingly.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201117165312.118257-17-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-12-10 12:15:22 -05:00
Paolo Bonzini
46ae6e8f82 Revert "kernel-doc: Use c:struct for Sphinx 3.0 and later"
This reverts commit 152d1967f6.
We will replace the commit with the fix from Linux.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201117165312.118257-16-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-12-10 12:15:22 -05:00
Paolo Bonzini
a832c9844d Revert "scripts/kerneldoc: For Sphinx 3 use c:macro for macros with arguments"
This reverts commit 92bb29f9b2.
We will replace the commit with the fix from Linux.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201117165312.118257-15-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-12-10 12:15:22 -05:00
Mauro Carvalho Chehab
306b015cfb scripts: kernel-doc: add support for typedef enum
The PHY kernel-doc markup has gained support for documenting
a typedef enum.

However, right now the parser was not prepared for it.

So, add support for parsing it.

Fixes: 4069a572d423 ("net: phy: Document core PHY structures")
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201117165312.118257-14-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-12-10 12:15:22 -05:00
Jonathan Cameron
5c51f435cb kernel-doc: add support for ____cacheline_aligned attribute
Subroutine dump_struct uses type attributes to check if the struct
syntax is valid. Then, it removes all attributes before using it for
output. `____cacheline_aligned` is an attribute that is
not included in both steps. Add it, since it is used by kernel structs.

Based on previous patch to add ____cacheline_aligned_in_smp.
Motivated by patches to reorder this attribute to before the
variable name.   Whilst we could do that in all cases, that would
be a massive change and it is more common in the kernel to place
this particular attribute after the variable name. A quick grep
suggests approximately 400 instances of which 341 have this
attribute just before a semicolon and hence after the variable name.

Signed-off-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Cc: Lee Jones <lee.jones@linaro.org>
Link: https://lore.kernel.org/r/20200910185415.653139-1-jic23@kernel.org
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201117165312.118257-13-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-12-10 12:15:21 -05:00
Mauro Carvalho Chehab
697f668ad1 kernel-doc: include line numbers for function prototypes
This should solve bad error reports like this one:

	./include/linux/iio/iio.h:0: WARNING: Unknown target name: "devm".

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/56eed0ba50cd726236acd12b11b55ce54854c5ea.1599660067.git.mchehab+huawei@kernel.org
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201117165312.118257-12-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-12-10 12:15:21 -05:00
Pierre-Louis Bossart
86cba21743 scripts/kernel-doc: optionally treat warnings as errors
The kbuild bot recently added the W=1 option, which triggered
documentation cleanups to squelch hundreds of kernel-doc warnings.

To make sure new kernel contributions don't add regressions to
kernel-doc descriptors, this patch suggests an option to treat
warnings as errors in CI/automated tests.

A -Werror command-line option is added to the kernel-doc script. When
this option is set, the script will return the number of warnings
found. The caller can then treat this positive return value as an
error and stop the build.

Using this command line option is however not straightforward when the
kernel-doc script is called from other scripts. To align with typical
kernel compilation or documentation generation, the Werror option is
also set by checking the KCFLAGS environment variable, or if
KDOC_WERROR is defined, as in the following examples:

KCFLAGS="-Wall -Werror" make W=1 sound/
KCFLAGS="-Wall -Werror" make W=1 drivers/soundwire/
KDOC_WERROR=1 make htmldocs

Note that in the last example the documentation build does not stop,
only an additional log is provided.

Credits to Randy Dunlap for suggesting the use of environment variables.

Suggested-by: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Pierre-Louis Bossart <pierre-louis.bossart@linux.intel.com>
Link: https://lore.kernel.org/r/20200728162040.92467-1-pierre-louis.bossart@linux.intel.com
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201117165312.118257-11-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-12-10 12:15:21 -05:00
Mauro Carvalho Chehab
2552f59a34 scripts/kernel-doc: handle function pointer prototypes
There are some function pointer prototypes inside the net
includes, like this one:

	int (*pcs_config)(struct phylink_config *config, unsigned int mode,
			  phy_interface_t interface, const unsigned long *advertising);

There's nothing wrong using it with kernel-doc, but we need to
add a rule for it to parse such kind of prototype.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/fec520dd731a273013ae06b7653a19c7d15b9562.1592895969.git.mchehab+huawei@kernel.org
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201117165312.118257-10-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-12-10 12:15:21 -05:00
Mauro Carvalho Chehab
f0fd307d25 scripts/kernel-doc: parse __ETHTOOL_DECLARE_LINK_MODE_MASK
The __ETHTOOL_DECLARE_LINK_MODE_MASK macro is a variant of
DECLARE_BITMAP(), used by phylink.h. As we have already a
parser for DECLARE_BITMAP(), let's add one for this macro,
in order to avoid such warnings:

	./include/linux/phylink.h:54: warning: Function parameter or member '__ETHTOOL_DECLARE_LINK_MODE_MASK(advertising' not described in 'phylink_link_state'
	./include/linux/phylink.h:54: warning: Function parameter or member '__ETHTOOL_DECLARE_LINK_MODE_MASK(lp_advertising' not described in 'phylink_link_state'

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/d1d1dea67a28117c0b0c33271b139c4455fef287.1592895969.git.mchehab+huawei@kernel.org
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201117165312.118257-9-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-12-10 12:15:21 -05:00
Alexander A. Klimov
cd08b80952 Replace HTTP links with HTTPS ones: documentation
Rationale:
Reduces attack surface on kernel devs opening the links for MITM
as HTTPS traffic is much harder to manipulate.

Deterministic algorithm:
For each file:
  For each line:
    If doesn't contain `\bxmlns\b`:
      For each link, `\bhttp://[^# \t\r\n]*(?:\w|/)`:
        If both the HTTP and HTTPS versions
        return 200 OK and serve the same content:
          Replace HTTP with HTTPS.

Signed-off-by: Alexander A. Klimov <grandmaster@al2klimov.de>
Link: https://lore.kernel.org/r/20200526060544.25127-1-grandmaster@al2klimov.de
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201117165312.118257-8-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-12-10 12:15:21 -05:00
Mauro Carvalho Chehab
9f2b463ad8 scripts: kernel-doc: accept blank lines on parameter description
Sphinx is very pedantic with respect to blank lines. Sometimes,
in order to make it to properly handle something, we need to
add a blank line. However, currently, any blank line inside a
kernel-doc comment like:

	/*
	 * @foo: bar
         *
	 *       foobar
	 *
	 * some description

will be considered as if "foobar" was part of the description.

This patch changes kernel-doc behavior. After it, foobar will
be considered as part of the parameter text. The description
will only be considered as such if it starts with:

zero spaces after asterisk:

	*foo

one space after asterisk:
	* foo

or have a explicit Description section:

	*   Description:

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/c07d2862792d75a2691d69c9eceb7b89a0164cc0.1586881715.git.mchehab+huawei@kernel.org
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201117165312.118257-7-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-12-10 12:15:20 -05:00
Mauro Carvalho Chehab
b1e8e720b6 scripts: kernel-doc: accept negation like !@var
On a few places, it sometimes need to indicate a negation of a
parameter, like:

	!@fshared

This pattern happens, for example, at:

	kernel/futex.c

and it is perfectly valid. However, kernel-doc currently
transforms it into:

	!**fshared**

This won't do what it would be expected.

Fortunately, fixing the script is a simple matter of storing
the "!" before "@" and adding it after the bold markup, like:

	**!fshared**

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/0314b47f8c3e1f9db00d5375a73dc3cddd8a21f2.1586881715.git.mchehab+huawei@kernel.org
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201117165312.118257-6-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-12-10 12:15:20 -05:00
Mauro Carvalho Chehab
76083982a5 scripts: kernel-doc: proper handle @foo->bar()
The pattern @foo->bar() is valid, as it can be used by a
function pointer inside a struct passed as a parameter.

Right now, it causes a warning:

	./drivers/firewire/core-transaction.c:606: WARNING: Inline strong start-string without end-string.

In this specific case, the kernel-doc markup is:

	/**
	 * fw_core_remove_address_handler() - unregister an address handler
	 * @handler: callback
	 *
	 * To be called in process context.
	 *
	 * When fw_core_remove_address_handler() returns, @handler->callback() is
	 * guaranteed to not run on any CPU anymore.
	 */

With seems valid on my eyes. So, instead of trying to hack
the kernel-doc markup, let's teach it about how to handle
such things. This should likely remove lots of other similar
warnings as well.

Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Link: https://lore.kernel.org/r/48b46426d7bf6ff7529f20e5718fbf4e9758e62c.1586881715.git.mchehab+huawei@kernel.org
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201117165312.118257-5-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-12-10 12:15:20 -05:00
Jonathan Neuschäfer
65fb67bf73 scripts/kernel-doc: Add support for named variable macro arguments
Currently, when kernel-doc encounters a macro with a named variable
argument[1], such as this:

   #define hlist_for_each_entry_rcu(pos, head, member, cond...)

... it expects the variable argument to be documented as `cond...`,
rather than `cond`. This is semantically wrong, because the name (as
used in the macro body) is actually `cond`.

With this patch, kernel-doc will accept the name without dots (`cond`
in the example above) in doc comments, and warn if the name with dots
(`cond...`) is used and verbose mode[2] is enabled.

The support for the `cond...` syntax can be removed later, when the
documentation of all such macros has been switched to the new syntax.

Testing this patch on top of v5.4-rc6, `make htmldocs` shows a few
changes in log output and HTML output:

 1) The following warnings[3] are eliminated:

   ./include/linux/rculist.h:374: warning:
        Excess function parameter 'cond' description in 'list_for_each_entry_rcu'
   ./include/linux/rculist.h:651: warning:
        Excess function parameter 'cond' description in 'hlist_for_each_entry_rcu'

 2) For list_for_each_entry_rcu and hlist_for_each_entry_rcu, the
    correct description is shown

 3) Named variable arguments are shown without dots

[1]: https://gcc.gnu.org/onlinedocs/cpp/Variadic-Macros.html
[2]: scripts/kernel-doc -v
[3]: See also https://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu.git/commit/?h=dev&id=5bc4bc0d6153617eabde275285b7b5a8137fdf3c

Signed-off-by: Jonathan Neuschäfer <j.neuschaefer@gmx.net>
Tested-by: Paul E. McKenney <paulmck@kernel.org>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201117165312.118257-4-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-12-10 12:15:20 -05:00
André Almeida
b5a8dfb532 kernel-doc: add support for ____cacheline_aligned_in_smp attribute
Subroutine dump_struct uses type attributes to check if the struct
syntax is valid. Then, it removes all attributes before using it for
output. `____cacheline_aligned_in_smp` is an attribute that is
not included in both steps. Add it, since it is used by kernel structs.

Signed-off-by: André Almeida <andrealmeid@collabora.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201117165312.118257-3-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-12-10 12:15:20 -05:00
André Almeida
872e6c47a0 kernel-doc: fix processing nested structs with attributes
The current regular expression for strip attributes of structs (and
for nested ones as well) also removes all whitespaces that may
surround the attribute. After that, the code will split structs and
iterate for each symbol separated by comma at the end of struct
definition (e.g. "} alias1, alias2;"). However, if the nested struct
does not have any alias and has an attribute, it will result in a
empty string at the closing bracket (e.g "};"). This will make the
split return nothing and $newmember will keep uninitialized. Fix
that, by ensuring that the attribute substitution will leave at least
one whitespace.

Signed-off-by: André Almeida <andrealmeid@collabora.com>
Signed-off-by: Jonathan Corbet <corbet@lwn.net>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20201117165312.118257-2-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-12-10 12:15:20 -05:00
Alex Bennée
e4b937d3c4 scripts/ci: clean up default args logic a little
This allows us to do:

  ./scripts/ci/gitlab-pipeline-status -w -b HEAD -p 2961854

to check out own pipeline status of a recently pushed branch.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20201117173635.29101-2-alex.bennee@linaro.org>
2020-11-23 09:48:25 +00:00
Stefan Weil
933c8fe781 meson: Fix argument for makensis (build regression)
`make installer` with a DLL directory was broken.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
Message-Id: <20201117190640.390359-1-sw@weilnetz.de>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-11-18 09:28:55 +01:00
Chetan Pant
61f3c91a67 nomaintainer: Fix Lesser GPL version number
There is no "version 2" of the "Lesser" General Public License.
It is either "GPL version 2.0" or "Lesser GPL version 2.1".
This patch replaces all occurrences of "Lesser GPL version 2" with
"Lesser GPL version 2.1" in comment section.

This patch contains all the files, whose maintainer I could not get
from ‘get_maintainer.pl’ script.

Signed-off-by: Chetan Pant <chetan4windows@gmail.com>
Message-Id: <20201023124424.20177-1-chetan4windows@gmail.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
[thuth: Adapted exec.c and qdev-monitor.c to new location]
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-11-15 17:04:40 +01:00
Chetan Pant
d6ea423635 overall/alpha tcg cpus|hppa: Fix Lesser GPL version number
There is no "version 2" of the "Lesser" General Public License.
It is either "GPL version 2.0" or "Lesser GPL version 2.1".
This patch replaces all occurrences of "Lesser GPL version 2" with
"Lesser GPL version 2.1" in comment section.

Signed-off-by: Chetan Pant <chetan4windows@gmail.com>
Message-Id: <20201023123353.19796-1-chetan4windows@gmail.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-11-15 16:43:54 +01:00
Stefan Hajnoczi
35e28cb0f2 scripts/tracetool: silence SystemTap dtrace(1) long long warnings
SystemTap's dtrace(1) prints the following warning when it encounters
long long arguments:

  Warning: /usr/bin/dtrace:trace/trace-dtrace-hw_virtio.dtrace:76: syntax error near:
  probe vhost_vdpa_dev_start

  Warning: Proceeding as if --no-pyparsing was given.

Use the uint64_t and int64_t types, respectively. This works with all
host CPU 32- and 64-bit data models (ILP32, LP64, and LLP64) that QEMU
supports.

Reported-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-id: 20201020094043.159935-1-stefanha@redhat.com
Suggested-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2020-11-11 13:10:38 +00:00
Alexander Bulekov
bb451d2487 scripts/oss-fuzz: give all fuzzers -target names
We switched to hardlinks in
a942f64cc4 ("scripts/oss-fuzz: use hardlinks instead of copying")

The motivation was to conserve space (50 fuzzers built with ASAN, can
weigh close to 9 GB).

Unfortunately, OSS-Fuzz (partially) treated the underlying copy of the
fuzzer as a standalone fuzzer. To attempt to fix, we tried:

f8b8f37463 ("scripts/oss-fuzz: rename bin/qemu-fuzz-i386")

This was also not a complete fix, because though OSS-Fuzz
ignores the renamed fuzzer, the underlying ClusterFuzz, doesn't:
https://storage.googleapis.com/clusterfuzz-builds/qemu/targets.list.address
https://oss-fuzz-build-logs.storage.googleapis.com/log-9bfb55f9-1c20-4aa6-a49c-ede12864eeb2.txt
(clusterfuzz still lists qemu-fuzz-i386.base as a fuzzer)

This change keeps the hard-links, but makes them all point to a file
with a qemu-fuzz-i386-target-.. name. If we have targets, A, B, C, the
result will be:

qemu-fuzz-i386-target-A (base file)
qemu-fuzz-i386-target-B -> qemu-fuzz-i386-target-A
qemu-fuzz-i386-target-C -> qemu-fuzz-i386-target-A

The result should be that every file that looks like a fuzzer to
OSS-Fuzz/ClusterFuzz, can run as a fuzzer (we don't have a separate base
copy). Unfortunately, there is not simple way to test this locally.

In the future, it might be worth it to link the majority of QEMU in as a
shared-object (see https://github.com/google/oss-fuzz/issues/4575 )

Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
Message-Id: <20201108171136.160607-1-alxndr@bu.edu>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-11-10 08:51:30 +01:00
Eduardo Habkost
8a47836548 device-crash-test: Check if path is actually an executable file
After the transition to Meson, the build directory now have
subdirectories named "qemu-system-*.p", and device-crash-test
will try to execute them as if they were binaries.  This results
in errors like:

  PermissionError: [Errno 13] Permission denied: './qemu-system-or1k.p'

When generating the default list of binaries to test, check if
the path is actually a file and if it's executable.

Reported-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Message-Id: <20201026125238.2752882-1-ehabkost@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-11-09 18:34:21 +01:00
Alexander Bulekov
f8b8f37463 scripts/oss-fuzz: rename bin/qemu-fuzz-i386
OSS-Fuzz changed the way it scans for fuzzers in $DEST_DIR. The new code
also scans subdirectories for fuzzers. This means that OSS-Fuzz is
considering bin/qemu-fuzz-i386 as an independent fuzzer (it is not - it
requires a --fuzz-target argument). This has led to coverage-build
failures and false crash reports. To work around this, we take advantage
of OSS-Fuzz' filename extension check - OSS-Fuzz will not run anything
that has an extension that is not ".exe":
https://github.com/google/oss-fuzz/blob/master/infra/utils.py#L115

Reported-by: OSS-Fuzz (Issue 26725)
Reported-by: OSS-Fuzz (Issue 26679)
Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
Message-Id: <20201101212245.185819-1-alxndr@bu.edu>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-11-03 09:42:53 -05:00
Peter Maydell
92bb29f9b2 scripts/kerneldoc: For Sphinx 3 use c:macro for macros with arguments
The kerneldoc script currently emits Sphinx markup for a macro with
arguments that uses the c:function directive. This is correct for
Sphinx versions earlier than Sphinx 3, where c:macro doesn't allow
documentation of macros with arguments and c:function is not picky
about the syntax of what it is passed. However, in Sphinx 3 the
c:macro directive was enhanced to support macros with arguments,
and c:function was made more picky about what syntax it accepted.

When kerneldoc is told that it needs to produce output for Sphinx
3 or later, make it emit c:function only for functions and c:macro
for macros with arguments. We assume that anything with a return
type is a function and anything without is a macro.

This fixes the Sphinx error:

/home/petmay01/linaro/qemu-from-laptop/qemu/docs/../include/qom/object.h:155:Error in declarator
If declarator-id with parameters (e.g., 'void f(int arg)'):
  Invalid C declaration: Expected identifier in nested name. [error at 25]
    DECLARE_INSTANCE_CHECKER ( InstanceType,  OBJ_NAME,  TYPENAME)
    -------------------------^
If parenthesis in noptr-declarator (e.g., 'void (*f(int arg))(double)'):
  Error in declarator or parameters
  Invalid C declaration: Expecting "(" in parameters. [error at 39]
    DECLARE_INSTANCE_CHECKER ( InstanceType,  OBJ_NAME,  TYPENAME)
    ---------------------------------------^

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Tested-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 20201030174700.7204-2-peter.maydell@linaro.org
2020-11-02 16:52:18 +00:00
Matthew Rosato
84567ea763 update-linux-headers: Add vfio_zdev.h
vfio_zdev.h is used by s390x zPCI support to pass device-specific
CLP information between host and userspace.

Signed-off-by: Matthew Rosato <mjrosato@linux.ibm.com>
Acked-by: Cornelia Huck <cohuck@redhat.com>
Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
2020-11-01 12:30:51 -07:00
Peter Maydell
c044400914 QMP patches patches for 2020-10-27
-----BEGIN PGP SIGNATURE-----
 
 iQJGBAABCAAwFiEENUvIs9frKmtoZ05fOHC0AOuRhlMFAl+X8qwSHGFybWJydUBy
 ZWRoYXQuY29tAAoJEDhwtADrkYZT8k0P/0AXcoP/clFP0x+TBlcPpk2YPFSS6BnA
 yIngvKMIKoIxekHtW3ukNANKIynWy2d3u7aD/c8aCLJtioWIZqzmBCW5zC/oYFyO
 DvOCnXzAK1T/9wy4sswaOjSVeqfI5dsMJKNvYVouXGs0+kYy74k7vtWx5OdjrV4D
 w4KVyGc5tos9RAafcYl7jYKFDfkYy2+5rZWorIuEG1P6GAjdXMl4r6ako+pbpzW2
 g1gbG/Kau9L4MfZR2wvtzblm5Ve+Mf/XivxKdUiHG/MdI9NWrRA033xCQWmrEoQw
 teeBzVlHHvw9xF5vaqFlYySs+WK9Ou6Q4h0Nc4WNYw6eWPMNog8T/vi2y9Ra+at/
 3gAR0GRrfpd/RXqhFh9xwbEe9n0KzodAA+eGAnT/f2+9CoQHLm436cgzk+Vtasn1
 6yFmBwxFnBQeaApAb3oARmXAQrI2hdQBNmxLB1BZuUDKNHrpyZp3Xeh/uTW6PthH
 3COdYpzNBeBYaLfcF46wvIIfbQJTrI9WrSuJBN/1oLsm0oxtDnBmgIv0Vij4QLl6
 uymPd+RXK3GCj51+6DnccUW6+5YManwdSHMcepOtlXsnXdVVt1t2VkhfyiECOWpA
 Gd2wBRLCiq810Laf2ywx8OKt7/6ujAE8ksUpAE8fY1MsUBQlzNA/Z/HKsTpXa6ZW
 nkwPtrz9YHu3
 =yTrz
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/armbru/tags/pull-qmp-2020-10-27' into staging

QMP patches patches for 2020-10-27

# gpg: Signature made Tue 27 Oct 2020 10:13:00 GMT
# gpg:                using RSA key 354BC8B3D7EB2A6B68674E5F3870B400EB918653
# gpg:                issuer "armbru@redhat.com"
# gpg: Good signature from "Markus Armbruster <armbru@redhat.com>" [full]
# gpg:                 aka "Markus Armbruster <armbru@pond.sub.org>" [full]
# Primary key fingerprint: 354B C8B3 D7EB 2A6B 6867  4E5F 3870 B400 EB91 8653

* remotes/armbru/tags/pull-qmp-2020-10-27:
  scripts/qmp: delete 'qmp' script
  qmp-shell: Sort by key when pretty-printing

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-10-29 10:03:32 +00:00
Peter Maydell
a5e7fb4d20 Testing and gitdm updates
- add some more individual contributors
   - include SDL2 in centos images
   - skip checkpatch check when no commits found
   - use random port for gdb reverse debugging
   - make gitlab use it's own mirrors to clone
   - fix detection of make -nqp
 -----BEGIN PGP SIGNATURE-----
 
 iQEzBAABCgAdFiEEZoWumedRZ7yvyN81+9DbCVqeKkQFAl+X7qsACgkQ+9DbCVqe
 KkSpmAf/To0P/c/B6qMEvNfCiWtn/Nuypau8W60q/ZvE/J4IzFarwa5uc7COmyNR
 xjvxdoeKp17ykFdwBzGhiKeO83SO8DUH1JmzA6STs4RH5VSFGYycuE0cQ9KugrpE
 4yOruOdEgAn2+35oYalE9u6/EugdIVBZsjMS/ofsgR9ysYPuju4Q9by6DEK6ArW0
 AmdwJzKAYw75v+D2wKNq9coSWvbqWClsV6j0gaKgn+OQYqWo25T/DzWdr1okCRET
 TvB0HZLCDQHz9Friq5PtBtVsmNbZaL+CM9RRekFBGUFEvZE8i4VEDHFfzXsN3U0w
 VA5yhlBJzl1/prBQe1qRbrZQL4x58w==
 =Bnt1
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/stsquad/tags/pull-testing-and-misc-271020-1' into staging

Testing and gitdm updates

  - add some more individual contributors
  - include SDL2 in centos images
  - skip checkpatch check when no commits found
  - use random port for gdb reverse debugging
  - make gitlab use it's own mirrors to clone
  - fix detection of make -nqp

# gpg: Signature made Tue 27 Oct 2020 09:55:55 GMT
# gpg:                using RSA key 6685AE99E75167BCAFC8DF35FBD0DB095A9E2A44
# gpg: Good signature from "Alex Bennée (Master Work Key) <alex.bennee@linaro.org>" [full]
# Primary key fingerprint: 6685 AE99 E751 67BC AFC8  DF35 FBD0 DB09 5A9E 2A44

* remotes/stsquad/tags/pull-testing-and-misc-271020-1:
  makefile: handle -n / -k / -q correctly
  gitlab-ci: Clone from GitLab itself
  tests/acceptance: pick a random gdb port for reverse debugging
  scripts: fix error from checkpatch.pl when no commits are found
  gitlab: skip checkpatch.pl checks if no commit delta on branch
  tests/docker/dockerfiles/centos: Use SDL2 instead of SDL1
  contrib/gitdm: Add more individual contributors
  Adding ani's email as an individual contributor

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-10-28 20:40:09 +00:00
Peter Maydell
5c27a85518 Update syscall numbers to 5.9-rc7
Fixes for prctl(), accept4() and xtensa
 -----BEGIN PGP SIGNATURE-----
 
 iQJGBAABCAAwFiEEzS913cjjpNwuT1Fz8ww4vT8vvjwFAl+XR5ASHGxhdXJlbnRA
 dml2aWVyLmV1AAoJEPMMOL0/L748liYP/j5YV8smHIK9gO3Vnn9tHAE82cgakyt+
 /GZUUUA8N5b3NQdNtHIiT3BMQFCESUiF3JPfcD+95CE4/yl9LW5fGFMygxb8e6U1
 BVei0Ggb69E5/+r+zWd1leGXa0t7ZHzLlOEwAjAwW4MQR3SXOYdM/q87HAbI6p/b
 wLOcJih5ZD3eCvC3XR99xfcWn0rIjDTuYgH9dXbEJxAHpaa/rH2auPU6i9DWxprn
 lgz8RsSFEfG9CJg/BLK+w3CtL9ndeSykFf/HO7GokHVzDM0mEKBpFWFx4y6Nkiq6
 Qaq0qj8jgZeQjWXog3YLrUna6EzkuHxmtbyOsfi9tdaqZay6xedDuREPfZ9GHHLT
 gdyX6tXVt3Qiot3P3P6RQwCU+iOMHpwRsaBnXJQnDkck7piNwUfWX5j6t8lqbWLB
 kKyTYRBLrhmxC9Tjq8dZZrQGXOrG38fkL+yg4S+xr7+ju9CKEtHk7Y4/knwkevIi
 gE1cuj9C0uhqGLoHZPYsawhFy6OMzuZQ5LaAEV7TH6cumSgLXbqWsw/xPoHtb3/1
 mE/vnOD1ArDAiInIVXImPUNF3I6pzSJR8IcKVW9BEMNjMZApMWVis65Owmyy0dic
 RSBF8WGUy4Xp9YLTXW1jQJWx9wHbqqN4cbQjzFLv+4ZDdjXHLYyjGGefhchUOJ+N
 Zvedbmlv0ROZ
 =7TIT
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/vivier2/tags/linux-user-for-5.2-pull-request' into staging

Update syscall numbers to 5.9-rc7
Fixes for prctl(), accept4() and xtensa

# gpg: Signature made Mon 26 Oct 2020 22:02:56 GMT
# gpg:                using RSA key CD2F75DDC8E3A4DC2E4F5173F30C38BD3F2FBE3C
# gpg:                issuer "laurent@vivier.eu"
# gpg: Good signature from "Laurent Vivier <lvivier@redhat.com>" [full]
# gpg:                 aka "Laurent Vivier <laurent@vivier.eu>" [full]
# gpg:                 aka "Laurent Vivier (Red Hat) <lvivier@redhat.com>" [full]
# Primary key fingerprint: CD2F 75DD C8E3 A4DC 2E4F  5173 F30C 38BD 3F2F BE3C

* remotes/vivier2/tags/linux-user-for-5.2-pull-request:
  target/xtensa: enable all coprocessors for linux-user
  linux-user: correct errno returned from accept4() syscall
  linux-user: remove _sysctl
  linux-user: update syscall.tbl to Linux 5.9-rc7
  linux-user: update mips/syscall-args-o32.c.inc to Linux 5.9-rc7
  linux-user: update syscall_nr.h to Linux 5.9-rc7
  linux-user: Support f_flags in statfs64 when available.
  Fix stack smashing when handling PR_GET_PDEATHSIG

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-10-28 15:08:36 +00:00
Peter Maydell
4a74626970 Pull request
v2:
  * Fix Anthony Perard's email address [Philippe]
 -----BEGIN PGP SIGNATURE-----
 
 iQEzBAABCAAdFiEEhpWov9P5fNqsNXdanKSrs4Grc8gFAl+XAbkACgkQnKSrs4Gr
 c8h0VggAvM5Smw+ALmAz9kh2l/a9Asjvd3y70imgXVX2Aj3hWWxZLlTKv459xIgO
 bYNQgXDt2QVnzhFT8tMrsTYvMi7OMcqZE3Kmou/5pfi2E7Wj9272OeHakrQbqVCb
 5gEAZkQKxtnEfxysiVIQpsyp5hy0OnsWF+f3QfJUYq+BvI9XMOPsu3WpYJR3A9Uo
 joKHFjbRKlA1bSSpZIu7++u3sIuvDtbCwg04k5YeGSFiqmO400IjUoMQIdps3RvH
 +fOPz9MamNIZF0m7xkw7jS2FyWvq9PfMTCASFXQkVxu5lKm1k+GSvRo8g2vLWr/6
 w8OOFcP4UTW87Ctzhr+2/5qi8SnCiA==
 =J+D3
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/stefanha-gitlab/tags/tracing-pull-request' into staging

Pull request

v2:
 * Fix Anthony Perard's email address [Philippe]

# gpg: Signature made Mon 26 Oct 2020 17:04:57 GMT
# gpg:                using RSA key 8695A8BFD3F97CDAAC35775A9CA4ABB381AB73C8
# gpg: Good signature from "Stefan Hajnoczi <stefanha@redhat.com>" [full]
# gpg:                 aka "Stefan Hajnoczi <stefanha@gmail.com>" [full]
# Primary key fingerprint: 8695 A8BF D3F9 7CDA AC35  775A 9CA4 ABB3 81AB 73C8

* remotes/stefanha-gitlab/tags/tracing-pull-request:
  Add execute bit back to scripts/tracetool.py
  trace/simple: Enable tracing on startup only if the user specifies a trace option

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-10-27 11:28:46 +00:00
John Snow
3600604667 scripts/qmp: delete 'qmp' script
This script has not seen a patch that was specifically for this script
since it was moved to this location in 2013, and I doubt it is used. It
uses "man qmp" for its help message, which does not exist. It also
presumes there is a manual page for qmp-XXX, for each defined qmp
command XXX. I don't think that's true.

The format it expects arguments in is something like:

block-dirty-bitmap-add --node=foo --name=bar

and has no capacity to support nested JSON arguments, either.

Most developers use either qmp-shell or socat (or pasting JSON directly
into qmp stdio), so this duplication and additional alternate syntax is
not helpful.

Remove it. Leave a breadcrumb script just in case, to be removed next
release cycle.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20201019210430.1063390-1-jsnow@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2020-10-27 11:11:06 +01:00
David Edmondson
fca9d72323 qmp-shell: Sort by key when pretty-printing
If the user selects pretty-printing (-p) the contents of any
dictionaries in the output are sorted by key.

Signed-off-by: David Edmondson <david.edmondson@oracle.com>
Message-Id: <20201013141414.18398-1-david.edmondson@oracle.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2020-10-27 11:11:06 +01:00
Daniel P. Berrangé
2d4274d6fe scripts: fix error from checkpatch.pl when no commits are found
The error message was supposed to mention the input revision list start
point, not the branch flag.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20201019143537.283094-3-berrange@redhat.com>
Message-Id: <20201021163136.27324-6-alex.bennee@linaro.org>
2020-10-27 09:53:50 +00:00
Anthony PERARD
ee8041013a Add execute bit back to scripts/tracetool.py
Commit a81df1b68b ("libqemuutil, qapi, trace: convert to meson")
removed it without explanation and it is useful to be able to run a
script without having to figure out which interpreter to use.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20200923103620.1980151-1-anthony.perard@citrix.com>
2020-10-26 17:03:26 +00:00
Peter Maydell
a95e0396c8 * fix --disable-tcg builds (Claudio)
* Fixes for macOS --enable-modules build and OpenBSD curses/iconv detection (myself)
 * Start preparing for meson 0.56 (myself)
 * Move directory configuration to meson (myself)
 * Start untangling qemu_init (myself)
 * Windows fixes (Sunil)
 * Remove -no-kbm (Thomas)
 -----BEGIN PGP SIGNATURE-----
 
 iQFIBAABCAAyFiEE8TM4V0tmI4mGbHaCv/vSX3jHroMFAl+WrxEUHHBib256aW5p
 QHJlZGhhdC5jb20ACgkQv/vSX3jHroNQAggAqfucqEQvz6s+DCPv2u572diyMvhe
 Y7vmaQF0qYKoAvy5OLqGlqXVsn8lwf19zJWo9Z7k4qNefWl84ii0J/kEmnolzTGq
 7Z0CRSnGbNQy9YedYXuymaR3E0VY+6lsPnzIpufQISzQRdjzT8OQ51DMAhc04oQl
 saXsts7y+om+tzvW2JFGtNsfFRUjcRKqjIAVfwneBXFW9TRD2epvYxz/S0o+XJwF
 eSiINvTqDxxPyy6XJykC46xf/TTfReHv6fQgTn7Jw3TQuo4m7qXLi5Vj8W1erZJv
 t3xhZNabt813T6ztNcAAuJ0srIn55Ac7Fuq3/1ecgeVD08ntmabe4WhKRg==
 =931x
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/bonzini-gitlab/tags/for-upstream' into staging

* fix --disable-tcg builds (Claudio)
* Fixes for macOS --enable-modules build and OpenBSD curses/iconv detection (myself)
* Start preparing for meson 0.56 (myself)
* Move directory configuration to meson (myself)
* Start untangling qemu_init (myself)
* Windows fixes (Sunil)
* Remove -no-kbm (Thomas)

# gpg: Signature made Mon 26 Oct 2020 11:12:17 GMT
# gpg:                using RSA key F13338574B662389866C7682BFFBD25F78C7AE83
# gpg:                issuer "pbonzini@redhat.com"
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>" [full]
# gpg:                 aka "Paolo Bonzini <pbonzini@redhat.com>" [full]
# Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4  E2F7 7E15 100C CD36 69B1
#      Subkey fingerprint: F133 3857 4B66 2389 866C  7682 BFFB D25F 78C7 AE83

* remotes/bonzini-gitlab/tags/for-upstream:
  machine: move SMP initialization from vl.c
  machine: move UP defaults to class_base_init
  machine: remove deprecated -machine enforce-config-section option
  win32: boot broken when bind & data dir are the same
  WHPX: Fix WHPX build break
  configure: move install_blobs from configure to meson
  configure: remove unused variable from config-host.mak
  configure: move directory options from config-host.mak to meson
  configure: allow configuring localedir
  Makefile: separate meson rerun from the rest of the ninja invocation
  Remove deprecated -no-kvm option
  replay: do not build if TCG is not available
  qtest: unbreak non-TCG builds in bios-tables-test
  hw/core/qdev-clock: add a reference on aliased clocks
  do not use colons in test names
  meson: rewrite curses/iconv test
  build: fix macOS --enable-modules build

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-10-26 15:49:11 +00:00
Laurent Vivier
29fe5029a4 linux-user: update syscall_nr.h to Linux 5.9-rc7
Update gensyscalls.sh not to generate an empty line at the end of the file

And then automatically update syscall_nr.h running scripts/gensyscalls.sh

Signed-off-by: Laurent Vivier <laurent@vivier.eu>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20200930003033.554124-2-laurent@vivier.eu>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
2020-10-26 11:39:23 +01:00
Alexander Bulekov
53e1a50d6b scripts/oss-fuzz: ignore the generic-fuzz target
generic-fuzz is not a standalone fuzzer - it requires some env variables
to be set. On oss-fuzz, we set these with some predefined
generic-fuzz-{...} targets, that are thin wrappers around generic-fuzz.
Do not make a link for the generic-fuzz from the oss-fuzz build, so
oss-fuzz does not treat it as a standalone fuzzer.

Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
Message-Id: <20201023150746.107063-18-alxndr@bu.edu>
Reviewed-by: Darren Kenny <darren.kenny@oracle.com>
[thuth: Reformatted one comment to stay within the 80 columns limit]
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-10-26 09:53:54 +01:00
Alexander Bulekov
a942f64cc4 scripts/oss-fuzz: use hardlinks instead of copying
Prior to this, fuzzers in the output oss-fuzz directory were exactly
the same executable, with a different name to do argv[0]-based
fuzz-target selection. This is a waste of space, especially since these
binaries can weigh many MB.

Instead of copying, use hard links, to cut down on wasted space. We need
to place the primary copy of the executable into DEST_DIR, since this is
a separate file-system on oss-fuzz. We should not place it directly into
$DEST_DIR, since oss-fuzz will treat it as an independent fuzzer and try
to run it for fuzzing. Instead, we create a DEST_DIR/bin directory to
store the primary copy.

Suggested-by: Darren Kenny <darren.kenny@oracle.com>
Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
Message-Id: <20201023150746.107063-17-alxndr@bu.edu>
Reviewed-by: Darren Kenny <darren.kenny@oracle.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-10-26 09:53:54 +01:00
Alexander Bulekov
cd3f0686dd scripts/oss-fuzz: Add crash trace minimization script
Once we find a crash, we can convert it into a QTest trace. Usually this
trace will contain many operations that are unneeded to reproduce the
crash. This script tries to minimize the crashing trace, by removing
operations and trimming QTest bufwrite(write addr len data...) commands.

Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
Reviewed-by: Darren Kenny <darren.kenny@oracle.com>
Message-Id: <20201023150746.107063-12-alxndr@bu.edu>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-10-26 09:53:53 +01:00
Alexander Bulekov
7c9b64ade9 scripts/oss-fuzz: Add script to reorder a generic-fuzzer trace
The generic-fuzzer uses hooks to fulfill DMA requests just-in-time.
This means that if we try to use QTEST_LOG=1 to build a reproducer, the
DMA writes will be logged _after_ the in/out/read/write that triggered
the DMA read. To work work around this, the generic-fuzzer annotates
these just-in time DMA fulfilments with a tag that we can use to
discern them. This script simply iterates over a raw qtest
trace (including log messages, errors, timestamps etc), filters it and
re-orders it so that DMA fulfillments are placed directly _before_ the
qtest command that will cause the DMA access.

Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
Reviewed-by: Darren Kenny <darren.kenny@oracle.com>
Message-Id: <20201023150746.107063-11-alxndr@bu.edu>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-10-26 09:53:53 +01:00
Paolo Bonzini
da0dfe251d build: fix macOS --enable-modules build
Apple's nm implementation includes empty lines in the output that are not
found in GNU binutils.  This confuses scripts/undefsym.py, though it did
not confuse the scripts/undefsym.sh script that it replaced.  To fix
this, ignore lines that do not have two fields.

Reported-by: Emmanuel Blot <eblot.ml@gmail.com>
Tested-by: Emmanuel Blot <eblot.ml@gmail.com>
Fixes: 604f3e4e90 ("meson: Convert undefsym.sh to undefsym.py", 2020-09-08)
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-10-22 11:53:52 -04:00
Peter Maydell
514101c0b9 decodetree: Fix codegen for non-overlapping group inside overlapping group
For nested groups like:

  {
    [
      pattern 1
      pattern 2
    ]
    pattern 3
  }

the intended behaviour is that patterns 1 and 2 must not
overlap with each other; if the insn matches neither then
we fall through to pattern 3 as the next thing in the
outer overlapping group.

Currently we generate incorrect code for this situation,
because in the code path for a failed match inside the
inner non-overlapping group we generate a "return" statement,
which causes decode to stop entirely rather than continuing
to the next thing in the outer group.

Generate a "break" instead, so that decode flow behaves
as required for this nested group case.

Suggested-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Message-id: 20201019151301.2046-2-peter.maydell@linaro.org
2020-10-20 16:12:01 +01:00
Paolo Bonzini
09e93326e4 build: replace ninjatool with ninja
Now that the build is done entirely by Meson, there is no need
to keep the Makefile conversion.  Instead, we can ask Ninja about
the targets it exposes and forward them.

The main advantages are, from smallest to largest:

- reducing the possible namespace pollution within the Makefile

- removal of a relatively large Python program

- faster build because parsing Makefile.ninja is slower than
parsing build.ninja; and faster build after Meson runs because
we do not have to generate Makefile.ninja.

- tracking of command lines, which provides more accurate rebuilds

In addition the change removes the requirement for GNU make 3.82, which
was annoying on Mac, and avoids bugs on Windows due to ninjatool not
knowing how to convert Windows escapes to POSIX escapes.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-10-17 10:45:51 -04:00