Commit Graph

91405 Commits

Author SHA1 Message Date
Fabrice Fontaine
3043320390 block/export/fuse.c: fix musl build
Include linux/falloc.h if CONFIG_FALLOCATE_ZERO_RANGE is defined to fix
50482fda98
and avoid the following build failure on musl:

../block/export/fuse.c: In function 'fuse_fallocate':
../block/export/fuse.c:643:21: error: 'FALLOC_FL_ZERO_RANGE' undeclared (first use in this function)
  643 |     else if (mode & FALLOC_FL_ZERO_RANGE) {
      |                     ^~~~~~~~~~~~~~~~~~~~

Fixes:
 - http://autobuild.buildroot.org/results/be24433a429fda681fb66698160132c1c99bc53b

Fixes: 50482fda98 ("block/export/fuse.c: fix musl build")
Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Message-Id: <20211022095209.1319671-1-fontaine.fabrice@gmail.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2021-11-02 13:02:46 +01:00
Samuel Thibault
46e018e9b7 ide: Cap LBA28 capacity announcement to 2^28-1
The LBA28 capacity (at offsets 60/61 of identification) is supposed to
express the maximum size supported by LBA28 commands. If the device is
larger than this, we have to cap it to 2^28-1.

At least NetBSD happens to be using this value to determine whether to use
LBA28 or LBA48 for its commands, using LBA28 for sectors that don't need
LBA48. This commit thus fixes NetBSD access to disks larger than 128GiB.

Signed-off-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
Message-Id: <20210824104344.3878849-1-samuel.thibault@ens-lyon.org>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2021-11-02 13:02:46 +01:00
Peter Lieven
0347a8fd4c block/rbd: implement bdrv_co_block_status
the qemu rbd driver currently lacks support for bdrv_co_block_status.
This results mainly in incorrect progress during block operations (e.g.
qemu-img convert with an rbd image as source).

This patch utilizes the rbd_diff_iterate2 call from librbd to detect
allocated and unallocated (all zero areas).

To avoid querying the ceph OSDs for the answer this is only done if
the image has the fast-diff feature which depends on the object-map and
exclusive-lock features. In this case it is guaranteed that the information
is present in memory in the librbd client and thus very fast.

If fast-diff is not available all areas are reported to be allocated
which is the current behaviour if bdrv_co_block_status is not implemented.

Signed-off-by: Peter Lieven <pl@kamp.de>
Message-Id: <20211012152231.24868-1-pl@kamp.de>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2021-11-02 13:02:46 +01:00
Kevin Wolf
bfb8aa6d58 block: Fail gracefully when blockdev-snapshot creates loops
Using blockdev-snapshot to append a node as an overlay to itself, or to
any of its parents, causes crashes. Catch the condition and return an
error for these cases instead.

Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1824363
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20211018134714.48438-1-kwolf@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2021-11-02 13:02:46 +01:00
Ari Sundholm
13a028336f block/file-posix: Fix return value translation for AIO discards
AIO discards regressed as a result of the following commit:
	0dfc7af2 block/file-posix: Optimize for macOS

When trying to run blkdiscard within a Linux guest, the request would
fail, with some errors in dmesg:

---- [ snip ] ----
[    4.010070] sd 2:0:0:0: [sda] tag#0 FAILED Result: hostbyte=DID_OK
driverbyte=DRIVER_SENSE
[    4.011061] sd 2:0:0:0: [sda] tag#0 Sense Key : Aborted Command
[current]
[    4.011061] sd 2:0:0:0: [sda] tag#0 Add. Sense: I/O process
terminated
[    4.011061] sd 2:0:0:0: [sda] tag#0 CDB: Unmap/Read sub-channel 42
00 00 00 00 00 00 00 18 00
[    4.011061] blk_update_request: I/O error, dev sda, sector 0
---- [ snip ] ----

This turns out to be a result of a flaw in changes to the error value
translation logic in handle_aiocb_discard(). The default return value
may be left untranslated in some configurations, and the wrong variable
is used in one translation.

Fix both issues.

Fixes: 0dfc7af2b2 ("block/file-posix: Optimize for macOS")
Cc: qemu-stable@nongnu.org
Signed-off-by: Ari Sundholm <ari@tuxera.com>
Signed-off-by: Emil Karlson <jkarlson@tuxera.com>
Reviewed-by: Akihiko Odaki <akihiko.odaki@gmail.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20211019110954.4170931-1-ari@tuxera.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2021-11-02 13:02:46 +01:00
Richard Henderson
8cb41fda78 Machine core patches
- Move GPIO code out of qdev.c
 - Move hotplug code out of qdev.c
 - Restrict various files to sysemu
 - Move SMP code out of machine.c
 - Add SMP parsing unit tests
 - Move dynamic sysbus device check earlier
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEE+qvnXhKRciHc/Wuy4+MsLN6twN4FAmGANZAACgkQ4+MsLN6t
 wN6Afw/+OnBSs8ZjSc/01nWX/lxk0lgS9np69gA6+sfJI7tuYimXUnfU6ZAv4laF
 65dPCAZ4F4nGzo10pQqnpeez+1d6YfxVniPvTO0Z75HiUR6L7E5B43f16HJ2U+zG
 qjXCP45uYH5u/jgzIQ//BeKiPE2hwr7XM6hzJb9DiUPyaLIETyjnzvARSyIx202a
 p/lP6qTsjZvGwt21rAurqoulmwrkbDh6v7znmNUw8siwkGuKlGsvhkJuENZlbRPA
 J3pGWm2d/Xvs8+fZEzRdg+LjFTt6jBdPm1IkSh4G2e2Nu3qtDrlxsg97W0D84sGy
 Fz0JlgVMF/aw+ExSImWc6Dnq26tdCixaZIQDgvBPNFop1SLD9wY7L4VKPdlly7bh
 a1YabtaYqfjS7R0h2sAX7FEBSoAzxa1Nrr2/pB2aCziutVCLsu5Yz87hk8C08oF7
 RyuSsYixZWtVMpmOaxOS9swYviM5OhFQkX6N6cew0d/0/D3ZRyAbr3XMcftqz/FW
 SO6n2ZMXcFQiHFZei39ZV72jx3lV9uAunADKAcSVVCaXfpyXVgGLSOxZ7mhOfcgd
 z63vpAqHhiTp82pzygArz1Lmr9OlhOi5TBt6gQ9dZvjCDRG3yoodVRJIEkMZLYEf
 kaFQ9WgB/apCG49tvjTFSZJvySgzN+aGiD6BriOGSnoYu5Gs3wU=
 =NzoC
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/philmd/tags/machine-20211101' into staging

Machine core patches

- Move GPIO code out of qdev.c
- Move hotplug code out of qdev.c
- Restrict various files to sysemu
- Move SMP code out of machine.c
- Add SMP parsing unit tests
- Move dynamic sysbus device check earlier

# gpg: Signature made Mon 01 Nov 2021 02:44:32 PM EDT
# gpg:                using RSA key FAABE75E12917221DCFD6BB2E3E32C2CDEADC0DE
# gpg: Good signature from "Philippe Mathieu-Daudé (F4BUG) <f4bug@amsat.org>" [full]

* remotes/philmd/tags/machine-20211101:
  machine: remove the done notifier for dynamic sysbus device type check
  qdev-monitor: Check sysbus device type before creating it
  machine: add device_type_is_dynamic_sysbus function
  tests/unit: Add an unit test for smp parsing
  hw/core/machine: Split out the smp parsing code
  hw/core: Restrict hotplug to system emulation
  hw/core: Extract hotplug-related functions to qdev-hotplug.c
  hw/core: Declare meson source set
  hw/core: Restrict sysemu specific files
  machine: Move gpio code to hw/core/gpio.c

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-11-02 05:53:45 -04:00
Damien Hedde
db78a60559 machine: remove the done notifier for dynamic sysbus device type check
Now that we check sysbus device types during device creation, we
can remove the check in the machine init done notifier.
This was the only thing done by this notifier, so we remove the
whole sysbus_notifier structure of the MachineState.

Note: This notifier was checking all /peripheral and /peripheral-anon
sysbus devices. Now we only check those added by -device cli option or
device_add qmp command when handling the command/option. So if there
are some devices added in one of these containers manually (eg in
machine C code), these will not be checked anymore.
This use case does not seem to appear apart from
hw/xen/xen-legacy-backend.c (it uses qdev_set_id() and in this case,
not for a sysbus device, so it's ok).

Signed-off-by: Damien Hedde <damien.hedde@greensocs.com>
Acked-by: Alistair Francis <alistair.francis@wdc.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Acked-by: Eduardo Habkost <ehabkost@redhat.com>
Message-Id: <20211029142258.484907-4-damien.hedde@greensocs.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2021-11-01 19:44:11 +01:00
Damien Hedde
312e1b1f3e qdev-monitor: Check sysbus device type before creating it
Add an early check to test if the requested sysbus device type
is allowed by the current machine before creating the device. This
impacts both -device cli option and device_add qmp command.

Before this patch, the check was done well after the device has
been created (in a machine init done notifier). We can now report
the error right away.

Signed-off-by: Damien Hedde <damien.hedde@greensocs.com>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Acked-by: Eduardo Habkost <ehabkost@redhat.com>
Message-Id: <20211029142258.484907-3-damien.hedde@greensocs.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2021-11-01 19:44:11 +01:00
Damien Hedde
b5fdf4102f machine: add device_type_is_dynamic_sysbus function
Right now the allowance check for adding a sysbus device using
-device cli option (or device_add qmp command) is done well after
the device has been created. It is done during the machine init done
notifier: machine_init_notify() in hw/core/machine.c

This new function will allow us to do the check at the right time and
issue an error if it fails.

Also make device_is_dynamic_sysbus() use the new function.

Signed-off-by: Damien Hedde <damien.hedde@greensocs.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Acked-by: Eduardo Habkost <ehabkost@redhat.com>
Message-Id: <20211029142258.484907-2-damien.hedde@greensocs.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2021-11-01 19:44:11 +01:00
Yanan Wang
9e8e393bb7 tests/unit: Add an unit test for smp parsing
Now that we have a generic parser smp_parse(), let's add an unit
test for the code. All possible valid/invalid SMP configurations
that the user can specify are covered.

Signed-off-by: Yanan Wang <wangyanan55@huawei.com>
Reviewed-by: Andrew Jones <drjones@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20211026034659.22040-3-wangyanan55@huawei.com>
Acked-by: Eduardo Habkost <ehabkost@redhat.com>
Message-Id: <bfed7144-af86-7098-e7a6-731ff13c2cf7@huawei.com>
[PMD: Squashed format string fixup from Yanan Wang]
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2021-11-01 19:44:11 +01:00
Yanan Wang
86ce2d28fa hw/core/machine: Split out the smp parsing code
We are going to introduce an unit test for the parser smp_parse()
in hw/core/machine.c, but now machine.c is only built in softmmu.

In order to solve the build dependency on the smp parsing code and
avoid building unrelated stuff for the unit tests, move the tested
code from machine.c into a separate file, i.e., machine-smp.c and
build it in common field.

Signed-off-by: Yanan Wang <wangyanan55@huawei.com>
Reviewed-by: Andrew Jones <drjones@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20211026034659.22040-2-wangyanan55@huawei.com>
Acked-by: Eduardo Habkost <ehabkost@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2021-11-01 19:44:11 +01:00
Philippe Mathieu-Daudé
09112bef74 hw/core: Restrict hotplug to system emulation
Restrict hotplug to system emulation, add stubs for the other uses.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Yanan Wang <wangyanan55@huawei.com>
Acked-by: Eduardo Habkost <ehabkost@redhat.com>
Message-Id: <20211028150521.1973821-5-philmd@redhat.com>
2021-11-01 19:44:11 +01:00
Philippe Mathieu-Daudé
81c7b38155 hw/core: Extract hotplug-related functions to qdev-hotplug.c
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Yanan Wang <wangyanan55@huawei.com>
Acked-by: Eduardo Habkost <ehabkost@redhat.com>
Message-Id: <20211028150521.1973821-4-philmd@redhat.com>
2021-11-01 19:44:11 +01:00
Philippe Mathieu-Daudé
f73fb06395 hw/core: Declare meson source set
As we want to be able to conditionally add files to the hw/core
file list, use a source set.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Yanan Wang <wangyanan55@huawei.com>
Tested-by: Yanan Wang <wangyanan55@huawei.com>
Acked-by: Eduardo Habkost <ehabkost@redhat.com>
Message-Id: <20211028150521.1973821-3-philmd@redhat.com>
2021-11-01 19:44:11 +01:00
Philippe Mathieu-Daudé
d675b44ecc hw/core: Restrict sysemu specific files
All these files don't make sense for tools and user emulation,
restrict them to system emulation.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Yanan Wang <wangyanan55@huawei.com>
Tested-by: Yanan Wang <wangyanan55@huawei.com>
Acked-by: Eduardo Habkost <ehabkost@redhat.com>
Message-Id: <20211028150521.1973821-2-philmd@redhat.com>
2021-11-01 19:44:11 +01:00
Eduardo Habkost
b19a3e2cd9 machine: Move gpio code to hw/core/gpio.c
Only softmmu code uses gpio, so move gpio code from qdev.c to
gpio.c and compile it only on softmmu mode.

Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20190425200051.19906-2-ehabkost@redhat.com>
[PMD: Rebased]
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2021-11-01 19:44:00 +01:00
Richard Henderson
4b0bf11c5a Pull request
-----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEE+ber27ys35W+dsvQfe+BBqr8OQ4FAmGAJA0ACgkQfe+BBqr8
 OQ5grBAAqTcjraHtvU6qptGTgpfQC7B4Zg4WaY7Ygcq61gOxE7x5UekJplKjfBld
 nFpZ2sAXMsFspcFxL+iZDWG12glJ/EvriPz12UqvC/OixSY9jlh53jObJxy2zKUq
 s66lSwl4N+W9ITDXBIFbawLo4SXwcXiclQodyo8Dw8czB4+gU7v7sAeVZ8RRYcJ0
 hJPwtOMuuNpZzxReBN7ox9kiF1EbrFLtgGufc2CW+60pIIVrUS55ccIpL/lNGD3k
 EnfhtrUscMu5o+vLfeQ/LPp4jzhwQjjymWhHF9GJIj3I7FGfv/U5AUMfgARpErcB
 AO2ofGVyvahoNoZ3T44MWyeAV8FzDNnPw5qYIsJ6YIVSvQvzLlKcehWY9ximmqw+
 GRXwvJrKDX/31Jc86EinYA+oUbm6HTeI4BCiMY+CHJJTGRhTAaNDviX9jRyh5naw
 B85RaK34aeDuGuyDgMbWEaTg7ZtvXhhtTXfx6EBeCencWQTQuRrmOjYSsruiifjJ
 ++8HDY/15UfQW8ArUnOX146cX6XknySCm+r3Y+2W4YEdKE6C37GC5UTKbJMKwaRv
 3PK7WVI0V4qUhU523yDYM5PhrCB62BUBjq93tUzPF44BTTnT8O7WGoKB6g7V0LJO
 0c8PZPP18/ij6vKfUdWTfVQKrwYD4/f2lJH/Ht1uhe2l571epAc=
 =YD4J
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/jsnow/tags/python-pull-request' into staging

Pull request

# gpg: Signature made Mon 01 Nov 2021 01:29:49 PM EDT
# gpg:                using RSA key F9B7ABDBBCACDF95BE76CBD07DEF8106AAFC390E
# gpg: Good signature from "John Snow (John Huston) <jsnow@redhat.com>" [full]

* remotes/jsnow/tags/python-pull-request: (22 commits)
  python, iotests: replace qmp with aqmp
  python/aqmp: Create sync QMP wrapper for iotests
  iotests/300: avoid abnormal shutdown race condition
  iotests: Conditionally silence certain AQMP errors
  iotests: Accommodate async QMP Exception classes
  python/aqmp: Remove scary message
  python/machine: Handle QMP errors on close more meticulously
  python/machine: remove has_quit argument
  python: Add iotest linters to test suite
  iotests/linters: Add workaround for mypy bug #9852
  iotests/linters: Add entry point for linting via Python CI
  iotests: split linters.py out from 297
  iotests/297: split test into sub-cases
  iotests/297: update tool availability checks
  iotests/297: Change run_linter() to raise an exception on failure
  iotests/297: refactor run_[mypy|pylint] as generic execution shim
  iotests/297: Split run_linters apart into run_pylint and run_mypy
  iotests/297: Don't rely on distro-specific linter binaries
  iotests/297: Create main() function
  iotests/297: Add get_files() function
  ...

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-11-01 14:34:15 -04:00
John Snow
76cd358671 python, iotests: replace qmp with aqmp
Swap out the synchronous QEMUMonitorProtocol from qemu.qmp with the sync
wrapper from qemu.aqmp instead.

Add an escape hatch in the form of the environment variable
QEMU_PYTHON_LEGACY_QMP which allows you to cajole QEMUMachine into using
the old implementation, proving that both implementations work
concurrently.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-id: 20211026175612.4127598-9-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-11-01 11:54:59 -04:00
John Snow
f122be6093 python/aqmp: Create sync QMP wrapper for iotests
This is a wrapper around the async QMPClient that mimics the old,
synchronous QEMUMonitorProtocol class. It is designed to be
interchangeable with the old implementation.

It does not, however, attempt to mimic Exception compatibility.

Signed-off-by: John Snow <jsnow@redhat.com>
Acked-by: Hanna Reitz <hreitz@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-id: 20211026175612.4127598-8-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-11-01 11:54:59 -04:00
John Snow
8f05aee533 iotests/300: avoid abnormal shutdown race condition
Wait for the destination VM to close itself instead of racing to shut it
down first, which produces different error log messages from AQMP
depending on precisely when we tried to shut it down.

(For example: We may try to issue 'quit' immediately prior to the target
VM closing its QMP socket, which will cause an ECONNRESET error to be
logged. Waiting for the VM to exit itself avoids the race on shutdown
behavior.)

Reported-by: Hanna Reitz <hreitz@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-id: 20211026175612.4127598-7-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-11-01 11:54:59 -04:00
John Snow
206dc47548 iotests: Conditionally silence certain AQMP errors
AQMP likes to be very chatty about errors it encounters. In general,
this is good because it allows us to get good diagnostic information for
otherwise complex async failures.

For example, during a failed QMP connection attempt, we might see:

+ERROR:qemu.aqmp.qmp_client.qemub-2536319:Negotiation failed: EOFError
+ERROR:qemu.aqmp.qmp_client.qemub-2536319:Failed to establish session: EOFError

This might be nice in iotests output, because failure scenarios
involving the new QMP library will be spelled out plainly in the output
diffs.

For tests that are intentionally causing this scenario though, filtering
that log output could be a hassle. For now, add a context manager that
simply lets us toggle this output off during a critical region.

(Additionally, a forthcoming patch allows the use of either legacy or
async QMP to be toggled with an environment variable. In this
circumstance, we can't amend the iotest output to just always expect the
error message, either. Just suppress it for now. More rigorous log
filtering can be investigated later if/when it is deemed safe to
permanently replace the legacy QMP library.)

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-id: 20211026175612.4127598-6-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-11-01 11:54:59 -04:00
John Snow
3bd559467d iotests: Accommodate async QMP Exception classes
(But continue to support the old ones for now, too.)

There are very few cases of any user of QEMUMachine or a subclass
thereof relying on a QMP Exception type. If you'd like to check for
yourself, you want to grep for all of the derivatives of QMPError,
excluding 'AQMPError' and its derivatives. That'd be these:

- QMPError
- QMPConnectError
- QMPCapabilitiesError
- QMPTimeoutError
- QMPProtocolError
- QMPResponseError
- QMPBadPortError

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-id: 20211026175612.4127598-5-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-11-01 11:54:59 -04:00
John Snow
0f71c9a936 python/aqmp: Remove scary message
The scary message interferes with the iotests output. Coincidentally, if
iotests works by removing this, then it's good evidence that we don't
really need to scare people away from using it.

Signed-off-by: John Snow <jsnow@redhat.com>
Acked-by: Hanna Reitz <hreitz@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-id: 20211026175612.4127598-4-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-11-01 11:54:59 -04:00
John Snow
49a608b8c2 python/machine: Handle QMP errors on close more meticulously
To use the AQMP backend, Machine just needs to be a little more diligent
about what happens when closing a QMP connection. The operation is no
longer a freebie in the async world; it may return errors encountered in
the async bottom half on incoming message receipt, etc.

(AQMP's disconnect, ultimately, serves as the quiescence point where all
async contexts are gathered together, and any final errors reported at
that point.)

Because async QMP continues to check for messages asynchronously, it's
almost certainly likely that the loop will have exited due to EOF after
issuing the last 'quit' command. That error will ultimately be bubbled
up when attempting to close the QMP connection. The manager class here
then is free to discard it -- if it was expected.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-id: 20211026175612.4127598-3-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-11-01 11:54:59 -04:00
John Snow
b9420e4f4b python/machine: remove has_quit argument
If we spy on the QMP commands instead, we don't need callers to remember
to pass it. Seems like a fair trade-off.

The one slightly weird bit is overloading this instance variable for
wait(), where we use it to mean "don't issue the qmp 'quit'
command". This means that wait() will "fail" if the QEMU process does
not terminate of its own accord.

In most cases, we probably did already actually issue quit -- some
iotests do this -- but in some others, we may be waiting for QEMU to
terminate for some other reason, such as a test wherein we tell the
guest (directly) to shut down.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-id: 20211026175612.4127598-2-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-11-01 11:54:59 -04:00
John Snow
461044ceb4 python: Add iotest linters to test suite
Run mypy and pylint on the iotests files directly from the Python CI
test infrastructure. This ensures that any accidental breakages to the
qemu.[qmp|aqmp|machine|utils] packages will be caught by that test
suite.

It also ensures that these linters are run with well-known versions and
test against a wide variety of python versions, which helps to find
accidental cross-version python compatibility issues.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-id: 20211019144918.3159078-15-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-11-01 11:54:59 -04:00
John Snow
558dbe9935 iotests/linters: Add workaround for mypy bug #9852
This one is insidious: if you write an import as "from {namespace}
import {subpackage}" as mirror-top-perms (now) does, mypy will fail on
every-other invocation *if* the package being imported is a typed,
installed, namespace-scoped package.

Upsettingly, that's exactly what 'qemu.[aqmp|qmp|machine]' et al are in
the context of Python CI tests.

Now, I could just edit mirror-top-perms to avoid this invocation, but
since I tripped on a landmine, I might as well head it off at the pass
and make sure nobody else trips on that same landmine.

It seems to have something to do with the order in which files are
checked as well, meaning the random order in which set(os.listdir())
produces the list of files to test will cause problems intermittently
and not just strictly "every other run".

This will be fixed in mypy >= 0.920, which is not released yet. The
workaround for now is to disable incremental checking, which avoids the
issue.

Note: This workaround is not applied when running iotest 297 directly,
because the bug does not surface there! Given the nature of CI jobs not
starting with any stale cache to begin with, this really only has a
half-second impact on manual runs of the Python test suite when executed
directly by a developer on their local machine. The workaround may be
removed when the Python package requirements can stipulate mypy 0.920 or
higher, which can happen as soon as it is released. (Barring any
unforseen compatibility issues that 0.920 may bring with it.)

See also:
 https://github.com/python/mypy/issues/11010
 https://github.com/python/mypy/issues/9852

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-id: 20211019144918.3159078-14-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-11-01 11:54:59 -04:00
John Snow
a429443530 iotests/linters: Add entry point for linting via Python CI
We need at least a tiny little shim here to join test file discovery
with test invocation. This logic could conceivably be hosted somewhere
in python/, but I felt it was strictly the least-rude thing to keep the
test logic here in iotests/, even if this small function isn't itself an
iotest.

Note that we don't actually even need the executable bit here, we'll be
relying on the ability to run this module as a script using Python CLI
arguments. No chance it gets misunderstood as an actual iotest that way.

(It's named, not in tests/, doesn't have the execute bit, and doesn't
have an execution shebang.)

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-id: 20211019144918.3159078-13-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-11-01 11:54:59 -04:00
John Snow
c293ba55c5 iotests: split linters.py out from 297
Now, 297 is just the iotests-specific incantations and linters.py is as
minimal as I can think to make it. The only remaining element in here
that ought to be configuration and not code is the list of skip files,
but they're still numerous enough that repeating them for mypy and
pylint configurations both would be ... a hassle.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-id: 20211019144918.3159078-12-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-11-01 11:54:59 -04:00
John Snow
85cfec53d0 iotests/297: split test into sub-cases
Take iotest 297's main() test function and split it into two sub-cases
that can be skipped individually. We can also drop custom environment
setup from the pylint test as it isn't needed.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-id: 20211019144918.3159078-11-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-11-01 11:54:59 -04:00
John Snow
7a90bcc269 iotests/297: update tool availability checks
As mentioned in 'iotests/297: Don't rely on distro-specific linter
binaries', these checks are overly strict. Update them to be in-line
with how we actually invoke the linters themselves.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-id: 20211019144918.3159078-10-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-11-01 11:54:59 -04:00
John Snow
752f425d83 iotests/297: Change run_linter() to raise an exception on failure
Instead of using a process return code as the python function return
value (or just not returning anything at all), allow run_linter() to
raise an exception instead.

The responsibility for printing output on error shifts from the function
itself to the caller, who will know best how to present/format that
information. (Also, "suppress_output" is now a lot more accurate of a
parameter name.)

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-id: 20211019144918.3159078-9-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-11-01 11:54:59 -04:00
John Snow
a4bde73629 iotests/297: refactor run_[mypy|pylint] as generic execution shim
There's virtually nothing special here anymore; we can combine these
into a single, rather generic function.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-id: 20211019144918.3159078-8-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-11-01 11:54:59 -04:00
John Snow
2d804f55b4 iotests/297: Split run_linters apart into run_pylint and run_mypy
Move environment setup into main(), and split the actual linter
execution into run_pylint and run_mypy, respectively.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-id: 20211019144918.3159078-7-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-11-01 11:54:59 -04:00
John Snow
f1be6219c5 iotests/297: Don't rely on distro-specific linter binaries
'pylint-3' is another Fedora-ism. Use "python3 -m pylint" or "python3 -m
mypy" to access these scripts instead. This style of invocation will
prefer the "correct" tool when run in a virtual environment.

Note that we still check for "pylint-3" before the test begins -- this
check is now "overly strict", but shouldn't cause anything that was
already running correctly to start failing. This is addressed by a
commit later in this series;
  'iotests/297: update tool availability checks'.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-id: 20211019144918.3159078-6-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-11-01 11:54:59 -04:00
John Snow
447aebda3f iotests/297: Create main() function
Instead of running "run_linters" directly, create a main() function that
will be responsible for environment setup, leaving run_linters()
responsible only for execution of the linters.

(That environment setup will be moved over in forthcoming commits.)

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-id: 20211019144918.3159078-5-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-11-01 11:54:59 -04:00
John Snow
3c1d5012e8 iotests/297: Add get_files() function
Split out file discovery into its own method to begin separating out
configuration/setup and test execution.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-id: 20211019144918.3159078-4-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-11-01 11:54:59 -04:00
John Snow
8f7960fa31 iotests/297: Split mypy configuration out into mypy.ini
More separation of code and configuration.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-id: 20211019144918.3159078-3-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-11-01 11:54:59 -04:00
John Snow
95a556e7c7 iotests/297: Move pylint config into pylintrc
Move --score=n and --notes=XXX,FIXME into pylintrc. This pulls
configuration out of code, which I think is probably a good thing in
general.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Message-id: 20211019144918.3159078-2-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
2021-11-01 11:54:59 -04:00
Richard Henderson
af531756d2 Renesas SH-4 patches queue
Patches from Zoltan:
 - Various clean up to align the code style with the rest of the code base
 - QOM'ify the SH_SERIAL device
 - Modify few memory region size to better match the hardware manual
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEE+qvnXhKRciHc/Wuy4+MsLN6twN4FAmF9ez8ACgkQ4+MsLN6t
 wN5jIBAAySm8Q7cDR/CVI+ubaAD1dW0sBAH6WVFd+P/pz6XotvLH9qvFjspTrKmw
 QujMLr/oDP3z/JvVZGpe3ArCkCoqoy9Q+hDFYfwgGmOun0p6bLLw7GC6qoWUwdfE
 BPVaZIbeW15ujn7Cawul+2rU2TIwcp+rtSg4KnKuLQp+1kKrkVlZDYp4YZeUjpIJ
 98KhnLRO38V/VUVvtuJjK/Vksc7mdP2es1kmStq5HKGYXKCTd68JAPi7OCxg4mKj
 IxGyamy8aK6khW1ccIeaCJv6NStHR1rVlFQDG10HildAcOvrJDUyQjBqYbDfh/Gj
 zsbpXw0fwz+GWmQlj2d7ndfYNNqr+G6IwBNrzVcnDoJYlVmLrJUOYnY509hcVhVY
 c33XgXYSurSskG5kSdhpZLIN4inrnRoX410gCi14mqxcjl88DTbeIjqUzizaHzRU
 xCgIAjNKnryEimGb8cEShftBT/U9t/VeXXxfe+MfoBCG0sKczUp6QVHhkq6dH5qV
 xFAZOQyxw6lLaTrcQQByCBh+NFyqtrYFsqS1b1Or3z0ZfKF65Y8M5sx1RufhybfV
 UutGY7JaZU3MJEeT6UY9XHSZXwGLajixjXuYR25Gj/LYgOSMIP0iqsci2V0AfBLQ
 u1Z54qIllDVAickaHLlBfI2jn6XXfzZ09+POHOsf//b39i1QjIk=
 =ktI+
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/philmd/tags/renesas-20211030' into staging

Renesas SH-4 patches queue

Patches from Zoltan:
- Various clean up to align the code style with the rest of the code base
- QOM'ify the SH_SERIAL device
- Modify few memory region size to better match the hardware manual

# gpg: Signature made Sat 30 Oct 2021 10:05:03 AM PDT
# gpg:                using RSA key FAABE75E12917221DCFD6BB2E3E32C2CDEADC0DE
# gpg: Good signature from "Philippe Mathieu-Daudé (F4BUG) <f4bug@amsat.org>" [full]

* remotes/philmd/tags/renesas-20211030: (30 commits)
  hw/timer/sh_timer: Remove use of hw_error
  hw/timer/sh_timer: Fix timer memory region size
  hw/timer/sh_timer: Do not wrap lines that are not too long
  hw/timer/sh_timer: Rename sh_timer_state to SHTimerState
  hw/intc/sh_intc: Remove unneeded local variable initialisers
  hw/intc/sh_intc: Simplify allocating sources array
  hw/intc/sh_intc: Avoid using continue in loops
  hw/intc/sh_intc: Replace abort() with g_assert_not_reached()
  hw/intc/sh_intc: Inline and drop sh_intc_source() function
  hw/intc/sh_intc: Use array index instead of pointer arithmetics
  hw/intc/sh_intc: Remove excessive parenthesis
  hw/intc/sh_intc: Move sh_intc_register() closer to its only user
  hw/intc/sh_intc: Drop another useless macro
  hw/intc/sh_intc: Rename iomem region
  hw/intc/sh_intc: Turn some defines into an enum
  hw/intc/sh_intc: Use existing macro instead of local one
  hw/char/sh_serial: Add device id to trace output
  hw/char/sh_serial: QOM-ify
  hw/char/sh_serial: Split off sh_serial_reset() from sh_serial_init()
  hw/char/sh_serial: Embed QEMUTimer in state struct
  ...

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-10-30 11:31:41 -07:00
BALATON Zoltan
46e44759fc hw/timer/sh_timer: Remove use of hw_error
The hw_error function calls abort and is not meant to be used by
devices. Use qemu_log_mask instead to log and ignore invalid accesses.
Also fix format strings to allow dropping type casts of hwaddr and use
__func__ instead of hard coding function name in the message which
were wrong in two cases.

Signed-off-by: BALATON Zoltan <balaton@eik.bme.hu>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <f818dc3dd2ac8c3b3d53067f316a716d7f9683d8.1635541329.git.balaton@eik.bme.hu>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2021-10-30 18:39:37 +02:00
BALATON Zoltan
65307c7792 hw/timer/sh_timer: Fix timer memory region size
The timer unit only has registers that fit in a region 0x30 bytes
long. No need to have the timer region larger than that.

Signed-off-by: BALATON Zoltan <balaton@eik.bme.hu>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <b1cd196cf1395a602c7a08a4f858e69e50c446a1.1635550060.git.balaton@eik.bme.hu>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2021-10-30 18:39:37 +02:00
BALATON Zoltan
f64ccec414 hw/timer/sh_timer: Do not wrap lines that are not too long
It's more readable to keep things on one line if it fits the length limit.

Signed-off-by: BALATON Zoltan <balaton@eik.bme.hu>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <97bc2a38991f33fd0c8cc2e4d0a3a29b20c47d1f.1635541329.git.balaton@eik.bme.hu>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2021-10-30 18:39:37 +02:00
BALATON Zoltan
5d9b737e51 hw/timer/sh_timer: Rename sh_timer_state to SHTimerState
According to coding style types should be camel case, also remove
unneded casts from void *.

Signed-off-by: BALATON Zoltan <balaton@eik.bme.hu>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <d9a9d160c1153a583397e366ab06477f5a31c507.1635541329.git.balaton@eik.bme.hu>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2021-10-30 18:39:37 +02:00
BALATON Zoltan
3233178789 hw/intc/sh_intc: Remove unneeded local variable initialisers
The sh_intc_locate function will either init these or not return so no
need to initialise them.

Signed-off-by: BALATON Zoltan <balaton@eik.bme.hu>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <15e04aa665c68ab5df47bbf505346d413be2fc1c.1635541329.git.balaton@eik.bme.hu>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2021-10-30 18:39:37 +02:00
BALATON Zoltan
36cf5ee885 hw/intc/sh_intc: Simplify allocating sources array
Use g_new0 instead of g_malloc0 and avoid some unneeded temporary
variable assignments.

Signed-off-by: BALATON Zoltan <balaton@eik.bme.hu>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <72efc4f2c4ff8b96848d03dca08e4541ee4076f6.1635541329.git.balaton@eik.bme.hu>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2021-10-30 18:39:37 +02:00
BALATON Zoltan
418a221c2b hw/intc/sh_intc: Avoid using continue in loops
Instead of if !expr continue else do something it is more straight
forward to say if expr then do something, especially if the action is
just a few lines. Remove such uses of continue to make the code easier
to follow.

Signed-off-by: BALATON Zoltan <balaton@eik.bme.hu>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <0efaa5e7a1a3ee11f82b3bb1942c287576c67f8b.1635541329.git.balaton@eik.bme.hu>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2021-10-30 18:39:37 +02:00
BALATON Zoltan
85208f7a97 hw/intc/sh_intc: Replace abort() with g_assert_not_reached()
All the places that call abort should not happen which is better
marked by g_assert_not_reached.

Signed-off-by: BALATON Zoltan <balaton@eik.bme.hu>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <039e6a784532f2af27f8adeafdb8e0391722f567.1635541329.git.balaton@eik.bme.hu>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2021-10-30 18:39:37 +02:00
BALATON Zoltan
9b12fb10b7 hw/intc/sh_intc: Inline and drop sh_intc_source() function
This function is very simple and provides no advantage. Call sites
become simpler without it so just write it in line and drop the
separate function.

Signed-off-by: BALATON Zoltan <balaton@eik.bme.hu>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <a98d1f7f94e91a42796b7d91e9153a7eaa3d1c44.1635541329.git.balaton@eik.bme.hu>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2021-10-30 18:39:37 +02:00
BALATON Zoltan
12201fe38a hw/intc/sh_intc: Use array index instead of pointer arithmetics
Address of element i is one word thus clearer than array + i.

Signed-off-by: BALATON Zoltan <balaton@eik.bme.hu>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <f49c9b1dee1fcaf374b092d862a6821907d5fcdc.1635541329.git.balaton@eik.bme.hu>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2021-10-30 18:39:37 +02:00