Commit Graph

224 Commits

Author SHA1 Message Date
Paolo Bonzini
993e38020f docs, tests: do not specify scsi=off
This has been the default forever.

Acked-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-06-05 11:00:56 +02:00
Marcin Juszkiewicz
c19779ce18 tests/avocado: update sbsa-ref firmware
Partial support for NUMA setup:
- cpu nodes
- memory nodes

Used versions:

- Trusted Firmware v2.11.0
- Tianocore EDK2 stable202405
- Tianocore EDK2 Platforms code commit 4bbd0ed

Firmware is built using Debian 'bookworm' cross toolchain (gcc 12.2.0).

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2024-05-30 13:21:06 +01:00
Peter Maydell
dcc5c018c7 tests/avocado: update sunxi kernel from armbian to 6.6.16
The Linux kernel 5.10.16 binary for sunxi has been removed from
apt.armbian.com. This means that the avocado tests for these machines
will be skipped (status CANCEL) if the old binary isn't present in
the avocado cache.

Update to 6.6.16, in the same way we did in commit e384db41d8
when we moved to 5.10.16 in 2021.

Cc: qemu-stable@nongnu.org
Resolves: https://gitlab.com/qemu-project/qemu/-/issues/2284
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Strahinja Jankovic <strahinja.p.jankovic@gmail.com>
Reviewed-by: Niek Linnenbank <nieklinnenbank@gmail.com>
Tested-by: Niek Linnenbank <nieklinnenbank@gmail.com>
Message-id: 20240415151845.1564201-1-peter.maydell@linaro.org
2024-04-30 15:06:26 +01:00
Philippe Mathieu-Daudé
6c3014858c target/nios2: Remove the deprecated Nios II target
The Nios II target is deprecated since v8.2 in commit 9997771bc1
("target/nios2: Deprecate the Nios II architecture").

Remove:
- Buildsys / CI infra
- User emulation
- System emulation (10m50-ghrd & nios2-generic-nommu machines)
- Tests

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Acked-by: Marek Vasut <marex@denx.de>
Message-Id: <20240327144806.11319-3-philmd@linaro.org>
2024-04-24 16:03:38 +02:00
Nicholas Piggin
b07a5bb736 tests/avocado: ppc_hv_tests.py set alpine time before setup-alpine
If the time is wrong, setup-alpine SSL certificate checks can fail.
setup-alpine is used to bring up the network, but it doesn't seem
to to set NTP time before the failing SSL checks. This test has
recently started failing presumably because the default time has
now fallen too far behind.

Fix this by setting time from the host time before running setup-alpine.

Fixes: c9cb496710 ("tests/avocado: ppc add hypervisor tests")
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
2024-03-30 18:50:26 +10:00
Nicholas Piggin
74eb04af18 tests/avocado: Fix ppc_hv_tests.py xorriso dependency guard
For some reason the skipIf missing_deps() check fails to skip the test
if it comes after the skipUnless lines, causing an error running on
systems without xorriso.

Avocado implements skipUnless is just an inverted skipIf, so it's not
clear what the bug is or why this fixes it. For now it's enough to
get things working.

Resolves: https://gitlab.com/qemu-project/qemu/-/issues/2246
Fixes: c9cb496710 ("tests/avocado: ppc add hypervisor tests")
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
2024-03-30 18:50:25 +10:00
Marcin Juszkiewicz
55900f5dcc tests/avocado: sbsa-ref: add OpenBSD tests for misc 'max' setup
PAuth makes run timeout on CI so add tests using 'max' without
it and with impdef one.

Signed-off-by: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20240318-sbsa-ref-firmware-update-v3-4-1c33b995a538@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2024-03-20 09:52:27 +00:00
Marcin Juszkiewicz
385ac766e8 tests/avocado: sbsa-ref: add Alpine tests for misc 'max' setup
PAuth makes run timeout on CI so add tests using 'max' without it
and with impdef one.

Signed-off-by: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20240318-sbsa-ref-firmware-update-v3-3-1c33b995a538@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2024-03-20 09:52:27 +00:00
Marcin Juszkiewicz
21f123f3c2 tests/avocado: drop virtio-rng from sbsa-ref tests
sbsa-ref is supposed to emulate real hardware so virtio-rng-pci
does not fit here

Signed-off-by: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>
Message-Id: <20240318-sbsa-ref-firmware-update-v3-2-1c33b995a538@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2024-03-20 09:52:27 +00:00
Marcin Juszkiewicz
06f6bbc3bd tests/avocado: update sbsa-ref firmware
We now have CI job to build those and publish in space with
readable urls.

Firmware is built using Debian 'bookworm' cross toolchain (gcc 12.2.0).

Used versions:

- Trusted Firmware v2.10.2
- Tianocore EDK2 stable202402
- Tianocore EDK2 Platforms code commit 085c2fb

Signed-off-by: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20240318-sbsa-ref-firmware-update-v3-1-1c33b995a538@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2024-03-20 09:52:27 +00:00
Peter Maydell
2f79d2ed64 tests/avocado/boot_linux_console.py: Add Rpi4b boot tests
Signed-off-by: Sergey Kambalin <sergey.kambalin@auriga.com>
Message-id: 20240226000259.2752893-31-sergey.kambalin@auriga.com
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
[PMM: Comment out use of USB, which depends on PCI]
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2024-02-27 13:01:43 +00:00
Nicholas Piggin
b41484ccb6 tests/avocado: Use default CPU for pseries machine
Use the default CPU with the pseries machine unless there is a
specific requirement.

Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
2024-02-23 23:24:31 +10:00
Nicholas Piggin
c9cb496710 tests/avocado: ppc add hypervisor tests
The powernv and pseries machines both provide hypervisor facilities
that are supported by KVM. This is a large and complicated set of
features that don't get much system-level testing in ppc tests.

Add a new test case for these which runs QEMU KVM inside the target.
This downloads an Alpine VM image, boots it and downloads and installs
the qemu package, then boots a virtual machine under it, re-using the
original Alpine VM image.

Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
2024-02-23 23:16:35 +10:00
Nicholas Piggin
9bf9479328 tests/avocado: Add pseries KVM boot_linux test
ppc has no avocado tests for the KVM backend. Add a KVM boot_linux.py
test for pseries.

Reviewed-by: Cédric Le Goater <clg@kaod.org>
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
2024-02-23 23:16:35 +10:00
Nicholas Piggin
8d07a8aef6 tests/avocado: Add ppc pseries and powernv hash MMU tests
POWER CPUs support hash and radix MMU modes. Linux supports running in
either mode, but defaults to radix. To keep up testing of QEMU's hash
MMU implementation, add some Linux hash boot tests.

Reviewed-by: Cédric Le Goater <clg@kaod.org>
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
2024-02-23 23:16:35 +10:00
Nicholas Piggin
234aa6d62a tests/avocado: ppc add powernv10 boot_linux_console test
Add test for POWER10.

Reviewed-by: Cédric Le Goater <clg@kaod.org>
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
2024-02-23 23:16:34 +10:00
Nicholas Piggin
458a6aa3b3 tests/avocado: improve flaky ppc/pnv boot_linux_console.py test
The expected MTD partition detection output does not always appear on
the console, despite the test reaching the boot loader and the string
appearing in dmesg. Possibly due to an init script that quietens the
console output. Using an earlier log message improves reliability.

Reviewed-by: Cédric Le Goater <clg@kaod.org>
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
2024-02-23 23:16:34 +10:00
Nicholas Piggin
e5f1500144 tests/avocado: mark boot_linux.py long runtime instead of flaky
The ppc64 and s390x tests were first marked skipIf GITLAB_CI by commit
c0c8687ef0 ("tests/avocado: disable BootLinuxPPC64 test in CI"), and
commit 0f26d94ec9 ("tests/acceptance: skip s390x_ccw_vrtio_tcg on
GitLab") due to being very heavy-weight for gitlab CI.

Commit 9b45cc9931 ("docs/devel: rationalise unstable gitlab tests under
FLAKY_TESTS") changed this to being flaky but it isn't really, it just
had a long runtime.

So take the SPEED=slow variable from qtests and introduce it to avocado,
and make these tests require it.

Reviewed-by: Cédric Le Goater <clg@kaod.org>
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
2024-02-23 23:16:34 +10:00
Manos Pitsidianakis
35a5a331e4 tests: correct typos
Correct typos automatically found with the `typos` tool
<https://crates.io/crates/typos>

Signed-off-by: Manos Pitsidianakis <manos.pitsidianakis@linaro.org>
Reviewed-by: Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
Reviewed-by: Michael Tokarev <mjt@tls.msk.ru>
Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>
2024-02-20 22:21:10 +03:00
Cédric Le Goater
560f254107 tests/avocado/machine_aspeed.py: Update buildroot images to 2023.11
Compared to mainline buildroot, these images have some customization :

- Linux version is bumped to 6.6.3 and built with a custom config
- U-Boot is switched to the one provided by OpenBMC for more support
- defconfigs extra tools for dev

See branch [1] for more details.

There are a few changes since last update, commit ed1f5ff842. Images
all have a password now and I2C devices have been updated in the Linux
ast2600-evb device tree [2]. Do the necessary adjustements.

[1] https://github.com/legoater/buildroot/commits/aspeed-2023.11
[2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=9deb10cf160e

Signed-off-by: Cédric Le Goater <clg@kaod.org>
2024-02-01 08:13:30 +01:00
Ani Sinha
7ef4c41e91 acpi/tests/avocado/bits: wait for 200 seconds for SHUTDOWN event from bits VM
By default, the timeout to receive any specified event from the QEMU VM is 60
seconds set by the python avocado test framework. Please see event_wait() and
events_wait() in python/qemu/machine/machine.py. If the matching event is not
triggered within that interval, an asyncio.TimeoutError is generated. Since the
timeout for the bits avocado test is 200 secs, we need to make event_wait()
timeout of the same value as well so that an early timeout is not triggered by
the avocado framework.

CC: peter.maydell@linaro.org
Resolves: https://gitlab.com/qemu-project/qemu/-/issues/2077
Signed-off-by: Ani Sinha <anisinha@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-id: 20240117042556.3360190-1-anisinha@redhat.com
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2024-01-18 12:47:25 +00:00
Alex Bennée
139db3cc59 tests/avocado: partially revert unmasking of replay_linux tests
It seems we were premature in declaring replay_linux.py fixed. The
x86_64 image still seems to hang occasionally. I've raised a new bug
to cover it:

    https://gitlab.com/qemu-project/qemu/-/issues/2094

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2024-01-12 13:23:48 +00:00
Alex Bennée
b9371a7b90 tests/avocado: use snapshot=on in kvm_xen_guest
This ensures the rootfs is never permanently changed as we don't need
persistence between tests anyway.

Message-Id: <20240103173349.398526-3-alex.bennee@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
2024-01-12 13:23:48 +00:00
Thomas Huth
b7f2aea63b tests/avocado: Add a test for a little-endian microblaze machine
We've already got a test for a big endian microblaze machine, but so
far we lack one for a little endian machine. Now that the QEMU advent
calendar featured such an image, we can test the little endian mode,
too.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20231215161851.71508-1-thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2024-01-12 13:23:48 +00:00
Peter Maydell
c1df5b4f16 Record/replay fixes for replay_kernel tests
- add a 32 bit x86 replay test case
   - fix some typos
   - use modern snapshot setting for tests
   - update replay_dump for current ABI
   - remove stale replay variables
   - improve kdoc for ReplayState
   - introduce common error path for replay
   - always fully drain chardevs when in replay
   - catch unexpected waitio on playback
   - remove flaky tags from replay_kernel tests
 -----BEGIN PGP SIGNATURE-----
 
 iQEzBAABCgAdFiEEZoWumedRZ7yvyN81+9DbCVqeKkQFAmWcAJgACgkQ+9DbCVqe
 KkS/TQf+PuIPtuX71ENajfRBjz6450IbGqLUJ1HEaPGYGRj+fR6rg5g5u8qaBrT7
 TUv9ef9L22NtyL+Gbs1OGpGDWKoqV6RQc+A/MHa8IKFpcS24nUo3k4psIC6NSGRH
 6w3++fPC1Q5cDk9Lei3Qt8fXzcnUZz+NTiIK05aC0xh7D6uGfdADvKqHeLav7qi+
 X2ztNdBsy/WJWCuWcMVzb/dGwDBtuyyxvqTD4EF+zn+gSYq9od2G8XdF+0o6ZVLM
 mXEHwNwB6UjOkLt2cYaay59SXcJFvwxKbEGTDnA7T+kgd3rknuBaWdVBIazoSPQh
 +522nPz5qq/3wO1l7+iQXuvd38fWyw==
 =nKRx
 -----END PGP SIGNATURE-----

Merge tag 'pull-replay-fixes-080124-1' of https://gitlab.com/stsquad/qemu into staging

Record/replay fixes for replay_kernel tests

  - add a 32 bit x86 replay test case
  - fix some typos
  - use modern snapshot setting for tests
  - update replay_dump for current ABI
  - remove stale replay variables
  - improve kdoc for ReplayState
  - introduce common error path for replay
  - always fully drain chardevs when in replay
  - catch unexpected waitio on playback
  - remove flaky tags from replay_kernel tests

# -----BEGIN PGP SIGNATURE-----
#
# iQEzBAABCgAdFiEEZoWumedRZ7yvyN81+9DbCVqeKkQFAmWcAJgACgkQ+9DbCVqe
# KkS/TQf+PuIPtuX71ENajfRBjz6450IbGqLUJ1HEaPGYGRj+fR6rg5g5u8qaBrT7
# TUv9ef9L22NtyL+Gbs1OGpGDWKoqV6RQc+A/MHa8IKFpcS24nUo3k4psIC6NSGRH
# 6w3++fPC1Q5cDk9Lei3Qt8fXzcnUZz+NTiIK05aC0xh7D6uGfdADvKqHeLav7qi+
# X2ztNdBsy/WJWCuWcMVzb/dGwDBtuyyxvqTD4EF+zn+gSYq9od2G8XdF+0o6ZVLM
# mXEHwNwB6UjOkLt2cYaay59SXcJFvwxKbEGTDnA7T+kgd3rknuBaWdVBIazoSPQh
# +522nPz5qq/3wO1l7+iQXuvd38fWyw==
# =nKRx
# -----END PGP SIGNATURE-----
# gpg: Signature made Mon 08 Jan 2024 14:03:04 GMT
# gpg:                using RSA key 6685AE99E75167BCAFC8DF35FBD0DB095A9E2A44
# gpg: Good signature from "Alex Bennée (Master Work Key) <alex.bennee@linaro.org>" [full]
# Primary key fingerprint: 6685 AE99 E751 67BC AFC8  DF35 FBD0 DB09 5A9E 2A44

* tag 'pull-replay-fixes-080124-1' of https://gitlab.com/stsquad/qemu:
  tests/avocado: remove skips from replay_kernel
  chardev: force write all when recording replay logs
  replay: stop us hanging in rr_wait_io_event
  replay/replay-char: use report_sync_error
  replay: introduce a central report point for sync errors
  replay: make has_unread_data a bool
  replay: add proper kdoc for ReplayState
  replay: remove host_clock_last
  scripts/replay_dump: track total number of instructions
  scripts/replay-dump: update to latest format
  tests/avocado: modernise the drive args for replay_linux
  tests/avocado: fix typo in replay_linux
  tests/avocado: add a simple i386 replay kernel test

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2024-01-09 10:32:04 +00:00
Ani Sinha
b345ffad3e acpi/tests/avocado/bits: disable smilatency tests
smilatncy tests in bios bits seems to generate some flakyness in running the
bits avocado tests. Disable them for now.

Resolves: https://gitlab.com/qemu-project/qemu/-/issues/2077

CC: peter.maydell@linaro.org
CC: crosa@redhat.com
CC: philmd@linaro.org
CC: bleal@redhat.com
CC: mst@redhat.com
CC: wainersm@redhat.com
Signed-off-by: Ani Sinha <anisinha@redhat.com>
Message-id: 20240108103643.4434-3-anisinha@redhat.com
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2024-01-09 10:31:47 +00:00
Ani Sinha
fc7c144b23 acpi/tests/avocado/bits: import smilatency test from bits in order to disable it
Add smilatency test script in the bits avocado tests from bios-bits. No changes
have been made to the original test script. The test will be disabled in the
subsequent patch.

CC: peter.maydell@linaro.org
CC: crosa@redhat.com
CC: philmd@linaro.org
CC: bleal@redhat.com
CC: mst@redhat.com
CC: wainersm@redhat.com
Signed-off-by: Ani Sinha <anisinha@redhat.com>
Message-id: 20240108103643.4434-2-anisinha@redhat.com
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2024-01-09 10:31:47 +00:00
Alex Bennée
c2ef5ee89d tests/avocado: remove skips from replay_kernel
With the latest fixes for #2010 and #2013 these tests look pretty
stable now. Of course the only way to be really sure is to run it in
the CI infrastructure and see what breaks.

Acked-by: Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231211091346.14616-14-alex.bennee@linaro.org>
2024-01-08 13:59:06 +00:00
Alex Bennée
6f6b71536c tests/avocado: modernise the drive args for replay_linux
QEMU complains about us not being explicit with setting snapshot so
lets do that. Also as cdroms are RO media we don't need to jump the
hoops of setting up snapshots and replay disks - just declare the
drive is a cdrom and nothing should change.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231211091346.14616-4-alex.bennee@linaro.org>
2024-01-08 13:58:31 +00:00
Alex Bennée
2d8508bbab tests/avocado: fix typo in replay_linux
Reviewed-by: Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231211091346.14616-3-alex.bennee@linaro.org>
2024-01-08 13:58:19 +00:00
Alex Bennée
940f70cc1c tests/avocado: add a simple i386 replay kernel test
There are a number of bugs against 32 bit x86 on the tracker. Lets at
least establish a baseline pure kernel boot can do record/replay
before we start looking at the devices.

Acked-by: Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
Acked-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231211091346.14616-2-alex.bennee@linaro.org>
2024-01-08 13:58:06 +00:00
Alex Bennée
2e8ed6a970 tests/avocado: mark ReplayKernelNormal.test_mips64el_malta as flaky
I missed this when going through the recent failure logs. I can run
the test 30 times without failure locally but it seems to hang pretty
reliably on GitLab's CI infra-structure.

Cc: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20231201201027.2689404-1-alex.bennee@linaro.org>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
2023-12-04 16:21:00 +01:00
Philippe Mathieu-Daudé
d369ad5558 tests/avocado: Update yamon-bin-02.22.zip URL
http://www.imgtec.com/tools/mips-tools/downloads/ redirects
to https://mips.com/downloads/yamon-version-02-22/ then points
to an invalid path to a s3 bucket. Use the correct path. The
site will eventually be fixed.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20231201205630.10837-1-philmd@linaro.org>
2023-12-04 16:21:00 +01:00
Alex Bennée
5d25fcb702 gitlab: add optional job to run flaky avocado tests
One problem with flaky tests is they often only fail under CI
conditions which makes it hard to debug. We add an optional allow_fail
job so developers can trigger the only the flaky tests in the CI
environment if they are debugging.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231201093633.2551497-8-alex.bennee@linaro.org>
2023-12-01 17:47:20 +00:00
Alex Bennée
970c6fc250 tests/avocado: tag sbsa tests as tcg only
As firmware runs at a higher privilege level than the hypervisor we
can only run these tests under TCG emulation.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231201093633.2551497-6-alex.bennee@linaro.org>
2023-12-01 17:47:20 +00:00
Alex Bennée
9b45cc9931 docs/devel: rationalise unstable gitlab tests under FLAKY_TESTS
It doesn't make sense to have two classes of flaky tests. While it may
take the constrained environment of CI to trigger failures easily it
doesn't mean they don't occasionally happen on developer machines. As
CI is the gating factor to passing there is no point developers
running the tests locally anyway unless they are trying to fix things.

While we are at it update the language in the docs to discourage the
QEMU_TEST_FLAKY_TESTS becoming a permanent solution.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231201093633.2551497-3-alex.bennee@linaro.org>
2023-12-01 14:11:55 +00:00
Thomas Huth
ea6a33e776 Revert "tests/avocado: Enable reverse_debugging.py tests in gitlab CI"
This reverts commit c4d74ab24a.

The reverse debugging test is sometimes still failing. See:
 https://gitlab.com/qemu-project/qemu/-/issues/1992

Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-ID: <20231121100842.677363-1-thuth@redhat.com>
2023-11-21 10:28:55 -05:00
Nicholas Piggin
c4d74ab24a tests/avocado: Enable reverse_debugging.py tests in gitlab CI
Let's try enable reverse_debugging.py in gitlab CI.

Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
Message-ID: <20231116115354.228678-3-npiggin@gmail.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-11-16 14:22:56 +01:00
Nicholas Piggin
cd43f00524 tests/avocado: reverse_debugging drain console to prevent hang
Like replay_linux.py, reverse_debugging.py starts the vm with console
set but does not interact with it (e.g., with wait_for_console_pattern).
In this situation, the console should have a drainer attached so the
socket does not fill. replay_linux.py has a drainer, but it is missing
from reverse_debugging.py.

Per analysis in Link: this can cause the console socket/pipe to fill and
QEMU get stuck in qemu_chr_write_buffer, leading to strange test case
failures (ppc64 fails because it prints a lot to console in early bios).
Attaching a drainer prevents this.

Note, this commit does not fix bugs introduced by the commits referenced
in the first two Fixes: tags, but together those commits conspire to
irritate the problem and cause test case failure, which this commit
fixes.

Link: https://lore.kernel.org/qemu-devel/ZVT-bY9YOr69QTPX@redhat.com/
Fixes: 1d4796cd00 ("python/machine: use socketpair() for console connections")
Fixes: 761a13b239 ("tests/avocado: ppc64 reverse debugging tests for pseries and powernv")
Fixes: be52eca309 ("tests/acceptance: add reverse debugging test")
Tested-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
Message-ID: <20231116115354.228678-1-npiggin@gmail.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-11-16 14:22:28 +01:00
Thomas Huth
572960cb23 tests/avocado/mem-addr-space-check: Replace assertEquals() for Python 3.12
assertEquals() has been removed in Python 3.12 and should be replaced by
assertEqual(). See: https://docs.python.org/3.12/whatsnew/3.12.html#id3

Message-ID: <20231116061956.14676-1-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Acked-by: Ani Sinha <anisinha@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-11-16 09:21:30 +01:00
Thomas Huth
3365f3d99a tests/avocado/replay_kernel: Mark the test_x86_64_pc as flaky
It's failing very often, so don't run this by default anymore
until it gets fixed.

Message-ID: <20231114153019.295131-1-thuth@redhat.com>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-11-16 07:13:41 +01:00
Philippe Mathieu-Daudé
9d72dd100c tests/avocado: Make fetch_asset() unconditionally require a crypto hash
In a perfect world we'd have reproducible tests,
but then we'd be sure we run the same binaries.
If a binary artifact isn't hashed, we have no idea
what we are running. Therefore enforce hashing for
all our artifacts.

With this change, unhashed artifacts produce:

  $ avocado run tests/avocado/multiprocess.py
   (1/2) tests/avocado/multiprocess.py:Multiprocess.test_multiprocess_x86_64:
   ERROR: QemuBaseTest.fetch_asset() missing 1 required positional argument: 'asset_hash' (0.19 s)

Inspired-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-ID: <20231115205149.90765-1-philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-11-16 07:13:41 +01:00
Thomas Huth
cfe52e91c4 tests/avocado/multiprocess: Add asset hashes to silence warnings
The multiprocess test is currently succeeding with an annoying warning:

 (1/2) tests/avocado/multiprocess.py:Multiprocess.test_multiprocess_x86_64:
       WARN: Test passed but there were warnings during execution. Check
       the log for details

In the log, you can find an entry like:

 WARNI| No hash provided. Cannot check the asset file integrity.

Add the proper asset hashes to avoid those warnings.

Message-ID: <20231115145852.494052-1-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-11-16 07:13:41 +01:00
Thomas Huth
9cdafe3b47 tests/avocado/intel_iommu: Add asset hashes to avoid warnings
The intel_iommu test is currently succeeding with annoying warnings.
Add the proper asset hashes to avoid those.

Message-ID: <20231114143531.291820-1-thuth@redhat.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-11-16 07:13:41 +01:00
Thomas Huth
ffab603bd5 tests/avocado/virtio-gpu: Fix test_vhost_user_vga_virgl for edid support
The "edid" feature has been added to vhost-user-gpu in commit
c06444261e ("contrib/vhost-user-gpu: implement get_edid feature"),
so waiting for "features: +virgl -edid" in the test does not work
anymore, it's "+edid" instead of "-edid" now!

While we're at it, move the expected string to the preceeding
exec_command_and_wait_for_pattern() instead (since waiting for
empty string here does not make too much sense).

Message-ID: <20231114203456.319093-1-thuth@redhat.com>
Reviewed-by: Antonio Caggiano <quic_acaggian@quicinc.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-11-16 07:13:41 +01:00
Philippe Mathieu-Daudé
f0a663b4ce tests/avocado: Replace assertRegexpMatches() for Python 3.12 compatibility
assertRegexpMatches() has been removed in Python 3.12 and should be replaced by
assertRegex(). See: https://docs.python.org/3.12/whatsnew/3.12.html#id3

Inspired-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20231114144832.71612-1-philmd@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-11-16 07:13:41 +01:00
Thomas Huth
861f724d03 tests/avocado: Replace assertEquals() for Python 3.12 compatibility
assertEquals() has been removed in Python 3.12 and should be replaced by
assertEqual(). See: https://docs.python.org/3.12/whatsnew/3.12.html#id3

Message-ID: <20231114134326.287242-1-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-11-16 07:13:41 +01:00
Thomas Huth
0b2af475e9 tests/avocado/machine_s390_ccw_virtio.py: Fix SyntaxWarnings from python 3.12
Python 3.12 now warns about backslashes in strings that aren't used
for escaping a special character from Python. Silence the warning
by using raw strings here instead.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20231113140721.46903-1-thuth@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
2023-11-13 19:06:06 +01:00
Ani Sinha
0034d0395e tests/avocado: add test to exercise processor address space memory bound checks
QEMU has validations to make sure that a VM is not started with more memory
(static and hotpluggable memory) than what the guest processor can address
directly with its addressing bits. This change adds a test to make sure QEMU
fails to start with a specific error message when an attempt is made to
start a VM with more memory than what the processor can directly address.
The test also checks for passing cases when the address space of the processor
is capable of addressing all memory. Boundary cases are tested.

CC: imammedo@redhat.com
CC: David Hildenbrand <david@redhat.com>
Acked-by: David Hildenbrand <david@redhat.com>
Acked-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Ani Sinha <anisinha@redhat.com>
Message-ID: <20231109045601.33349-1-anisinha@redhat.com>
Message-ID: <D5D8D419-76BA-4FB0-9BAC-4F7470A052FC@redhat.com>
[PMD: Use SPDX tag]
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
2023-11-13 16:56:24 +01:00
Alex Bennée
119599965e tests/avocado: update the tcg_plugins test
There are a number of things that are broken on the test currently so
lets fix that up:

  - replace retired Debian kernel for tuxrun_baseline one
  - remove "detected repeat instructions test" since ea185a55
  - log total counted instructions/memory accesses

Acked-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231106185112.2755262-8-alex.bennee@linaro.org>
2023-11-08 15:15:23 +00:00