Move the OrangePi tests from tests/avocado/boot_linux_console.py into
a new file dedicated for OrangePi tests in the functional framework
and update the hash sums of the assets to sha256 along the way.
For the buildroot image and the Armbian image, we've got to switch to
a newer version since the old images have been removed from the server,
and the NetBSD image has been moved to the archive, so we need to update
this URL as well.
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20241029092440.25021-3-thuth@redhat.com>
Move the BananaPi tests from tests/avocado/boot_linux_console.py into
a new file dedicated for Banana Pi tests in the functional framework.
Update the hash sums of the assets to sha256 along the way and fix the
broken link for the buildroot image from storage.kernelci.org.
(Note: The test_arm_bpim2u_openwrt_22_03_3 test is currently broken
due to a regression in commit 4c2c047469 ("target/arm: Fix usage of MMU
indexes when EL3 is AArch32") - it works if that commit gets reverted)
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20241029092440.25021-2-thuth@redhat.com>
A straight forward conversion, only the usual changes were required
here (i.e. adjustment for asset downloading, machine selection).
Message-ID: <20241023051754.813412-1-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
I forgot to add the tests to the meson.build file and looks
like I even managed to somehow mix up the hashsums in the
ppc64 test!
Message-ID: <20241023141919.930689-1-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Now that we are aware of binaries that are available for sh4eb,
we should make sure that there are no regressions with this
target and test it regularly in our CI.
Message-ID: <20241024082735.42324-3-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
This ensures that if a functional test runs QEMU with a writable
disk pointing to a cached asset, an error will be reported, rather
than silently modifying the cache file.
As an example, tweaking test_sbsaref.py to set snapshot=off,
results in a clear error:
Command: ./build/qemu-system-aarch64 ...snip... -drive file=/var/home/berrange/.cache/qemu/download/44cdbae275ef1bb6dab1d5fbb59473d4f741e1c8ea8a80fd9e906b531d6ad461,format=raw,snapshot=off -cpu max,pauth=off
Output: qemu-system-aarch64: Could not open '/var/home/berrange/.cache/qemu/download/44cdbae275ef1bb6dab1d5fbb59473d4f741e1c8ea8a80fd9e906b531d6ad461': Permission denied
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Tested-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20241025092659.2312118-3-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
The zstd command will preserve the input archive permissions on the
output file. So when we decompress the readonly cached image, the
resulting per-test run private disk image will also be readonly.
We need it to be writable, so make it so.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20241025092659.2312118-2-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Add a functional test for the sx1 board that uses the kernel and
rootfs provided by Guenter Roeck in the linux-test-downloads repo:
https://github.com/groeck/linux-test-downloads/
We have three variants of the test for this board:
* just boot initrd
* boot with filesystem on SD card
* boot from flash
In all cases these images have a userspace that is configured to
immediately reboot the system on successful boot, and the board
itself supports telling QEMU to do the reboot, so we only need to
wait for QEMU to exit (via -no-reboot).
Since there are three subtests, the test as a whole takes about
80s on my local machine. That's about the same as the aarch64_virt
test, so give it the same overall test timeout as that one.
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-id: 20241017163247.711244-3-peter.maydell@linaro.org
Add a functional test for the collie board that uses the kernel and
rootfs provided by Guenter Roeck in the linux-test-downloads repo:
https://github.com/groeck/linux-test-downloads/
This just boots Linux with a userspace that immediately reboots
the board, so we wait for the reboot log line.
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-id: 20241017163247.711244-2-peter.maydell@linaro.org
This is a simple conversion of the tests with some cleanups and
adjustments to match the new test framework. Replace the zephyr image
MD5 hashes with SHA256 hashes while at it.
The SDK tests depend on a ssh class from avocado.utils which is
difficult to replace. To be addressed separately.
Signed-off-by: Cédric Le Goater <clg@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Tested-by: Thomas Huth <thuth@redhat.com>
Move the test into a new file so that it can be run via
qemu-system-sh4 in the functional framework.
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20241011131937.377223-18-thuth@redhat.com>
Move the test into a new file so that it can be run via
qemu-system-ppc in the functional framework.
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20241011131937.377223-16-thuth@redhat.com>
Move the test into a new file so that it can be run via
qemu-system-mips64el in the functional framework.
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20241011131937.377223-15-thuth@redhat.com>
Move the test into a new file so that it can be run via
qemu-system-mips64 in the functional framework.
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20241011131937.377223-14-thuth@redhat.com>
Move the test into a new file so that it can be run via
qemu-system-mipsel in the functional framework.
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20241011131937.377223-13-thuth@redhat.com>
Move the test into a new file so that it can be run via
qemu-system-mips in the functional framework.
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20241011131937.377223-12-thuth@redhat.com>
Move the tests to a new file so that they can be run via
qemu-system-x86_64 in the functional framework.
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20241011131937.377223-11-thuth@redhat.com>
Move the tests to a new file so that they can be run via
qemu-system-i386 in the functional framework.
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20241011131937.377223-10-thuth@redhat.com>
Move the tests to a new file so that they can be run via
qemu-system-riscv64 in the functional framework.
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20241011131937.377223-9-thuth@redhat.com>
Move the tests to a new file so that they can be run via
qemu-system-riscv32 in the functional framework.
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20241011131937.377223-8-thuth@redhat.com>
Move the tests to a new file so that they can be run via
qemu-system-arm in the functional framework.
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20241011131937.377223-7-thuth@redhat.com>
Move the test to a new file so that it can be run via
qemu-system-s390x in the functional framework.
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20241011131937.377223-6-thuth@redhat.com>
Move the test to a new file so that it can be run via
qemu-system-sparc64 in the functional framework.
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20241011131937.377223-5-thuth@redhat.com>
Move the tests to a new file so that they can be run via
qemu-system-ppc64 in the functional framework.
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20241011131937.377223-3-thuth@redhat.com>
Add a base class for the TuxRun tests, based on the code from
tests/avocado/tuxrun_baselines.py (the test have to be put into
separate file in the following commits, depending on the target
architecture that gets tested).
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20241011131937.377223-2-thuth@redhat.com>
These tests occasionally time out when the host system is under heavy
load. Increase the timeout setting to allow for more headroom here.
Message-ID: <20241002140212.350467-1-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Shortly after we switched to the original URLs on qemu-advent-calendar.org,
the server went offline - looks like we are better off using the gitlab
URLs again instead.
Message-ID: <20241007083649.204886-1-thuth@redhat.com>
Reviewed-by: Manos Pitsidianakis <manos.pitsidianakis@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
The _check() function is supposed to check whether the hash of the
downloaded file matches the expected one. Unfortunately, during the
last rework of this function, the check was accidentally turned into
returning the hash value itself instead of a True/False value,
effectively accepting each hash as valid. Let's do a proper check
again now.
Fixes:05e303210d ("tests/functional/qemu_test: Use Python hashlib ...")
Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Michael Tokarev <mjt@tls.msk.ru>
Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>
Move the tests into the already existing test_ppc64_powernv.py
file.
Message-ID: <20240920150319.81723-1-thuth@redhat.com>
Reviewed-by: Cédric Le Goater <clg@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
This is the last test that is using the do_test_advcal_2018()
function, so we can now remove that function from boot_linux_console.py,
too.
Message-ID: <20240919185749.71222-8-thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
The g3beige and mac99 tests use the same asset, so put them together
in a new test_ppc_mac.py file.
Message-ID: <20240919185749.71222-7-thuth@redhat.com>
Acked-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Use the new launch_kernel function to convert this test in a simple way.
Message-ID: <20240919185749.71222-6-thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Use the new launch_kernel function to convert this test in a simple way.
Message-ID: <20240919185749.71222-5-thuth@redhat.com>
Acked-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Use the new launch_kernel function to convert this test in a simple way.
Message-ID: <20240919185749.71222-4-thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Use the new launch_kernel function to convert this test in a simple way.
Message-ID: <20240919185749.71222-3-thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
The task for launching a kernel is quite repetitive: Set the serial
console, set the -kernel and maybe -initrd and -dtb parameters,
launch the VM and then wait for the expected console output. So
it's easier in some tests to provide these steps via a separate
function.
Message-ID: <20240919185749.71222-2-thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Cc: Alex Bennée <alex.bennee@linaro.org>
Fixes: 9f95111474 ("tests/avocado: re-factor igb test to avoid timeouts")
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>
Looks like a copy-n-paste mistake while adding the or1k_sim test
here: The test downloads an asset from the internet, so it should
be in the thorough category, not in the quick one.
Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>
OpenBSD 7.3 we use is EoL. Both 7.4 and 7.5 releases do not work on
anything above Neoverse-N1 due to PAC emulation:
https://marc.info/?l=openbsd-arm&m=171050428327850&w=2
OpenBSD 7.6 is not yet released.
Signed-off-by: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>
Message-id: 20240910-b4-move-to-freebsd-v5-4-0fb66d803c93@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
'Test might timeout' means nothing. Replace it with useful information
that it is emulation of pointer authentication what makes this test run
too long.
Signed-off-by: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>
Message-id: 20240910-b4-move-to-freebsd-v5-3-0fb66d803c93@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
FreeBSD has longer support cycle for stable release (14.x EoL in 2028)
than OpenBSD (7.3 we use is already EoL). Also bugfixes are backported
so we can stay on 14.x for longer.
Signed-off-by: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>
Message-id: 20240910-b4-move-to-freebsd-v5-2-0fb66d803c93@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
We want to run tests using default cpu without having to remember which
Arm core is it.
Change Neoverse-N1 (old default) test to use default cpu (Neoverse-N2 at
the moment).
Signed-off-by: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>
Message-id: 20240910-b4-move-to-freebsd-v5-1-0fb66d803c93@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Commit d2fce37597 added a test that downloads an asset from the
internet, so this test should not be run by default anymore and be
put into the thorough category instead.
Message-ID: <20240913175140.3329083-1-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Some systems (like OpenBSD) do not have the sha256sum or sha512sum programs
installed by default, or use different names for those. Use the Python
hashlib instead so we don't have to rely on the external programs.
Reported-by: Peter Maydell <peter.maydell@linaro.org>
Message-ID: <20240910201742.239559-1-thuth@redhat.com>
Reviewed-by: Brian Cain <bcain@quicinc.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
has_cmd returns a tuple, not a boolean value. This fixes a crash when
e.g. "tesseract" is not available in the test_m68k_nextcube test.
Reported-by: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <20240910075820.51346-1-thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
This test handles both, aarch64 and x86_64, with the same test code
(apart from some initial setup), so don't split this file by target
but add a check for self.arch in the main test function.
Message-ID: <20240903051333.102494-1-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
We've got to do_test_advcal_2018() here now that the test resides
in a separate file. Also switch back to the original URL (since
the site did not vanish as originally expected) and update the
hashsum to use SHA256.
Message-ID: <20240906180549.792832-15-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
We've got to do_test_advcal_2018() here now that the test resides
in a separate file. Also switch back to the original URL (since
the site did not vanish as originally expected) and update the
hashsum to use SHA256.
Message-ID: <20240906180549.792832-14-thuth@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Straight forward conversion, just switch to SHA256 hashsum now.
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240906180549.792832-13-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
