Move the tests to a new file so that they can be run via
qemu-system-riscv64 in the functional framework.
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20241011131937.377223-9-thuth@redhat.com>
Move the tests to a new file so that they can be run via
qemu-system-riscv32 in the functional framework.
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20241011131937.377223-8-thuth@redhat.com>
Move the tests to a new file so that they can be run via
qemu-system-arm in the functional framework.
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20241011131937.377223-7-thuth@redhat.com>
Move the test to a new file so that it can be run via
qemu-system-s390x in the functional framework.
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20241011131937.377223-6-thuth@redhat.com>
Move the test to a new file so that it can be run via
qemu-system-sparc64 in the functional framework.
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20241011131937.377223-5-thuth@redhat.com>
Move the tests to a new file so that they can be run via
qemu-system-ppc64 in the functional framework.
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20241011131937.377223-3-thuth@redhat.com>
Add a base class for the TuxRun tests, based on the code from
tests/avocado/tuxrun_baselines.py (the test have to be put into
separate file in the following commits, depending on the target
architecture that gets tested).
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20241011131937.377223-2-thuth@redhat.com>
These tests occasionally time out when the host system is under heavy
load. Increase the timeout setting to allow for more headroom here.
Message-ID: <20241002140212.350467-1-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Shortly after we switched to the original URLs on qemu-advent-calendar.org,
the server went offline - looks like we are better off using the gitlab
URLs again instead.
Message-ID: <20241007083649.204886-1-thuth@redhat.com>
Reviewed-by: Manos Pitsidianakis <manos.pitsidianakis@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
The _check() function is supposed to check whether the hash of the
downloaded file matches the expected one. Unfortunately, during the
last rework of this function, the check was accidentally turned into
returning the hash value itself instead of a True/False value,
effectively accepting each hash as valid. Let's do a proper check
again now.
Fixes:05e303210d ("tests/functional/qemu_test: Use Python hashlib ...")
Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Michael Tokarev <mjt@tls.msk.ru>
Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>
Move the tests into the already existing test_ppc64_powernv.py
file.
Message-ID: <20240920150319.81723-1-thuth@redhat.com>
Reviewed-by: Cédric Le Goater <clg@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
This is the last test that is using the do_test_advcal_2018()
function, so we can now remove that function from boot_linux_console.py,
too.
Message-ID: <20240919185749.71222-8-thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
The g3beige and mac99 tests use the same asset, so put them together
in a new test_ppc_mac.py file.
Message-ID: <20240919185749.71222-7-thuth@redhat.com>
Acked-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Use the new launch_kernel function to convert this test in a simple way.
Message-ID: <20240919185749.71222-6-thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Use the new launch_kernel function to convert this test in a simple way.
Message-ID: <20240919185749.71222-5-thuth@redhat.com>
Acked-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Use the new launch_kernel function to convert this test in a simple way.
Message-ID: <20240919185749.71222-4-thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Use the new launch_kernel function to convert this test in a simple way.
Message-ID: <20240919185749.71222-3-thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
The task for launching a kernel is quite repetitive: Set the serial
console, set the -kernel and maybe -initrd and -dtb parameters,
launch the VM and then wait for the expected console output. So
it's easier in some tests to provide these steps via a separate
function.
Message-ID: <20240919185749.71222-2-thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Cc: Alex Bennée <alex.bennee@linaro.org>
Fixes: 9f95111474 ("tests/avocado: re-factor igb test to avoid timeouts")
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>
Looks like a copy-n-paste mistake while adding the or1k_sim test
here: The test downloads an asset from the internet, so it should
be in the thorough category, not in the quick one.
Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>
OpenBSD 7.3 we use is EoL. Both 7.4 and 7.5 releases do not work on
anything above Neoverse-N1 due to PAC emulation:
https://marc.info/?l=openbsd-arm&m=171050428327850&w=2
OpenBSD 7.6 is not yet released.
Signed-off-by: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>
Message-id: 20240910-b4-move-to-freebsd-v5-4-0fb66d803c93@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
'Test might timeout' means nothing. Replace it with useful information
that it is emulation of pointer authentication what makes this test run
too long.
Signed-off-by: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>
Message-id: 20240910-b4-move-to-freebsd-v5-3-0fb66d803c93@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
FreeBSD has longer support cycle for stable release (14.x EoL in 2028)
than OpenBSD (7.3 we use is already EoL). Also bugfixes are backported
so we can stay on 14.x for longer.
Signed-off-by: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>
Message-id: 20240910-b4-move-to-freebsd-v5-2-0fb66d803c93@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
We want to run tests using default cpu without having to remember which
Arm core is it.
Change Neoverse-N1 (old default) test to use default cpu (Neoverse-N2 at
the moment).
Signed-off-by: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>
Message-id: 20240910-b4-move-to-freebsd-v5-1-0fb66d803c93@linaro.org
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Commit d2fce37597 added a test that downloads an asset from the
internet, so this test should not be run by default anymore and be
put into the thorough category instead.
Message-ID: <20240913175140.3329083-1-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Some systems (like OpenBSD) do not have the sha256sum or sha512sum programs
installed by default, or use different names for those. Use the Python
hashlib instead so we don't have to rely on the external programs.
Reported-by: Peter Maydell <peter.maydell@linaro.org>
Message-ID: <20240910201742.239559-1-thuth@redhat.com>
Reviewed-by: Brian Cain <bcain@quicinc.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
has_cmd returns a tuple, not a boolean value. This fixes a crash when
e.g. "tesseract" is not available in the test_m68k_nextcube test.
Reported-by: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <20240910075820.51346-1-thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
This test handles both, aarch64 and x86_64, with the same test code
(apart from some initial setup), so don't split this file by target
but add a check for self.arch in the main test function.
Message-ID: <20240903051333.102494-1-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
We've got to do_test_advcal_2018() here now that the test resides
in a separate file. Also switch back to the original URL (since
the site did not vanish as originally expected) and update the
hashsum to use SHA256.
Message-ID: <20240906180549.792832-15-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
We've got to do_test_advcal_2018() here now that the test resides
in a separate file. Also switch back to the original URL (since
the site did not vanish as originally expected) and update the
hashsum to use SHA256.
Message-ID: <20240906180549.792832-14-thuth@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Straight forward conversion, just switch to SHA256 hashsum now.
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240906180549.792832-13-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Straight forward conversion. Update the SHA1 hashes to SHA256
hashes since SHA1 should not be used anymore nowadays.
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240826221058.75126-5-philmd@linaro.org>
[thuth: Use LinuxKernelTest class]
Message-ID: <20240906180549.792832-12-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Straight forward conversion. Update the SHA1 hashes to SHA256
hashes since SHA1 should not be used anymore nowadays.
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240826221058.75126-4-philmd@linaro.org>
[thuth: Use the LinuxKernelTest class]
Message-ID: <20240906180549.792832-11-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Straight forward conversion. Update the SHA1 hashes to SHA256
hashes since SHA1 should not be used anymore nowadays.
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240826221058.75126-3-philmd@linaro.org>
[thuth: Use LinuxKernelTest class]
Message-ID: <20240906180549.792832-10-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Straight forward conversion. Update the SHA1 hashes to
SHA256 hashes since SHA1 should not be used anymore nowadays.
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240827094905.80648-8-philmd@linaro.org>
[thuth: Use LinuxKernelTest class]
Message-ID: <20240906180549.792832-9-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Straight forward conversion. Update the SHA1 hashes to
SHA256 hashes since SHA1 should not be used anymore nowadays.
$ QEMU_TEST_ALLOW_UNTRUSTED_CODE=1 \
make check-functional-mipsel
...
▶ 4/4 test_mipsel_malta.MaltaMachineConsole.test_mips_malta32el_nanomips_16k_up OK
▶ 4/4 test_mipsel_malta.MaltaMachineConsole.test_mips_malta32el_nanomips_4k OK
▶ 4/4 test_mipsel_malta.MaltaMachineConsole.test_mips_malta32el_nanomips_64k_dbg OK
▶ 4/4 test_mipsel_malta.MaltaMachineYAMON.test_mipsel_malta_yamon OK
4/4 qemu:func-thorough+func-mipsel-thorough+thorough / func-mipsel-mipsel_malta OK 9.95s 4 subtests passed
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240827094905.80648-7-philmd@linaro.org>
[thuth: Use LinuxKernelTest]
Message-ID: <20240906180549.792832-8-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Straight forward conversion using the Python standard zipfile
module instead of avocado.utils package. Update the SHA1 hashes
to SHA256 hashes since SHA1 should not be used anymore nowadays.
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240827094905.80648-6-philmd@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20240906180549.792832-7-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Straight forward conversion. Update the SHA1 hashes to
SHA256 hashes since SHA1 should not be used anymore nowadays.
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240827094905.80648-5-philmd@linaro.org>
Message-ID: <20240906180549.792832-6-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Straight forward conversion. Update the SHA1 hashes to SHA256
hashes since SHA1 should not be used anymore nowadays.
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240827094905.80648-4-philmd@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20240906180549.792832-5-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Straight forward conversion. Update the SHA1 hashes to
SHA256 hashes since SHA1 should not be used anymore nowadays.
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240827094905.80648-3-philmd@linaro.org>
[thuth: Use LinuxKernelTest class for this test]
Message-ID: <20240906180549.792832-4-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Just had to update the asset checksum to use SHA256 instead of SHA1,
but apart from that it is a pretty much straightforward conversion.
Message-ID: <20240906180549.792832-3-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Copy the LinuxKernelTest from tests/acceptance/boot_linux_console.py
to be able to convert the related tests to the functional test framework
in the following patches.
Message-ID: <20240906180549.792832-2-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Straight forward conversion. Update the SHA1 hashes to
SHA256 hashes since SHA1 should not be used anymore nowadays.
Expose cpio_extract() in qemu_test.utils for possible reuse.
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240822104238.75045-3-philmd@linaro.org>
[thuth: Add test to meson.build]
Message-ID: <20240830133841.142644-39-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Per commit 5334df4822 ("tests/avocado: Introduce
QemuUserTest base class"):
Similarly to the 'System' Test base class with methods
for testing system emulation, the QemuUserTest class
contains methods useful to test user-mode emulation.
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240822104238.75045-2-philmd@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20240830133841.142644-38-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Straight forward conversion. Update the SHA1 hashes to
SHA256 hashes since SHA1 should not be used anymore nowadays.
Since the asset is expected locally and the test is guarded
with RESCUE_YL_PATH, keep it under the 'quick' category.
$ RESCUE_YL_PATH=/path/to/rescue-yl QEMU_TEST_ALLOW_UNTRUSTED_CODE=1 \
make check-functional-mips64el
1/4 qemu:func-quick+func-mips64el / func-mips64el-empty_cpu_model OK 0.12s 1 subtests passed
2/4 qemu:func-quick+func-mips64el / func-mips64el-version OK 0.13s 1 subtests passed
3/4 qemu:func-quick+func-mips64el / func-mips64el-info_usernet OK 0.15s 1 subtests passed
4/4 qemu:func-quick+func-mips64el / func-mips64el-mips64el_fuloong2e OK 0.19s 1 subtests passed
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240827094905.80648-2-philmd@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20240830133841.142644-37-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Straight forward conversion. Update the SHA1 hash to
SHA256 since SHA1 should not be used anymore nowadays.
Provide get_qemu_img() helper in qemu_test.
$ make check-functional-aarch64 V=1
▶ 1/4 test_aarch64_virt.Aarch64VirtMachine.test_aarch64_virt_gicv2 OK
▶ 1/4 test_aarch64_virt.Aarch64VirtMachine.test_aarch64_virt_gicv3 OK
▶ 1/4 test_aarch64_virt.Aarch64VirtMachine.test_alpine_virt_tcg_gic_max OK
1/4 qemu:func-thorough+func-aarch64-thorough+thorough / func-aarch64-aarch64_virt OK 30.44s 3 subtests passed
...
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240826103634.52384-1-philmd@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20240830133841.142644-36-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Straight forward conversion. Since SBSA_FLASH files are not
tarballs, use lzma_uncompress() method.
Avocado used to set a timeout of 11 tests * 180s = 1980s.
Hopefully 600s should be sufficient.
Running on macOS Sonoma / Apple silicon M1:
$ QEMU_TEST_TIMEOUT_EXPECTED=1 make check-functional-aarch64 V=1
▶ 1/5 test_aarch64_sbsaref.Aarch64SbsarefMachine.test_sbsaref_alpine_linux_max OK
▶ 1/5 test_aarch64_sbsaref.Aarch64SbsarefMachine.test_sbsaref_alpine_linux_max_pauth_impdef OK
▶ 1/5 test_aarch64_sbsaref.Aarch64SbsarefMachine.test_sbsaref_alpine_linux_max_pauth_off OK
▶ 1/5 test_aarch64_sbsaref.Aarch64SbsarefMachine.test_sbsaref_alpine_linux_neoverse_n1 OK
▶ 1/5 test_aarch64_sbsaref.Aarch64SbsarefMachine.test_sbsaref_edk2_firmware OK
▶ 1/5 test_aarch64_sbsaref.Aarch64SbsarefMachine.test_sbsaref_openbsd73_cortex_a57 OK
▶ 1/5 test_aarch64_sbsaref.Aarch64SbsarefMachine.test_sbsaref_openbsd73_max OK
▶ 1/5 test_aarch64_sbsaref.Aarch64SbsarefMachine.test_sbsaref_openbsd73_max_pauth_impdef OK
▶ 1/5 test_aarch64_sbsaref.Aarch64SbsarefMachine.test_sbsaref_openbsd73_max_pauth_off OK
▶ 1/5 test_aarch64_sbsaref.Aarch64SbsarefMachine.test_sbsaref_openbsd73_neoverse_n1 OK
1/5 qemu:func-thorough+func-aarch64-thorough+thorough / func-aarch64-aarch64_sbsaref OK 241.79s 11 subtests passed
...
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240823131614.10269-1-philmd@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>
Message-ID: <20240830133841.142644-35-thuth@redhat.com>
[thuth: remove remaining Avocado tags]
Signed-off-by: Thomas Huth <thuth@redhat.com>
Straight forward conversion. Update the SHA1 hashes to SHA256
hashes since SHA1 should not be used anymore nowadays.
$ QEMU_TEST_ALLOW_UNTRUSTED_CODE=1 make check-functional-arm
...
6/6 qemu:func-thorough+func-arm-thorough+thorough / func-arm-arm_integratorcp OK 3.90s 2 subtests passed
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240822110238.82312-1-philmd@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20240830133841.142644-34-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
While we're at it, update the SHA1 hashes to SHA256 hashes since
SHA1 should not be used anymore nowadays.
Message-ID: <20240830133841.142644-32-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
