Commit Graph

46 Commits

Author SHA1 Message Date
Thomas Huth
0d2a8acf3f tests/functional: Move the mips64el fuloong2e test into the thorough category
Commit d2fce37597 added a test that downloads an asset from the
internet, so this test should not be run by default anymore and be
put into the thorough category instead.

Message-ID: <20240913175140.3329083-1-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-17 10:52:44 +02:00
Thomas Huth
05e303210d tests/functional/qemu_test: Use Python hashlib instead of external programs
Some systems (like OpenBSD) do not have the sha256sum or sha512sum programs
installed by default, or use different names for those. Use the Python
hashlib instead so we don't have to rely on the external programs.

Reported-by: Peter Maydell <peter.maydell@linaro.org>
Message-ID: <20240910201742.239559-1-thuth@redhat.com>
Reviewed-by: Brian Cain <bcain@quicinc.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-11 09:49:12 +02:00
Thomas Huth
59d100243d tests/functional: Fix bad usage of has_cmd
has_cmd returns a tuple, not a boolean value. This fixes a crash when
e.g. "tesseract" is not available in the test_m68k_nextcube test.

Reported-by: Richard Henderson <richard.henderson@linaro.org>
Message-ID: <20240910075820.51346-1-thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-11 09:49:12 +02:00
Thomas Huth
69e4fbd0ea tests/functional: Convert the multiprocess avocado test into a standalone test
This test handles both, aarch64 and x86_64, with the same test code
(apart from some initial setup), so don't split this file by target
but add a check for self.arch in the main test function.

Message-ID: <20240903051333.102494-1-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-11 09:49:12 +02:00
Thomas Huth
3a07875fd3 tests/functional: Convert the or1k-sim Avocado test
We've got to do_test_advcal_2018() here now that the test resides
in a separate file. Also switch back to the original URL (since
the site did not vanish as originally expected) and update the
hashsum to use SHA256.

Message-ID: <20240906180549.792832-15-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-11 09:49:11 +02:00
Thomas Huth
9f989d6557 tests/functional: Convert the m68k MCF5208EVB Avocado test
We've got to do_test_advcal_2018() here now that the test resides
in a separate file. Also switch back to the original URL (since
the site did not vanish as originally expected) and update the
hashsum to use SHA256.

Message-ID: <20240906180549.792832-14-thuth@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-11 09:49:11 +02:00
Thomas Huth
9a76bc0433 tests/functional: Convert the Alpha Clipper Avocado test
Straight forward conversion, just switch to SHA256 hashsum now.

Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240906180549.792832-13-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-11 09:49:11 +02:00
Philippe Mathieu-Daudé
547738be48 tests/functional: Convert Aarch64 Raspi4 avocado tests
Straight forward conversion. Update the SHA1 hashes to SHA256
hashes since SHA1 should not be used anymore nowadays.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240826221058.75126-5-philmd@linaro.org>
[thuth: Use LinuxKernelTest class]
Message-ID: <20240906180549.792832-12-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-11 09:49:11 +02:00
Philippe Mathieu-Daudé
471f7ff715 tests/functional: Convert Aarch64 Raspi3 avocado tests
Straight forward conversion. Update the SHA1 hashes to SHA256
hashes since SHA1 should not be used anymore nowadays.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240826221058.75126-4-philmd@linaro.org>
[thuth: Use the LinuxKernelTest class]
Message-ID: <20240906180549.792832-11-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-11 09:49:11 +02:00
Philippe Mathieu-Daudé
165ab27447 tests/functional: Convert ARM Raspi2 avocado tests
Straight forward conversion. Update the SHA1 hashes to SHA256
hashes since SHA1 should not be used anymore nowadays.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240826221058.75126-3-philmd@linaro.org>
[thuth: Use LinuxKernelTest class]
Message-ID: <20240906180549.792832-10-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-11 09:49:11 +02:00
Philippe Mathieu-Daudé
79cb4a14cb tests/functional: Convert mips32eb 4Kc Malta avocado tests
Straight forward conversion. Update the SHA1 hashes to
SHA256 hashes since SHA1 should not be used anymore nowadays.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240827094905.80648-8-philmd@linaro.org>
[thuth: Use LinuxKernelTest class]
Message-ID: <20240906180549.792832-9-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-11 09:49:11 +02:00
Philippe Mathieu-Daudé
0e85442408 tests/functional: Convert nanomips Malta avocado tests
Straight forward conversion. Update the SHA1 hashes to
SHA256 hashes since SHA1 should not be used anymore nowadays.

  $ QEMU_TEST_ALLOW_UNTRUSTED_CODE=1 \
        make check-functional-mipsel
  ...
  ▶ 4/4 test_mipsel_malta.MaltaMachineConsole.test_mips_malta32el_nanomips_16k_up   OK
  ▶ 4/4 test_mipsel_malta.MaltaMachineConsole.test_mips_malta32el_nanomips_4k       OK
  ▶ 4/4 test_mipsel_malta.MaltaMachineConsole.test_mips_malta32el_nanomips_64k_dbg  OK
  ▶ 4/4 test_mipsel_malta.MaltaMachineYAMON.test_mipsel_malta_yamon                 OK
  4/4 qemu:func-thorough+func-mipsel-thorough+thorough / func-mipsel-mipsel_malta   OK   9.95s   4 subtests passed

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240827094905.80648-7-philmd@linaro.org>
[thuth: Use LinuxKernelTest]
Message-ID: <20240906180549.792832-8-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-11 09:49:11 +02:00
Philippe Mathieu-Daudé
a4b60bec10 tests/functional: Convert mips32el Malta YAMON avocado test
Straight forward conversion using the Python standard zipfile
module instead of avocado.utils package. Update the SHA1 hashes
to SHA256 hashes since SHA1 should not be used anymore nowadays.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240827094905.80648-6-philmd@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20240906180549.792832-7-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-11 09:49:11 +02:00
Philippe Mathieu-Daudé
0e165dfc52 tests/functional: Convert mips64el 5KEc Malta avocado tests
Straight forward conversion. Update the SHA1 hashes to
SHA256 hashes since SHA1 should not be used anymore nowadays.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240827094905.80648-5-philmd@linaro.org>
Message-ID: <20240906180549.792832-6-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-11 09:49:11 +02:00
Philippe Mathieu-Daudé
fcf2658b34 tests/functional: Convert mips64el I6400 Malta avocado tests
Straight forward conversion. Update the SHA1 hashes to SHA256
hashes since SHA1 should not be used anymore nowadays.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240827094905.80648-4-philmd@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20240906180549.792832-5-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-11 09:49:11 +02:00
Philippe Mathieu-Daudé
d2fce37597 tests/functional: Convert mips64el Fuloong2e avocado test (2/2)
Straight forward conversion. Update the SHA1 hashes to
SHA256 hashes since SHA1 should not be used anymore nowadays.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240827094905.80648-3-philmd@linaro.org>
[thuth: Use LinuxKernelTest class for this test]
Message-ID: <20240906180549.792832-4-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-11 09:49:11 +02:00
Thomas Huth
c7f3663c60 tests/functional: Convert the m68k Q800 Avocado test into a functional test
Just had to update the asset checksum to use SHA256 instead of SHA1,
but apart from that it is a pretty much straightforward conversion.

Message-ID: <20240906180549.792832-3-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-11 09:49:11 +02:00
Thomas Huth
8720471efd tests/functional: Add the LinuxKernelTest for testing the Linux boot process
Copy the LinuxKernelTest from tests/acceptance/boot_linux_console.py
to be able to convert the related tests to the functional test framework
in the following patches.

Message-ID: <20240906180549.792832-2-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-11 09:49:11 +02:00
Philippe Mathieu-Daudé
34917ead72 tests/functional: Convert ARM bFLT linux-user avocado test
Straight forward conversion. Update the SHA1 hashes to
SHA256 hashes since SHA1 should not be used anymore nowadays.
Expose cpio_extract() in qemu_test.utils for possible reuse.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240822104238.75045-3-philmd@linaro.org>
[thuth: Add test to meson.build]
Message-ID: <20240830133841.142644-39-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-04 12:28:00 +02:00
Philippe Mathieu-Daudé
99465d3fe4 tests/functional: Add QemuUserTest class
Per commit 5334df4822 ("tests/avocado: Introduce
QemuUserTest base class"):

  Similarly to the 'System' Test base class with methods
  for testing system emulation, the QemuUserTest class
  contains methods useful to test user-mode emulation.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240822104238.75045-2-philmd@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20240830133841.142644-38-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-04 12:28:00 +02:00
Philippe Mathieu-Daudé
bce9e48415 tests/functional: Convert mips64el Fuloong2e avocado test (1/2)
Straight forward conversion. Update the SHA1 hashes to
SHA256 hashes since SHA1 should not be used anymore nowadays.

Since the asset is expected locally and the test is guarded
with RESCUE_YL_PATH, keep it under the 'quick' category.

  $ RESCUE_YL_PATH=/path/to/rescue-yl QEMU_TEST_ALLOW_UNTRUSTED_CODE=1 \
    make check-functional-mips64el
  1/4 qemu:func-quick+func-mips64el / func-mips64el-empty_cpu_model      OK   0.12s   1 subtests passed
  2/4 qemu:func-quick+func-mips64el / func-mips64el-version              OK   0.13s   1 subtests passed
  3/4 qemu:func-quick+func-mips64el / func-mips64el-info_usernet         OK   0.15s   1 subtests passed
  4/4 qemu:func-quick+func-mips64el / func-mips64el-mips64el_fuloong2e   OK   0.19s   1 subtests passed

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240827094905.80648-2-philmd@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20240830133841.142644-37-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-04 12:28:00 +02:00
Philippe Mathieu-Daudé
1255f5e496 tests/functional: Convert Aarch64 Virt machine avocado tests
Straight forward conversion. Update the SHA1 hash to
SHA256 since SHA1 should not be used anymore nowadays.

Provide get_qemu_img() helper in qemu_test.

  $ make check-functional-aarch64 V=1
  ▶ 1/4 test_aarch64_virt.Aarch64VirtMachine.test_aarch64_virt_gicv2                 OK
  ▶ 1/4 test_aarch64_virt.Aarch64VirtMachine.test_aarch64_virt_gicv3                 OK
  ▶ 1/4 test_aarch64_virt.Aarch64VirtMachine.test_alpine_virt_tcg_gic_max            OK
  1/4 qemu:func-thorough+func-aarch64-thorough+thorough / func-aarch64-aarch64_virt  OK   30.44s  3 subtests passed
  ...

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240826103634.52384-1-philmd@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20240830133841.142644-36-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-04 12:28:00 +02:00
Philippe Mathieu-Daudé
8f16cd80fc tests/functional: Convert Aarch64 SBSA-Ref avocado tests
Straight forward conversion. Since SBSA_FLASH files are not
tarballs, use lzma_uncompress() method.

Avocado used to set a timeout of 11 tests * 180s = 1980s.
Hopefully 600s should be sufficient.

Running on macOS Sonoma / Apple silicon M1:

  $ QEMU_TEST_TIMEOUT_EXPECTED=1 make check-functional-aarch64 V=1
  ▶ 1/5 test_aarch64_sbsaref.Aarch64SbsarefMachine.test_sbsaref_alpine_linux_max              OK
  ▶ 1/5 test_aarch64_sbsaref.Aarch64SbsarefMachine.test_sbsaref_alpine_linux_max_pauth_impdef OK
  ▶ 1/5 test_aarch64_sbsaref.Aarch64SbsarefMachine.test_sbsaref_alpine_linux_max_pauth_off    OK
  ▶ 1/5 test_aarch64_sbsaref.Aarch64SbsarefMachine.test_sbsaref_alpine_linux_neoverse_n1      OK
  ▶ 1/5 test_aarch64_sbsaref.Aarch64SbsarefMachine.test_sbsaref_edk2_firmware                 OK
  ▶ 1/5 test_aarch64_sbsaref.Aarch64SbsarefMachine.test_sbsaref_openbsd73_cortex_a57          OK
  ▶ 1/5 test_aarch64_sbsaref.Aarch64SbsarefMachine.test_sbsaref_openbsd73_max                 OK
  ▶ 1/5 test_aarch64_sbsaref.Aarch64SbsarefMachine.test_sbsaref_openbsd73_max_pauth_impdef    OK
  ▶ 1/5 test_aarch64_sbsaref.Aarch64SbsarefMachine.test_sbsaref_openbsd73_max_pauth_off       OK
  ▶ 1/5 test_aarch64_sbsaref.Aarch64SbsarefMachine.test_sbsaref_openbsd73_neoverse_n1         OK
  1/5 qemu:func-thorough+func-aarch64-thorough+thorough / func-aarch64-aarch64_sbsaref        OK   241.79s  11 subtests passed
  ...

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240823131614.10269-1-philmd@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Marcin Juszkiewicz <marcin.juszkiewicz@linaro.org>
Message-ID: <20240830133841.142644-35-thuth@redhat.com>
[thuth: remove remaining Avocado tags]
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-04 12:27:52 +02:00
Philippe Mathieu-Daudé
ef83aea0a3 tests/functional: Convert ARM Integrator/CP avocado tests
Straight forward conversion. Update the SHA1 hashes to SHA256
hashes since SHA1 should not be used anymore nowadays.

  $ QEMU_TEST_ALLOW_UNTRUSTED_CODE=1 make check-functional-arm
  ...
  6/6 qemu:func-thorough+func-arm-thorough+thorough / func-arm-arm_integratorcp   OK   3.90s   2 subtests passed

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240822110238.82312-1-philmd@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20240830133841.142644-34-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-04 11:14:51 +02:00
Thomas Huth
8dcac1cf71 tests/functional: Convert the linux_initrd avocado test into a standalone test
While we're at it, update the SHA1 hashes to SHA256 hashes since
SHA1 should not be used anymore nowadays.

Message-ID: <20240830133841.142644-32-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-04 11:14:33 +02:00
Thomas Huth
d5674412ba tests/functional: Convert the rx_gdbsim avocado test into a standalone test
Provide a "gzip_uncompress" function based on the standard "gzip" module
to avoid the usage of avocado.utils here.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240830133841.142644-31-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-04 11:14:33 +02:00
Thomas Huth
05caa06242 tests/functional: Convert the acpi-bits test into a standalone test
Mostly a straight-forward conversion. Looks like we can simply drop
the avocado datadrainer stuff when not using the avocado framework
anymore.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240830133841.142644-30-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-04 11:14:33 +02:00
Thomas Huth
576fffbc8e tests/functional: Convert the m68k nextcube test with tesseract
The code that handles running of tesseract needs to be tweaked a little
bit to be able to run without the functions from avocado.utils, and
while we're at it, drop some legacy stuff that was still there due to
Tesseract 3 support that we already dropped a while ago.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240830133841.142644-29-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-04 11:14:33 +02:00
Thomas Huth
88c907199a tests/functional: Convert the ppc_hv avocado test into a standalone test
A straight forward conversion, we just also have to remove the decorator
@skipUnless(os.getenv('SPEED')) since all non-trivial functional tests
are running in SPEED=thorough mode now. Also make sure that the extracted
assets are writable, so that the test does not fail if it gets re-run
and there are stale read-only files already around.

Message-ID: <20240830133841.142644-28-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-04 11:14:33 +02:00
Thomas Huth
cef1becb9f tests/functional: Convert the ppc_amiga avocado test into a standalone test
Use the Python standard zipfile module instead of avocado.utils for
extracting the ZIP file that we download here, and use the standard
subprocess module for running the "tail" command.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240830133841.142644-27-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-04 11:14:33 +02:00
Thomas Huth
407a688315 tests/functional: Convert most ppc avocado tests into standalone tests
Nothing thrilling in here, just straight forward conversions.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240830133841.142644-26-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-04 11:14:33 +02:00
Thomas Huth
239d08aa96 tests/functional: Convert the virtio_gpu avocado test into a standalone test
Nothing thrilling in here, it's just a straight forward conversion.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240830133841.142644-25-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-04 11:14:33 +02:00
Thomas Huth
624fb343df tests/functional: Convert the microblaze avocado tests into standalone tests
The machine_microblaze.py file contained two tests, one for each
endianness. Since we only support one QEMU target binary per file
in the new functional test environment, we have to split this file
up into two files now.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240830133841.142644-23-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-04 11:14:33 +02:00
Thomas Huth
be849ef715 tests/functional: Convert the x86_cpu_model_versions test
Nothing thrilling in here, it's just a straight forward conversion.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240830133841.142644-22-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-04 11:14:33 +02:00
Thomas Huth
e2e9fd256e tests/functional: Convert the s390x avocado tests into standalone tests
These tests use archive.lzma_uncompress() from the Avocado utils,
so provide a small helper function for this, based on the
standard lzma module from Python instead.

And while we're at it, replace the MD5 hashes in the topology test
with proper SHA256 hashes, since MD5 should not be used anymore
nowadays.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240830133841.142644-21-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-04 11:14:33 +02:00
Thomas Huth
e3fc99b164 tests/functional: Convert some avocado tests that needed avocado.utils.archive
Instead of using the "archive" module from avocado.utils, switch
these tests to use the new wrapper function that is based on the
"tarfile" module instead.

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240830133841.142644-20-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-04 11:14:29 +02:00
Thomas Huth
850a1951b7 tests/functional: Add a function for extracting files from an archive
Some Avocado-based tests use the "archive" module from avocado.utils
to extract files from an archive. To be able to use these tests
without Avocado, we have to provide our own function for extracting
files. Fortunately, there is already the tarfile module that will
provide us with this functionality, so let's just add a nice wrapper
function around that.

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20240830133841.142644-19-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-04 11:13:48 +02:00
Thomas Huth
4c0a2df81c tests/functional: Convert some tests that download files via fetch_asset()
Now that we've got the Asset class with pre-caching, we can convert
some Avocado tests that use fetch_asset() for downloading their
required files.

Message-ID: <20240830133841.142644-18-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-04 11:11:31 +02:00
Thomas Huth
34b17c0a65 tests/functional: Allow asset downloading with concurrent threads
When running "make -j$(nproc) check-functional", tests that use the
same asset might be running in parallel. Improve the downloading to
detect this situation and wait for the other thread to finish the
download.

Message-ID: <20240830133841.142644-17-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-04 10:52:29 +02:00
Daniel P. Berrangé
f57213f85b tests/functional: enable pre-emptive caching of assets
Many tests need to access assets stored on remote sites. We don't want
to download these during test execution when run by meson, since this
risks hitting test timeouts when data transfers are slow.

Add support for pre-emptive caching of assets by setting the env var
QEMU_TEST_PRECACHE to point to a timestamp file. When this is set,
instead of running the test, the assets will be downloaded and saved
to the cache, then the timestamp file created.

A meson custom target is created as a dependency of each test suite
to trigger the pre-emptive caching logic before the test runs.

When run in caching mode, it will locate assets by looking for class
level variables with a name prefix "ASSET_", and type "Asset".

At the ninja level

   ninja test --suite functional

will speculatively download any assets that are not already cached,
so it is advisable to set a timeout multiplier.

   QEMU_TEST_NO_DOWNLOAD=1 ninja test --suite functional

will fail the test if a required asset is not already cached

   ninja precache-functional

will download and cache all assets required by the functional
tests

At the make level, precaching is always done by

   make check-functional

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Tested-by: Richard Henderson <richard.henderson@linaro.org>
[thuth: Remove the duplicated "path = os.path.basename(...)" line]
Message-ID: <20240830133841.142644-16-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-04 10:52:29 +02:00
Daniel P. Berrangé
9903217a4e tests/functional: add a module for handling asset download & caching
The 'Asset' class is a simple module that declares a downloadable
asset that can be cached locally. Downloads are stored in the user's
home dir at ~/.cache/qemu/download, using a sha256 sum of the URL.

[thuth: Drop sha1 support, use hash on file content for naming instead of URL,
        add the possibility to specify the cache dir via environment variable]

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240830133841.142644-15-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-04 10:52:29 +02:00
Thomas Huth
eeba3d7365 tests/functional: Convert avocado tests that just need a small adjustment
These simple tests can be converted to stand-alone tests quite easily,
e.g. by just setting the machine to 'none' now manually or by adding
"-cpu" command line parameters, since we don't support the corresponding
avocado tags in the new python test framework.

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240830133841.142644-14-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-04 10:52:29 +02:00
Thomas Huth
cce85725f1 tests/functional: Convert simple avocado tests into standalone python tests
These test are rather simple and don't need any modifications apart
from adjusting the "from avocado_qemu" line. To ease debugging, make
the files executable and add a shebang line and Python '__main__'
handling, too, so that these tests can now be run by executing them
directly.

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240830133841.142644-13-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-04 10:52:29 +02:00
Thomas Huth
1497377857 tests/functional: Prepare the meson build system for the functional tests
Provide a meson.build file for the upcoming python-based functional
tests, and add some wrapper glue targets to the tests/Makefile.include
file. We are going to use two "speed" modes for the functional tests:
The "quick" tests can be run at any time (i.e. also during "make check"),
while the "thorough" tests should only be run when running a
"make check-functional" test run (since these tests might download
additional assets from the internet).

The changes to the meson.build files are partly based on an earlier
patch by Ani Sinha.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240830133841.142644-12-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-04 10:52:29 +02:00
Thomas Huth
84e4a27fed tests/functional: Set up logging
Create log files for each test separately, one file that contains
the basic logging and one that contains the console output.

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20240830133841.142644-10-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-04 10:52:29 +02:00
Thomas Huth
fa32a63432 tests/functional: Add base classes for the upcoming pytest-based tests
The files are mostly a copy of the tests/avocado/avocado_qemu/__init__.py
file with some adjustments to get rid of the Avocado dependencies (i.e.
we also have to drop the LinuxSSHMixIn and LinuxTest for now).

The emulator binary and build directory are now passed via
environment variables that will be set via meson.build later.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20240830133841.142644-9-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-04 10:52:29 +02:00