Commit Graph

12 Commits

Author SHA1 Message Date
Cleber Rosa
efe30d5011 Acceptance Tests: introduce method for requiring an accelerator
Some tests explicitly require a QEMU accelerator to be available.
Given that this depends on some runtime aspects not known before
the test is started, such as the currently set QEMU binary, it's
left to be checked also at runtime.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20210203172357.1422425-17-crosa@redhat.com>
Reviewed-by: Beraldo Leal <bleal@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Signed-off-by: Cleber Rosa <crosa@redhat.com>
2021-02-15 22:23:40 -05:00
Cleber Rosa
6dd7457916 Acceptance Tests: introduce LinuxTest base class
This is basically the infrastructure around "boot_linux.py" tests, but
now made into a base class for general use.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20210203172357.1422425-15-crosa@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Signed-off-by: Cleber Rosa <crosa@redhat.com>
2021-02-15 22:17:25 -05:00
Cleber Rosa
b5a86c482d tests/acceptance/boot_linux: rename misleading cloudinit method
There's no downloading happening on that method, so let's call it
"prepare" instead.  While at it, and because of it, the current
"prepare_boot" and "prepare_cloudinit" are also renamed.

The reasoning here is that "prepare_" methods will just work on the
images, while "set_up_" will make them effective to the VM that will
be launched.  Inspiration comes from the "virtiofs_submounts.py"
tests, which this expects to converge more into.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Beraldo Leal <bleal@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20210203172357.1422425-3-crosa@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2021-02-08 17:22:36 +01:00
Cleber Rosa
834736c9d1 tests/acceptance/boot_linux: fix typo on cloudinit error message
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Beraldo Leal <bleal@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210203172357.1422425-2-crosa@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2021-02-08 17:22:36 +01:00
Max Reitz
0a7798037a tests/acceptance/boot_linux: Accept SSH pubkey
Let download_cloudinit() take an optional pubkey, which subclasses of
BootLinux can pass through setUp().

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20201102161859.156603-7-mreitz@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2020-11-02 19:22:51 +00:00
Alex Williamson
33dc9914ea Revert series: virtiofsd: Announce submounts to the guest
This reverts the following commits due to their basis on a bogus
linux kernel header update:

c93a656f7b ("tests/acceptance: Add virtiofs_submounts.py")
45ced7ca2f ("tests/acceptance/boot_linux: Accept SSH pubkey")
08dce386e7 ("virtiofsd: Announce sub-mount points")
eba8b096c1 ("virtiofsd: Store every lo_inode's parent_dev")
ede24b6be7 ("virtiofsd: Add fuse_reply_attr_with_flags()")
e2577435d3 ("virtiofsd: Add attr_flags to fuse_entry_param")
2f10415abf ("virtiofsd: Announce FUSE_ATTR_FLAGS")
97d741cc96 ("linux/fuse.h: Pull in from Linux")

Cc: Max Reitz <mreitz@redhat.com>
Cc: Stefan Hajnoczi <stefanha@redhat.com>
Cc: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 160385090886.20017.13382256442750027666.stgit@gimli.home
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-10-28 13:17:32 +00:00
Peter Maydell
cfc1105649 Acceptance testing patches
- More ARM tests
 - Documentation update
 
 CI jobs results:
 . https://cirrus-ci.com/build/5754555684093952
 . https://gitlab.com/philmd/qemu/-/pipelines/207807648
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEE+qvnXhKRciHc/Wuy4+MsLN6twN4FAl+XHkwACgkQ4+MsLN6t
 wN4WJBAAxfsfZV3DjoKP/mAjVKJCHkA82MzzoN+YH5WQs4QG1Hytod7OyWC101xj
 wPuL85NR+bo+he12Uylmbm35TCyKxH3kOk2GAqiHvS16mxNZtAssehVt5a1hoUlw
 N9R37dtccGbURX3WzEgJralaZaST96ZbP+y76A95ygriXLnZRisVJG0t/FKBb22m
 8B8+lgHpb11hFop/COY93Aa/BPx9S7cPpBQX4p8e/gXhdJTtqzbhWYSkLO60FqJh
 l1D8KAB7dUtVPoW1ZPSL0YsSGVlxKslOQ8T2Ikrhu8whDZNpYWaoXP0nK3aFK5Th
 jSYObXj/fXtj+APJZILoYAzEtlFsX0FFPfPZy2ffvX6+P1S4COsU2mkN9jJSu42C
 i8Ii5r1WAN6OPFbgPVQ2tQP9jq+scwBUT50GIzgLvja2dWPBM7drxB/hAByItgRt
 ARaDupnC6ShXPLMPm14fSfThyQQw+wiTHo9sZaks5NuoOz9WAh6XUK3JLR5S5zEG
 OxYunho2zSoqYAmnYtymNQ3L0ABW679EbbB2uZJX9m4iFGSxnNJoFFZyAXsZm/tM
 sRQBfTdBH1OCsVdy9/uYrNdZmj0QMBU5mJmv8Qsv722PLV8GuBF8O64C4F4Y42BP
 ApADllyRIONxXm95iBXISuZAF7qAMfGcvI9ldpUyetUT59oz4YI=
 =5Xi7
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/philmd-gitlab/tags/acceptance-testing-20201026' into staging

Acceptance testing patches

- More ARM tests
- Documentation update

CI jobs results:
. https://cirrus-ci.com/build/5754555684093952
. https://gitlab.com/philmd/qemu/-/pipelines/207807648

# gpg: Signature made Mon 26 Oct 2020 19:06:52 GMT
# gpg:                using RSA key FAABE75E12917221DCFD6BB2E3E32C2CDEADC0DE
# gpg: Good signature from "Philippe Mathieu-Daudé (F4BUG) <f4bug@amsat.org>" [full]
# Primary key fingerprint: FAAB E75E 1291 7221 DCFD  6BB2 E3E3 2C2C DEAD C0DE

* remotes/philmd-gitlab/tags/acceptance-testing-20201026:
  docs/devel/testing.rst: Update outdated Avocado URLs
  tests/acceptance: Allow running Orange Pi test using cached artifacts
  tests/acceptance: Add a 'virt_kvm' test using the GICv3
  tests/boot_linux_console: Boot Trusted Firmware-A on the Raspberry Pi 3

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-10-27 16:58:39 +00:00
Max Reitz
45ced7ca2f tests/acceptance/boot_linux: Accept SSH pubkey
Let download_cloudinit() take an optional pubkey, which subclasses of
BootLinux can pass through setUp().

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200909184028.262297-8-mreitz@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: WIllian Rampazzo <willianr@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2020-10-26 18:35:32 +00:00
Philippe Mathieu-Daudé
9923375807 tests/acceptance: Add a 'virt_kvm' test using the GICv3
The current 'virt_kvm' test is restricted to GICv2, but can also
work with a GICv3. Duplicate it but add a GICv3 test which can be
tested on some hardware.

Noticed while running:

 $ avocado --show=app run -t machine:virt tests/acceptance/
 ...
 (2/6) tests/acceptance/boot_linux.py:BootLinuxAarch64.test_virt_kvm: ERROR: Unexpected empty reply from server (1.82 s)

The job.log content is:

  L0351 DEBUG| Output: 'qemu-system-aarch64: host does not support in-kernel GICv2 emulation\n'

With this patch:

 $ avocado --show=app run -t device:gicv3 tests/acceptance/
 (1/1) tests/acceptance/boot_linux.py:BootLinuxAarch64.test_virt_kvm_gicv3: PASS (55.10 s)

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20200929224857.1225107-1-philmd@redhat.com>
2020-10-26 18:20:51 +01:00
Alex Bennée
0f26d94ec9 tests/acceptance: skip s390x_ccw_vrtio_tcg on GitLab
Currently the test takes more the 900 seconds on GitLab and then times
out. Running on Travis seems to be OK.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200701135652.1366-27-alex.bennee@linaro.org>
2020-07-11 15:53:00 +01:00
Pavel Dovgalyuk
1c80c87c8c tests/acceptance: refactor boot_linux to allow code reuse
This patch moves image downloading functions to the separate class to allow
reusing them from record/replay tests.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <159073593167.20809.17582679291556188984.stgit@pasha-ThinkPad-X280>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-05-31 18:25:31 +02:00
Cleber Rosa
6fd52d671d Acceptance test: add "boot_linux" tests
This acceptance test, validates that a full blown Linux guest can
successfully boot in QEMU.  In this specific case, the guest chosen is
Fedora version 31.

 * x86_64, pc-i440fx and pc-q35 machine types, with TCG and KVM as
   accelerators

 * aarch64 and virt machine type, with TCG and KVM as accelerators

 * ppc64 and pseries machine type with TCG as accelerator

 * s390x and s390-ccw-virtio machine type with TCG as accelerator

The Avocado vmimage utils library is used to download and cache the
Linux guest images, and from those images a snapshot image is created
and given to QEMU.  If a qemu-img binary is available in the build
directory, it's used to create the snapshot image, so that matching
qemu-system-* and qemu-img are used in the same test run.  If qemu-img
is not available in the build tree, one is attempted to be found
installed system-wide (in the $PATH).  If qemu-img is not found in the
build dir or in the $PATH, the test is canceled.

The method for checking the successful boot is based on "cloudinit"
and its "phone home" feature.  The guest is given an ISO image with
the location of the phone home server, and the information to post
(the instance ID).  Upon receiving the correct information, from the
guest, the test is considered to have PASSed.

This test is currently limited to user mode networking only, and
instructs the guest to connect to the "router" address that is hard
coded in QEMU.

To create the cloudinit ISO image that will be used to configure the
guest, the pycdlib library is also required and has been added as
requirement to the virtual environment created by "check-venv".

The console output is read by a separate thread, by means of the
Avocado datadrainer utility module.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Tested-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20200317141654.29355-3-crosa@redhat.com>
Signed-off-by: Cleber Rosa <crosa@redhat.com>
2020-03-17 18:55:52 -04:00