Commit Graph

389 Commits

Author SHA1 Message Date
Peter Maydell
8b47ec7abe .gitlab-ci.d: Drop cross-win32-system job
We don't support 32-bit Windows any more, so we don't need to defend it
with this CI job.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: "Daniel P. Berrangé" <berrange@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-ID: <20240222130920.362517-3-peter.maydell@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-02-23 08:13:52 +01:00
Daniel P. Berrangé
a8bf9de2f4 gitlab: force allow use of pip in Cirrus jobs
Python is transitioning to a world where you're not allowed to use 'pip
install' outside of a virutal env by default. The rationale is to stop
use of pip clashing with distro provided python packages, which creates
a major headache on distro upgrades.

All our CI environments, however, are 100% disposable so the upgrade
headaches don't exist. Thus we can undo the python defaults to allow
pip to work.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Thomas Huth <thuth@redhat.com>
Message-id: 20240222114038.2348718-1-berrange@redhat.com
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2024-02-22 15:36:17 +00:00
Peter Maydell
5cd3ae4903 .gitlab-ci.d/windows.yml: Drop msys2-32bit job
MSYS2 is dropping support for 32-bit Windows.  This shows up for us
as various packages we were using in our CI job no longer being
available to install, which causes the job to fail.  In commit
8e31b744fd we dropped the dependency on libusb and spice, but the
dtc package has also now been removed.

For us as QEMU upstream, "32 bit x86 hosts for system emulation" have
already been deprecated as of QEMU 8.0, so we are ready to drop them
anyway.

Drop the msys2-32bit CI job, as the first step in doing this.

This is cc'd to stable, because this job will also be broken for CI
on the stable branches.  We can't drop 32-bit support entirely there,
but we will still be covering at least compilation for 32-bit Windows
via the cross-win32-system job.

Cc: qemu-stable@nongnu.org
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-id: 20240220165602.135695-1-peter.maydell@linaro.org
2024-02-22 12:42:42 +00:00
Peter Maydell
760b4dcddd * Some hw/isa cleanups
* Fixes for x86 CPUID
 * Cleanups for configure, hw/isa and x86
 -----BEGIN PGP SIGNATURE-----
 
 iQFIBAABCAAyFiEE8TM4V0tmI4mGbHaCv/vSX3jHroMFAmXPW5sUHHBib256aW5p
 QHJlZGhhdC5jb20ACgkQv/vSX3jHroO1Mgf/USSqZxAyS1tyutpVslKmQjCyUiNW
 xEuHhAqoTz4Xg/8Q1J9aCcs2hdpBx2ULEQryUiTlH3LPLy3j9lGxMfg9Ma65mPVg
 KOSm/vfnw2VPXav8MJVNbm0gSI5fPmprRYgFi4TrkknmKnDo2TElrCG6Oe6LN/X+
 kiJD9K8PE+OE7xVhNDf7JAxUHt531oX7iM8p6JGGBYQdPOZ8rGf4K5PDrDxnHckG
 dbKHhqkbrSHaX/gu4q0ukgfIprs40ChYQ10vZ95mYyuWgIHOi51PciPmQ/qLPof0
 b/au08F0SpTE5cxx1Rk4cAuhP/PtzVln5lsSl9r9KzJ3Hcv6qnwXUU1nUg==
 =U+N2
 -----END PGP SIGNATURE-----

Merge tag 'for-upstream' of https://gitlab.com/bonzini/qemu into staging

* Some hw/isa cleanups
* Fixes for x86 CPUID
* Cleanups for configure, hw/isa and x86

# -----BEGIN PGP SIGNATURE-----
#
# iQFIBAABCAAyFiEE8TM4V0tmI4mGbHaCv/vSX3jHroMFAmXPW5sUHHBib256aW5p
# QHJlZGhhdC5jb20ACgkQv/vSX3jHroO1Mgf/USSqZxAyS1tyutpVslKmQjCyUiNW
# xEuHhAqoTz4Xg/8Q1J9aCcs2hdpBx2ULEQryUiTlH3LPLy3j9lGxMfg9Ma65mPVg
# KOSm/vfnw2VPXav8MJVNbm0gSI5fPmprRYgFi4TrkknmKnDo2TElrCG6Oe6LN/X+
# kiJD9K8PE+OE7xVhNDf7JAxUHt531oX7iM8p6JGGBYQdPOZ8rGf4K5PDrDxnHckG
# dbKHhqkbrSHaX/gu4q0ukgfIprs40ChYQ10vZ95mYyuWgIHOi51PciPmQ/qLPof0
# b/au08F0SpTE5cxx1Rk4cAuhP/PtzVln5lsSl9r9KzJ3Hcv6qnwXUU1nUg==
# =U+N2
# -----END PGP SIGNATURE-----
# gpg: Signature made Fri 16 Feb 2024 12:56:59 GMT
# gpg:                using RSA key F13338574B662389866C7682BFFBD25F78C7AE83
# gpg:                issuer "pbonzini@redhat.com"
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>" [full]
# gpg:                 aka "Paolo Bonzini <pbonzini@redhat.com>" [full]
# Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4  E2F7 7E15 100C CD36 69B1
#      Subkey fingerprint: F133 3857 4B66 2389 866C  7682 BFFB D25F 78C7 AE83

* tag 'for-upstream' of https://gitlab.com/bonzini/qemu:
  ci: Fix again build-previous-qemu
  usb: inline device creation functions
  target/i386: Generate an illegal opcode exception on cmp instructions with lock prefix
  i386: xen: fix compilation --without-default-devices
  configure: put all symlink creation together
  configure: do not create legacy symlinks
  smc37c669: remove useless is_enabled functions
  isa-superio: validate floppy.count value
  mips: remove unnecessary "select PTIMER"
  i386/cpuid: Move leaf 7 to correct group
  i386/cpuid: Remove subleaf constraint on CPUID leaf 1F
  i386/cpuid: Decrease cpuid_i when skipping CPUID leaf 1F
  physmem: replace function name with __func__ in ram_block_discard_range()
  i386/pc: Drop pc_machine_kvm_type()
  target/i386: Add support of KVM_FEATURE_ASYNC_PF_VMEXIT for guest
  i386/cpu: Mask with XCR0/XSS mask for FEAT_XSAVE_XCR0_HI and FEAT_XSAVE_XSS_HI leafs
  i386/cpu: Clear FEAT_XSAVE_XSS_LO/HI leafs when CPUID_EXT_XSAVE is not available

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2024-02-20 10:11:08 +00:00
Paolo Bonzini
5f9beb5001 ci: Fix again build-previous-qemu
The build-previous-qemu job is now trying to fetch from the upstream
repository, but the tag is only fetched into FETCH_HEAD:

$ git remote add upstream https://gitlab.com/qemu-project/qemu 00:00
$ git fetch upstream $QEMU_PREV_VERSION 00:02
warning: redirecting to https://gitlab.com/qemu-project/qemu.git/
From https://gitlab.com/qemu-project/qemu
 * tag                     v8.2.0     -> FETCH_HEAD
$ git checkout $QEMU_PREV_VERSION 00:02
error: pathspec v8.2.0 did not match any file(s) known to git

Fix by fetching the tag into the checkout itself.

Reviewed-by: Fabiano Rosas <farosas@suse.de>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-02-16 13:56:09 +01:00
Peter Maydell
8e31b744fd .gitlab-ci/windows.yml: Don't install libusb or spice packages on 32-bit
When msys2 updated their libusb packages to libusb 1.0.27, they
dropped support for building them for mingw32, leaving only mingw64
packages.  This broke our CI job, as the 'pacman' package install now
fails with:

error: target not found: mingw-w64-i686-libusb
error: target not found: mingw-w64-i686-usbredir

(both these binary packages are from the libusb source package).

Similarly, spice is now 64-bit only:
error: target not found: mingw-w64-i686-spice

Fix this by dropping these packages from the list we install for our
msys2-32bit build.  We do this with a simple mechanism for the
msys2-64bit and msys2-32bit jobs to specify a list of extra packages
to install on top of the common ones we install for both jobs.

Cc: qemu-stable@nongnu.org
Resolves: https://gitlab.com/qemu-project/qemu/-/issues/2160
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Michael Tokarev <mjt@tls.msk.ru>
Message-id: 20240215155009.2422335-1-peter.maydell@linaro.org
2024-02-16 11:05:04 +00:00
Peter Xu
940bf8ff1c ci: Update comment for migration-compat-aarch64
It turns out that we may not be able to enable this test even for the
upcoming v9.0.  Document what we're still missing.

Reviewed-by: "Daniel P. Berrangé" <berrange@redhat.com>
Link: https://lore.kernel.org/r/20240207005403.242235-4-peterx@redhat.com
Signed-off-by: Peter Xu <peterx@redhat.com>
2024-02-07 10:51:27 +08:00
Peter Xu
7070477902 ci: Remove tag dependency for build-previous-qemu
The new build-previous-qemu job relies on QEMU release tag being present,
while that may not be always true for personal git repositories since by
default tag is not pushed.  The job can fail on those CI kicks, as reported
by Peter Maydell.

Fix it by fetching the tags remotely from the official repository, as
suggested by Dan.

[1] https://lore.kernel.org/r/ZcC9ScKJ7VvqektA@redhat.com

Reported-by: Peter Maydell <peter.maydell@linaro.org>
Suggested-by: "Daniel P. Berrangé" <berrange@redhat.com>
Reviewed-by: "Daniel P. Berrangé" <berrange@redhat.com>
Link: https://lore.kernel.org/r/20240207005403.242235-3-peterx@redhat.com
Signed-off-by: Peter Xu <peterx@redhat.com>
2024-02-07 10:51:27 +08:00
Fabiano Rosas
079687e418 ci: Disable migration compatibility tests for aarch64
Until 9.0 is out, we need to keep the aarch64 job disabled because the
tests always use the n-1 version of migration-test. That happens to be
broken for aarch64 in 8.2. Once 9.0 is out, it will become the n-1
version and it will bring the fixed tests.

We can revert this patch when 9.0 releases.

Signed-off-by: Fabiano Rosas <farosas@suse.de>
Link: https://lore.kernel.org/r/20240118164951.30350-4-farosas@suse.de
[peterx: use _SKIPPED rather than _OPTIONAL]
Signed-off-by: Peter Xu <peterx@redhat.com>
2024-01-29 11:02:12 +08:00
Fabiano Rosas
94766edb35 ci: Add a migration compatibility test job
The migration tests have support for being passed two QEMU binaries to
test migration compatibility.

Add a CI job that builds the lastest release of QEMU and another job
that uses that version plus an already present build of the current
version and run the migration tests with the two, both as source and
destination. I.e.:

 old QEMU (n-1) -> current QEMU (development tree)
 current QEMU (development tree) -> old QEMU (n-1)

The purpose of this CI job is to ensure the code we're about to merge
will not cause a migration compatibility problem when migrating the
next release (which will contain that code) to/from the previous
release.

The version of migration-test used will be the one matching the older
QEMU. That way we can avoid special-casing new tests that wouldn't be
compatible with the older QEMU.

Note: for user forks, the version tags need to be pushed to gitlab
otherwise it won't be able to checkout a different version.

Signed-off-by: Fabiano Rosas <farosas@suse.de>
Link: https://lore.kernel.org/r/20240118164951.30350-3-farosas@suse.de
Signed-off-by: Peter Xu <peterx@redhat.com>
2024-01-29 11:02:12 +08:00
Peter Maydell
977542ded7 testing and misc updates
- add LE microblaze test to avocado
   - use modern snapshot=on to avoid trashing disk image
   - use plain bool for fe_is_open
   - various updates to qtest timeouts
   - enable meson test timeouts
   - tweak the readthedocs environment
   - partially revert un-flaking x86_64
 -----BEGIN PGP SIGNATURE-----
 
 iQEzBAABCgAdFiEEZoWumedRZ7yvyN81+9DbCVqeKkQFAmWhPccACgkQ+9DbCVqe
 KkS5agf+OoW6HOitt34YeL6cGGtIKaxbta+Fs6jq+ucIbN63TmLTuKrmPiRNxjuo
 Fj2Qvh9R7Tl7Q/a7ZAym0Fze7GtsvvsidkiQS4pmi9vYuJrhS734CxXHT8JS6zJr
 ymQ0nGZODg1cVB4oAR9sXo/OwEQdDTSgKp8wdNr930fxYwokUKBUgcOqElu3SWHv
 duSYDuaflnP5B8ZGbb1ZnOlwS9lZIHTwjZyN5J1YtxF0T8Ez4A+xseEOpQ/00MoE
 Ecjdp3ELCxzOI+1U33Yni7ol//fxQpRKi+xf2fGIxhuSA3i32rmY5NWTvl7VwuS1
 gXryjX2rukSujySP3vkdtTp0dmkbpg==
 =ZuDd
 -----END PGP SIGNATURE-----

Merge tag 'pull-testing-updates-120124-2' of https://gitlab.com/stsquad/qemu into staging

testing and misc updates

  - add LE microblaze test to avocado
  - use modern snapshot=on to avoid trashing disk image
  - use plain bool for fe_is_open
  - various updates to qtest timeouts
  - enable meson test timeouts
  - tweak the readthedocs environment
  - partially revert un-flaking x86_64

# -----BEGIN PGP SIGNATURE-----
#
# iQEzBAABCgAdFiEEZoWumedRZ7yvyN81+9DbCVqeKkQFAmWhPccACgkQ+9DbCVqe
# KkS5agf+OoW6HOitt34YeL6cGGtIKaxbta+Fs6jq+ucIbN63TmLTuKrmPiRNxjuo
# Fj2Qvh9R7Tl7Q/a7ZAym0Fze7GtsvvsidkiQS4pmi9vYuJrhS734CxXHT8JS6zJr
# ymQ0nGZODg1cVB4oAR9sXo/OwEQdDTSgKp8wdNr930fxYwokUKBUgcOqElu3SWHv
# duSYDuaflnP5B8ZGbb1ZnOlwS9lZIHTwjZyN5J1YtxF0T8Ez4A+xseEOpQ/00MoE
# Ecjdp3ELCxzOI+1U33Yni7ol//fxQpRKi+xf2fGIxhuSA3i32rmY5NWTvl7VwuS1
# gXryjX2rukSujySP3vkdtTp0dmkbpg==
# =ZuDd
# -----END PGP SIGNATURE-----
# gpg: Signature made Fri 12 Jan 2024 13:25:27 GMT
# gpg:                using RSA key 6685AE99E75167BCAFC8DF35FBD0DB095A9E2A44
# gpg: Good signature from "Alex Bennée (Master Work Key) <alex.bennee@linaro.org>" [full]
# Primary key fingerprint: 6685 AE99 E751 67BC AFC8  DF35 FBD0 DB09 5A9E 2A44

* tag 'pull-testing-updates-120124-2' of https://gitlab.com/stsquad/qemu: (22 commits)
  tests/avocado: partially revert unmasking of replay_linux tests
  readthodocs: fully specify a build environment
  mtest2make: stop disabling meson test timeouts
  tests/fp: Bump fp-test-mulAdd test timeout to 3 minutes
  tests/unit: Bump test-crypto-block test timeout to 5 minutes
  tests/unit: Bump test-aio-multithread test timeout to 2 minutes
  tests/qtest: Bump the device-introspect-test timeout to 12 minutes
  qtest: bump bios-table-test timeout to 9 minutes
  qtest: bump aspeed_smc-test timeout to 6 minutes
  qtest: bump qos-test timeout to 2 minutes
  qtest: bump boot-serial-test timeout to 3 minutes
  qtest: bump prom-env-test timeout to 6 minutes
  qtest: bump pxe-test timeout to 10 minutes
  qtest: bump test-hmp timeout to 4 minutes
  qtest: bump npcm7xx_pwm-test timeout to 5 minutes
  qtest: bump qom-test timeout to 15 minutes
  qtest: bump migration-test timeout to 8 minutes
  qtest: bump min meson timeout to 60 seconds
  chardev: use bool for fe_is_open
  gitlab: include microblazeel in testing
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2024-01-12 14:02:53 +00:00
Alex Bennée
2d41bf0fe1 gitlab: include microblazeel in testing
This reverts aeb5f8f248 (gitlab: build the correct microblaze target)
now we actually have a little-endian test in avocado thanks to this
years advent calendar.

Message-Id: <20240103173349.398526-4-alex.bennee@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
2024-01-12 13:23:48 +00:00
Peter Maydell
52a21689cd .gitlab-ci.d/buildtest.yml: Work around htags bug when environment is large
Sometimes the CI "pages" job fails with a message like this from
htags:

$ htags -anT --tree-view=filetree -m qemu_init -t "Welcome to the QEMU sourcecode"
htags: Negative exec line limit = -371

This is due to a bug in hflags where if the environment is too large it
falls over:
https://lists.gnu.org/archive/html/bug-global/2024-01/msg00000.html

This happens to us because GitLab CI puts the commit message of the
commit under test into the CI_COMMIT_MESSAGE and/or CI_COMMIT_TAG_MESSAGE
environment variables, so the job will fail if the commit happens to
have a verbose commit message.

Work around the htags bug by unsetting these variables while running
htags.

Cc: qemu-stable@nongnu.org
Resolves: https://gitlab.com/qemu-project/qemu/-/issues/2080
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240111125543.1573473-1-peter.maydell@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-01-11 17:49:21 +01:00
Nicholas Piggin
cdd30f369a gitlab: fix s390x tag for avocado-system-centos
The 390x tag should be s390x.

Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
Message-ID: <20240107170119.82222-2-npiggin@gmail.com>
Reviewed-by: Cédric Le Goater <clg@kaod.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-01-11 11:39:26 +01:00
Alex Bennée
5d25fcb702 gitlab: add optional job to run flaky avocado tests
One problem with flaky tests is they often only fail under CI
conditions which makes it hard to debug. We add an optional allow_fail
job so developers can trigger the only the flaky tests in the CI
environment if they are debugging.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231201093633.2551497-8-alex.bennee@linaro.org>
2023-12-01 17:47:20 +00:00
Alex Bennée
aeb5f8f248 gitlab: build the correct microblaze target
We inadvertently built the LE target for BE tests.

Fixes: 78ebc00b06 (gitlab: shuffle some targets and reduce avocado noise)
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231201093633.2551497-7-alex.bennee@linaro.org>
2023-12-01 17:47:20 +00:00
Philippe Mathieu-Daudé
541069e653 .gitlab-ci.d/cirrus: Add manual testing of macOS 14 (Sonoma)
Upgrade libvirt-ci so it covers macOS 14. Add a manual entry
(QEMU_JOB_OPTIONAL: 1) to test on Sonoma release. Refresh the
lci-tool generated files.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20231109160504.93677-3-philmd@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-11-24 16:21:55 +01:00
Greg Manning
4789f9d3a1 plugins: fix win plugin tests on cross compile
Resolves: https://gitlab.com/qemu-project/qemu/-/issues/1972

Cross compile gcc is more picky about argument order than msys. Changed
the meson command to take the (now renamed) libqemu_plugin_api.a as a
lib, rather than an object. This puts it in the right place on both
native and cross compile gcc commands

Reenable plugins on crossbuilds

Signed-off-by: Greg Manning <gmanning@rapitasystems.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20231109124326.21106-2-gmanning@rapitasystems.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231120150833.2552739-5-alex.bennee@linaro.org>
2023-11-23 14:10:04 +00:00
Alex Bennée
8e721c3277 tests/docker: merge debian-native with debian-amd64
debian-native isn't really needed and suffers from the problem of
tracking a distros dependencies rather than the projects. With a
little surgery we can make the debian-amd64 container architecture
neutral and allow people to use it to build a native QEMU.

Rename it so it follows the same non-arch pattern of the other distro
containers.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Anders Roxell <anders.roxell@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231120150833.2552739-4-alex.bennee@linaro.org>
2023-11-23 14:10:04 +00:00
Philippe Mathieu-Daudé
7528ef7321 .gitlab-ci.d/cirrus: Upgrade macOS to 13 (Ventura)
macOS 14 "Sonoma" was released on September 2023 [1].

According to QEMU's support policy, we stop supporting the
previous major release two years after the the new major
release has been published. Replace the macOS 12 (Monterey)
testing by macOS 13 (Ventura, released on October 2022, [2]).

Refresh the generated files by running:

  $ make lcitool-refresh

[1] https://www.apple.com/newsroom/2023/09/macos-sonoma-is-available-today/
[2] https://www.apple.com/newsroom/2022/10/macos-ventura-is-now-available/

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20231108162022.76189-1-philmd@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231120150833.2552739-3-alex.bennee@linaro.org>
2023-11-23 14:10:02 +00:00
Daniel P. Berrangé
aa5730b07e tests/docker: replace fedora-i386 with debian-i686
Fedora is gradually killing off i386 packages in its repos, via a
death-by-1000-cuts process. Thus Debian looks like a better long
term bet for i686 build testing. It has the added advantage that
we can generate it via lcitool too.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20231107164109.1449014-1-berrange@redhat.com>
[AJB: tweak commit msg, set correct prefix]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231120150833.2552739-2-alex.bennee@linaro.org>
2023-11-23 11:55:32 +00:00
Alex Bennée
f8347d05df gitlab: add dlltool to Windows CI
We also --disable-plugins for the two mingw based cross builds as
although they have dlltool they seem to be unhappy linking.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2023-11-08 15:15:23 +00:00
Alex Bennée
7ccb4153fe tests/docker: use debian-all-test-cross for sparc64
Maintaining two sets of containers for test building is silly. While
it makes sense for the QEMU cross-compile targets to have their own
fat containers built by lcitool we might as well merge the other
random debian based compilers into the same one used on gitlab.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231029145033.592566-15-alex.bennee@linaro.org>
2023-10-31 14:10:21 +00:00
Alex Bennée
26025d8e36 tests/docker: use debian-all-test-cross for riscv64
Maintaining two sets of containers for test building is silly. While
it makes sense for the QEMU cross-compile targets to have their own
fat containers built by lcitool we might as well merge the other
random debian based compilers into the same one used on gitlab.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231029145033.592566-14-alex.bennee@linaro.org>
2023-10-31 14:10:14 +00:00
Alex Bennée
b09bb6d1b8 tests/docker: use debian-all-test-cross for mips
Maintaining two sets of containers for test building is silly. While
it makes sense for the QEMU cross-compile targets to have their own
fat containers built by lcitool we might as well merge the other
random debian based compilers into the same one used on gitlab.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231029145033.592566-13-alex.bennee@linaro.org>
2023-10-31 14:10:14 +00:00
Alex Bennée
92a3165e1a tests/docker: use debian-all-test-cross for mips64
Maintaining two sets of containers for test building is silly. While
it makes sense for the QEMU cross-compile targets to have their own
fat containers built by lcitool we might as well merge the other
random debian based compilers into the same one used on gitlab.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231029145033.592566-12-alex.bennee@linaro.org>
2023-10-31 14:10:14 +00:00
Alex Bennée
9d9a573612 tests/docker: use debian-all-test-cross for m68k
Maintaining two sets of containers for test building is silly. While
it makes sense for the QEMU cross-compile targets to have their own
fat containers built by lcitool we might as well merge the other
random debian based compilers into the same one used on gitlab.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231029145033.592566-11-alex.bennee@linaro.org>
2023-10-31 14:10:14 +00:00
Alex Bennée
95f5bf9521 tests/docker: use debian-all-test-cross for hppa
Maintaining two sets of containers for test building is silly. While
it makes sense for the QEMU cross-compile targets to have their own
fat containers built by lcitool we might as well merge the other
random debian based compilers into the same one used on gitlab.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231029145033.592566-10-alex.bennee@linaro.org>
2023-10-31 14:10:14 +00:00
Alex Bennée
eb4cb4ed1f tests/docker: use debian-all-test-cross for power
Maintaining two sets of containers for test building is silly. While
it makes sense for the QEMU cross-compile targets to have their own
fat containers built by lcitool we might as well merge the other
random debian based compilers into the same one used on gitlab.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231029145033.592566-9-alex.bennee@linaro.org>
2023-10-31 14:10:06 +00:00
Alex Bennée
4e76d98ae9 tests/docker: move sh4 to use debian-legacy-test-cross
sh4 is another target which doesn't work with bookworm compilers. To
keep on buster move across to the debian-legacy-test-cross image and
update accordingly.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20231030135715.800164-1-alex.bennee@linaro.org>
2023-10-31 13:28:34 +00:00
Alex Bennée
d004e27b4e tests/docker: use debian-legacy-test-cross for alpha
Maintaining two sets of containers for test building is silly. While
it makes sense for the QEMU cross-compile targets to have their own
fat containers built by lcitool we might as well merge the other
random debian based compilers into the same one used on gitlab.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231029145033.592566-7-alex.bennee@linaro.org>
2023-10-31 13:28:30 +00:00
Alex Bennée
cb8715bf0c gitlab: add build-loongarch to matrix
We have the compiler and with a few updates a container that can build
QEMU so we should at least run the check-tcg smoke tests.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231029145033.592566-6-alex.bennee@linaro.org>
2023-10-31 13:28:30 +00:00
Alex Bennée
6f6c399976 gitlab: clean-up build-soft-softmmu job
Having dropped alpha we also now drop xtensa as we don't have the
compiler in this image. It's not all doom and gloom though as a number
of other targets have gained softmmu TCG tests so we can add them. We
will take care of the other targets with their own containers in
future commits.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231029145033.592566-5-alex.bennee@linaro.org>
2023-10-31 13:28:30 +00:00
Alex Bennée
aa39c4c098 gitlab: split alpha testing into a legacy container
The current bookworm compiler doesn't build the static binaries due to
bug #1054412 and it might be awhile before it gets fixed. The problem
of keeping older architecture compilers running isn't going to go away
so lets prepare the ground. Create a legacy container and move some
tests around so the others can get upgraded.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231029145033.592566-4-alex.bennee@linaro.org>
2023-10-31 13:28:25 +00:00
Thomas Huth
f51f90c65e gitlab-ci: Disable the riscv64-debian-cross-container by default
This job is failing since weeks. Let's mark it as manual until
it gets fixed.

Message-Id: <82aa015a-ca94-49ce-beec-679cc175b726@redhat.com>
Acked-by: Michael Tokarev <mjt@tls.msk.ru>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-10-12 14:18:03 +02:00
Alex Bennée
78ebc00b06 gitlab: shuffle some targets and reduce avocado noise
We move a couple of targets out of the avocado runs because there are
no tests to run. Tricore already has some coverage.  The cris target
only really has check-tcg tests but its getting harder to find
anything that packages the compiler.

To reduce the noise of CANCEL messages we also set AVOCADO_TAGS
appropriately so we filter down the number of tests we attempt.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231009164104.369749-5-alex.bennee@linaro.org>
2023-10-11 08:46:33 +01:00
Alex Bennée
3e3df0d84f tests/lcitool: add swtpm to the package list
We need this to test some TPM stuff.

Reviewed-by: "Daniel P. Berrangé" <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231009164104.369749-4-alex.bennee@linaro.org>
2023-10-11 08:46:23 +01:00
Daniel P. Berrangé
eca74afd7d gitlab: make Cirrus CI jobs gating
The Cirrus CI jobs have been non-gating for a while to let us build
confidence in their reliability. Aside from periodic dependancy
problems when FreeBSD Ports switches to be based on a new FreeBSD
image version, the jobs have been reliable. It is thus worth making
them gating to prevent build failures being missed during merges.

Signed-off-by: "Daniel P. Berrangé" <berrange@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20230912184130.3056054-5-berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230914155422.426639-8-alex.bennee@linaro.org>
2023-09-20 15:06:33 +01:00
Daniel P. Berrangé
c576d8bfbb gitlab: make Cirrus CI timeout explicit
On the GitLab side we're invoking the Cirrus CI job using the
cirrus-run tool which speaks to the Cirrus REST API. Cirrus
sometimes tasks 5-10 minutes to actually schedule the task,
and thus the execution time of 'cirrus-run' inside GitLab will
be slightly longer than the execution time of the Cirrus CI
task.

Setting the timeout in the GitLab CI job should thus be done
in relation to the timeout set for the Cirrus CI job. While
Cirrus CI defaults to 60 minutes, it is better to set this
explicitly, and make the relationship between the jobs
explicit

Signed-off-by: "Daniel P. Berrangé" <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20230912184130.3056054-4-berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230914155422.426639-7-alex.bennee@linaro.org>
2023-09-20 15:06:33 +01:00
Alex Bennée
3aa2cfa0e6 gitlab: fix typo/spelling in comments
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230914155422.426639-3-alex.bennee@linaro.org>
2023-09-20 15:06:33 +01:00
Thomas Huth
e05a4beaec .gitlab-ci.d/cirrus.yml: Update FreeBSD to v13.2
The FreeBSD CI job started to fail due to linking problems ... time
to update to the latest version to get this fixed.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20230823144533.230477-1-thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230829161528.2707696-6-alex.bennee@linaro.org>
2023-08-30 14:57:50 +01:00
Daniel P. Berrangé
2f7350cd43 gitlab: enable ccache for many build jobs
The `ccache` tool can be very effective at reducing compilation times
when re-running pipelines with only minor changes each time. For example
a fresh 'build-system-fedora' job will typically take 20 minutes on the
gitlab.com shared runners. With ccache this is reduced to as little as
6 minutes.

Normally meson would auto-detect existance of ccache in $PATH and use
it automatically, but the way we wrap meson from configure breaks this,
as we're passing in an config file with explicitly set compiler paths.
Thus we need to add $CCACHE_WRAPPERSPATH to the front of $PATH. For
unknown reasons if doing this in msys though, gcc becomes unable to
invoke 'cc1' when run from meson. For msys we thus set CC='ccache gcc'
before invoking 'configure' instead.

A second problem with msys is that cache misses are incredibly
expensive, so enabling ccache massively slows down the build when
the cache isn't well populated. This is suspected to be a result of
the cost of spawning processes under the msys architecture. To deal
with this we set CCACHE_DEPEND=1 which enables ccache's 'depend_only'
strategy. This avoids extra spawning of the pre-processor during
cache misses, with the downside that is it less likely ccache will
find a cache hit after semantically benign compiler flag changes.
This is the lesser of two evils, as otherwise we can't use ccache
at all under msys and remain inside the job time limit.

If people are finding ccache to hurt their pipelines, it can be
disabled by setting the 'CCACHE_DISABLE=1' env variable against
their gitlab fork CI settings.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20230804111054.281802-2-berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230829161528.2707696-2-alex.bennee@linaro.org>
2023-08-30 14:57:33 +01:00
Paolo Bonzini
7ace219303 tests/docker: add python3-tomli dependency to containers
Instead of having CI pick tomli from the vendored wheel at configure
time, place it in the containers.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-08-28 09:55:48 +02:00
Paolo Bonzini
c03f57fd5b Revert "tests: Use separate virtual environment for avocado"
This reverts commit e8e4298fea.

ensuregroup allows to specify both the acceptable versions of avocado,
and a locked version to be used when avocado is not installed as a system
pacakge.  This lets us install avocado in pyvenv/ using "mkvenv.py" and
reuse the distro package on Fedora and CentOS Stream (the only distros
where it's available).

ensuregroup's usage of "(>=..., <=...)" constraints when evaluating
the distro package, and "==" constraints when installing it from PyPI,
makes it possible to avoid conflicts between the known-good version and
a package plugins included in the distro.

This is because package plugins have "==" constraints on the version
that is included in the distro, and, using "pip install avocado==88.1"
on a venv that includes system packages will result in an error:

   avocado-framework-plugin-varianter-yaml-to-mux 98.0 requires avocado-framework==98.0, but you have avocado-framework 88.1 which is incompatible.
   avocado-framework-plugin-result-html 98.0 requires avocado-framework==98.0, but you have avocado-framework 88.1 which is incompatible.

But at the same time, if the venv does not include a system distribution
of avocado then we can install a known-good version and stick to LTS
releases.

Resolves: https://gitlab.com/qemu-project/qemu/-/issues/1663
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-08-28 09:55:48 +02:00
Paolo Bonzini
d9ab1f1f4d ci: install meson in CirrusCI KVM build environment
scripts/archive-source.sh needs meson in order to download the subprojects,
therefore meson needs to be part of the host environment in which VM-based
build jobs run.

Fixes: 2019cabfee ("meson: subprojects: replace submodules with wrap files", 2023-06-06)
Reported-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-08-04 13:56:17 +02:00
Daniel P. Berrangé
f54ba56dad gitlab: disable FF_SCRIPT_SECTIONS on msys jobs
The FF_SCRIPT_SECTIONS=1 variable should ordinarily cause output from
each line of the job script to be presented in a collapsible section
with execution time listed.

While it works on Linux shared runners, when used with Windows runners
with PowerShell, this option does not create any sections, and actually
causes echo'ing of commands to be disabled, making it even worse to
debug the jobs.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Acked-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20230801130403.164060-9-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-08-03 13:04:48 +02:00
Daniel P. Berrangé
63f5365cd4 gitlab: disable optimization and debug symbols in msys build
Building at -O2, adds 33% to the build time, over -O2. IOW a build that
takes 45 minutes at -O0, takes 60 minutes at -O2. Turning off debug
symbols drops it further, down to 38 minutes.

IOW, a "-O2 -g" build is 58% slower than a "-O0" build on msys in the
gitlab CI windows shared runners.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20230801130403.164060-8-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-08-03 13:04:48 +02:00
Daniel P. Berrangé
ef4fe31f33 gitlab: always populate cache for windows msys jobs
The cache is used to hold the msys installer. Even if the build phase
fails, we should still populate the cache as the installer will be
valid for next time.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20230801130403.164060-6-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-08-03 13:04:47 +02:00
Daniel P. Berrangé
5ef56e3b18 gitlab: drop $CI_PROJECT_DIR from cache path
The gitlab cache is limited to only handle content within the
$CI_PROJECT_DIR hierarchy, and as such relative paths are always
implicitly relative to $CI_PROJECT_DIR.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20230801130403.164060-5-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-08-03 13:04:47 +02:00
Daniel P. Berrangé
46aedd34b4 gitlab: always use updated msys installer
We current reference an msys installer binary from mid-2022, which means
after installation, it immediately has to re-download a bunch of newer
content. This wastes precious CI time.

The msys project publishes an installer binary with a fixed URL that
always references the latest content. We cache the downloads in gitlab
though and so once downloaded we would never re-fetch the installer
leading back to the same problem.

To deal with this we also fetch the pgp signature for the installer
on every run, and compare that to the previously cached signature. If
the signature changes, we re-download the full installer.

This ensures we always have the latest installer for msys, while also
maximising use of the gitlab cache.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20230801130403.164060-4-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-08-03 13:04:47 +02:00
Daniel P. Berrangé
b64052cdad gitlab: print timestamps during windows msys jobs
It is hard to get visibility into where time is consumed in our Windows
msys jobs. Adding a few log console messages with the timestamp will
aid in our debugging.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20230801130403.164060-3-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-08-03 13:04:47 +02:00
Daniel P. Berrangé
11961d08fc gitlab: remove duplication between msys jobs
Although they share a common parent, the two msys jobs still have
massive duplication in their script definitions that can easily be
collapsed.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20230801130403.164060-2-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-08-03 13:04:47 +02:00
Alex Bennée
dc389ce1b3 gitlab: reduce testing scope of check-gcov
This keeps timing out on gitlab due to some qtests taking a long time.
As this is just ensuring the gcov machinery is working and not
attempting to be comprehensive lets skip qtest in this run.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230630180423.558337-4-alex.bennee@linaro.org>
2023-07-03 12:51:14 +01:00
Daniel P. Berrangé
dcb242c849 gitlab: ensure coverage job also publishes meson log
The coverage job wants to publish a coverage report on success, but the
tests might fail and in that case we need the meson logs for debugging.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20230503145535.91325-3-berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230630180423.558337-3-alex.bennee@linaro.org>
2023-07-03 12:51:08 +01:00
Daniel P. Berrangé
cef63308e6 gitlab: explicit set artifacts publishing criteria
If not set explicitly, gitlab assumes 'when: on_success" as the
publishing criteria for artifacts. This is reasonable if the
artifact is an output deliverable of the job. This is useless
if the artifact is a log file to be used for debugging job
failures.

This change makes the desired criteria explicit for every job
that publishes artifacts.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20230503145535.91325-2-berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230630180423.558337-2-alex.bennee@linaro.org>
2023-07-03 12:51:01 +01:00
Richard Henderson
3b746a43eb gitlab: Disable plugins for cross-i386-tci
There are timeouts in the cross-i386-tci job that are related to plugins.
Restrict this job to basic TCI testing.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Acked-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20230629130844.151453-1-richard.henderson@linaro.org>
2023-06-29 18:04:06 +02:00
Marc-André Lureau
81a12315ad gitlab-ci: add msys2 meson test to junit report
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20230620153720.514882-3-marcandre.lureau@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-06-26 08:58:02 +02:00
Marc-André Lureau
a3fcbb4790 gitlab-ci: grab msys2 meson-logs as artifacts
Rename build directory to "build", like most other CI builds.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20230620153720.514882-2-marcandre.lureau@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-06-26 08:58:02 +02:00
Daniel P. Berrangé
49ac76c2a0 gitlab: support disabling job auto-run in upstream
In forks QEMU_CI=1 can be used to create a pipeline but not auto-run any
jobs. In upstream jobs always auto-run, which is equiv of QEMU_CI=2.

This supports setting QEMU_CI=1 in upstream, to disable job auto-run.
This can be used to preserve CI minutes if repushing a branch to staging
with a specific fix that only needs testing in limited scenarios.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20230608164018.2520330-6-berrange@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-06-26 08:58:02 +02:00
Daniel P. Berrangé
a77ef83cf8 gitlab: avoid extra pipelines for tags and stable branches
In upstream context we only run pipelines on staging branches, and
limited publishing jobs on the default branch.

We don't want to run pipelines on stable branches, or tags, because
the content will have already been tested on a staging branch before
getting pushed.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Michael Tokarev <mjt@tls.msk.ru>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20230608164018.2520330-5-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-06-26 08:58:02 +02:00
Daniel P. Berrangé
e28112d007 gitlab: stable staging branches publish containers in a separate tag
If the stable staging branches publish containers under the 'latest' tag
they will clash with containers published on the primary staging branch,
as well  as with each other. This introduces logic that overrides the
container tag when jobs run against the stable staging branches.

The CI_COMMIT_REF_SLUG variable we use expands to the git branch name,
but with most special characters removed, such that it is valid as a
docker tag name. eg 'staging-8.0' will get a slug of 'staging-8-0'

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Michael Tokarev <mjt@tls.msk.ru>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20230608164018.2520330-4-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-06-26 08:58:02 +02:00
Daniel P. Berrangé
1ddd2ff9cd gitlab: allow overriding name of the upstream repository
The CI rules have special logic for what happens in upstream. To enable
contributors who modify CI rules to test this logic, however, they need
to be able to override which repo is considered upstream. This
introduces the 'QEMU_CI_UPSTREAM' variable

  git push gitlab <branch> -o ci.variable=QEMU_CI_UPSTREAM=berrange

to make it look as if my namespace is the actual upstream. Namespace in
this context refers to the path fragment in gitlab URLs that is above
the repository. Typically this will be the contributor's gitlab login
name.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Michael Tokarev <mjt@tls.msk.ru>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20230608164018.2520330-3-berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-06-26 08:58:02 +02:00
Daniel P. Berrangé
d4c7a56539 gitlab: centralize the container tag name
We use a fixed container tag of 'latest' so that contributors' forks
don't end up with an ever growing number of containers as they work
on throwaway feature branches.

This fixed tag causes problems running CI upstream in stable staging
branches, however, because the stable staging branch will publish old
container content that clashes with that needed by primary staging
branch. This makes it impossible to reliably run CI pipelines in
parallel in upstream for different staging branches.

This introduces $QEMU_CI_CONTAINER_TAG global variable as a way to
change which tag container publishing uses. Initially it can be set
by contributors as a git push option if they want to override the
default use of 'latest' eg

  git push gitlab <branch> -o ci.variable=QEMU_CONTAINER_TAG=fish

this is useful if contributors need to run pipelines for different
branches concurrently in their forks.

Reviewed-by: Michael Tokarev <mjt@tls.msk.ru>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20230608164018.2520330-2-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-06-26 08:58:02 +02:00
Richard Henderson
c81e2d5477 gitlab: Add cross-arm64-kvm-only
We are not currently running a --disable-tcg test for arm64,
like we are for mips, ppc and s390x.  We have a job for the
native aarch64 runner, but it is not run by default and it
is not helpful for normal developer testing without access
to qemu's private runner.

Use --without-default-features to eliminate most tests.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-06-07 08:35:24 -07:00
Richard Henderson
80bdd58ae4 * finish atomics revamp
* meson.build tweaks
 * revert avocado update
 * always upgrade/downgrade locally installed Python packages
 * switch from submodules to subprojects
 * remove --with-git= option
 * rename --enable-pypi to --enable-download, control submodules and subprojects too
 -----BEGIN PGP SIGNATURE-----
 
 iQFIBAABCAAyFiEE8TM4V0tmI4mGbHaCv/vSX3jHroMFAmR/Qu8UHHBib256aW5p
 QHJlZGhhdC5jb20ACgkQv/vSX3jHroMmSwgAj5SHD8R+5D1UmptzBvI/72CfgqVv
 MJa4O1LvHwUkuSmxX1MFFhRa0mo0bu6j+bPpvJ29zKS61ybVwJl87gnsRcDAMXe7
 08YbcG35Chox6aZxbidUQtXm18JZ3F2aMtmxUuP0PR7LDjVXLV5FsjrHTIt8KuEZ
 vUqq3IsVbc4FxCCC0ke2DzrtgpRCxYSdfPrj/t5WzAztAXId9r1zvUlCLN+FUpri
 E3KIZYpkXZyOnJQ9W30KnsZo5QtDACwlIMBK6whSdoCjyNN7TwDdhNW8QkOueNO6
 q3tLfwf5+u6uyEoaQTW+teE2oMXT8N4IJllRJj2RyQ1BFD49XhUUJmc33Q==
 =b9QD
 -----END PGP SIGNATURE-----

Merge tag 'for-upstream' of https://gitlab.com/bonzini/qemu into staging

* finish atomics revamp
* meson.build tweaks
* revert avocado update
* always upgrade/downgrade locally installed Python packages
* switch from submodules to subprojects
* remove --with-git= option
* rename --enable-pypi to --enable-download, control submodules and subprojects too

# -----BEGIN PGP SIGNATURE-----
#
# iQFIBAABCAAyFiEE8TM4V0tmI4mGbHaCv/vSX3jHroMFAmR/Qu8UHHBib256aW5p
# QHJlZGhhdC5jb20ACgkQv/vSX3jHroMmSwgAj5SHD8R+5D1UmptzBvI/72CfgqVv
# MJa4O1LvHwUkuSmxX1MFFhRa0mo0bu6j+bPpvJ29zKS61ybVwJl87gnsRcDAMXe7
# 08YbcG35Chox6aZxbidUQtXm18JZ3F2aMtmxUuP0PR7LDjVXLV5FsjrHTIt8KuEZ
# vUqq3IsVbc4FxCCC0ke2DzrtgpRCxYSdfPrj/t5WzAztAXId9r1zvUlCLN+FUpri
# E3KIZYpkXZyOnJQ9W30KnsZo5QtDACwlIMBK6whSdoCjyNN7TwDdhNW8QkOueNO6
# q3tLfwf5+u6uyEoaQTW+teE2oMXT8N4IJllRJj2RyQ1BFD49XhUUJmc33Q==
# =b9QD
# -----END PGP SIGNATURE-----
# gpg: Signature made Tue 06 Jun 2023 07:30:07 AM PDT
# gpg:                using RSA key F13338574B662389866C7682BFFBD25F78C7AE83
# gpg:                issuer "pbonzini@redhat.com"
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>" [unknown]
# gpg:                 aka "Paolo Bonzini <pbonzini@redhat.com>" [unknown]
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg:          There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4  E2F7 7E15 100C CD36 69B1
#      Subkey fingerprint: F133 3857 4B66 2389 866C  7682 BFFB D25F 78C7 AE83

* tag 'for-upstream' of https://gitlab.com/bonzini/qemu: (21 commits)
  configure: remove --with-git-submodules=
  build: remove git submodule handling from main makefile
  meson: subprojects: replace berkeley-{soft,test}float-3 with wraps
  pc-bios/s390-ccw: always build network bootloader
  configure: move SLOF submodule handling to pc-bios/s390-ccw
  meson: subprojects: replace submodules with wrap files
  build: log submodule update from git-submodule.sh
  git-submodule: allow partial update of .git-submodule-status
  configure: rename --enable-pypi to --enable-download, control subprojects too
  configure: remove --with-git= option
  mkvenv: always pass locally-installed packages to pip
  tests: Use separate virtual environment for avocado
  Revert "tests/requirements.txt: bump up avocado-framework version to 101.0"
  scsi/qemu-pr-helper: Drop support for 'old' libmultipath API
  meson.build: Use -Wno-undef only for SDL2 versions that need it
  meson.build: Group the audio backend entries in a separate summary section
  meson.build: Group the network backend entries in a separate summary section
  meson.build: Group the UI entries in a separate summary section
  scripts: remove dead file
  atomics: eliminate mb_read/mb_set
  ...

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-06-06 10:17:20 -07:00
Paolo Bonzini
1f468152fb build: remove git submodule handling from main makefile
The only remaining user of submodules at build time is roms/SLOF,
which is handled in pc-bios/s390-ccw/Makefile.  Remove the relevant
code from the main makefile.

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-06-06 16:30:01 +02:00
Paolo Bonzini
b11f9bd96f configure: move SLOF submodule handling to pc-bios/s390-ccw
Move the handling of the roms/SLOF submodule out of the main Makefile,
since we are going to remove submodules from the build process of QEMU.

Acked-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-06-06 16:30:01 +02:00
Paolo Bonzini
2019cabfee meson: subprojects: replace submodules with wrap files
Compared to submodules, .wrap files have several advantages:

* option parsing and downloading is delegated to meson

* the commit is stored in a text file instead of a magic entry in the
  git tree object

* we could stop shipping external dependencies that are only used as a
  fallback, but not break compilation on platforms that lack them.
  For example it may make sense to download dtc at build time, controlled
  by --enable-download, even when building from a tarball.  Right now,
  this patch does the opposite: make-release treats dtc like libvfio-user
  (which is not stable API and therefore hasn't found its way into any
  distros) and keycodemap (which is a copylib, for better or worse).

dependency() can fall back to a wrap automatically.  However, this
is only possible for libraries that come with a .pc file, and this
is not very common for libfdt even though the upstream project in
principle provides it; it also removes the control that we provide with
--enable-fdt={system,internal}.  Therefore, the logic to pick system
vs. internal libfdt is left untouched.

--enable-fdt=git is removed; it was already a synonym for
--enable-fdt=internal.

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-06-06 16:30:01 +02:00
Paolo Bonzini
e8e4298fea tests: Use separate virtual environment for avocado
This reverts commits eea2d14117 ("Makefile: remove $(TESTS_PYTHON)",
2023-05-26) and 9c6692db55 ("tests: Use configure-provided pyvenv for
tests", 2023-05-18).

Right now, there is a conflict between wanting a ">=" constraint when
using a distro-provided package and wanting a "==" constraint when
installing Avocado from PyPI; this would provide the best of both worlds
in terms of resiliency for both distros that have required packages and
distros that don't.

The conflict is visible also for meson, where we would like to install
the latest 0.63.x version but also accept a distro 1.1.x version.
But it is worse for avocado, for two reasons:

1) we cannot use an "==" constraint to install avocado if the venv
includes a system avocado.  The distro will package plugins that have
"==" constraints on the version that is included in the distro, and, using
"pip install avocado==88.1" on a venv that includes system packages will
result in this error:

   ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
   avocado-framework-plugin-varianter-yaml-to-mux 98.0 requires avocado-framework==98.0, but you have avocado-framework 88.1 which is incompatible.
   avocado-framework-plugin-result-html 98.0 requires avocado-framework==98.0, but you have avocado-framework 88.1 which is incompatible.
   make[1]: Leaving directory '/home/berrange/src/virt/qemu/build'

2) we cannot use ">=" either if the venv does _not_ include a system
avocado, because that would result in the installation of v101.0 which
is the one we've just reverted.

So the idea is to encode the dependencies as an (acceptable, locked)
tuple, like this hypothetical TOML that would be committed inside
python/ and used by mkvenv.py:

  [meson]
  meson = { minimum = "0.63.0", install = "0.63.3", canary = "meson" }

  [docs]
  # 6.0 drops support for Python 3.7
  sphinx = { minimum = "1.6", install = "<6.0", canary = "sphinx-build" }
  sphinx_rtd_theme = { minimum = "0.5" }

  [avocado]
  avocado-framework = { minimum = "88.1", install = "88.1", canary = "avocado" }

Once this is implemented, it would also be possible to install avocado in
pyvenv/ using "mkvenv.py ensure", thus using the distro package on Fedora
and CentOS Stream (the only distros where it's available).  But until
this is implemented, keep avocado in a separate venv.  There is still the
benefit of using a single python for meson custom_targets and for sphinx.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-06-06 16:29:53 +02:00
Camilla Conte
98300bcdba gitlab-ci: Remove unused Python package
Python should have been removed in this commit:
94b8b146df

Signed-off-by: Camilla Conte <cconte@redhat.com>
Message-Id: <20230531150824.32349-2-cconte@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-06-05 20:48:34 +02:00
Daniel P. Berrangé
ab72522797 gitlab: switch from 'stable' to 'latest' docker container tags
The 'stable' and 'stable-dind' tags are not documented as supported
tags at:

  https://hub.docker.com/_/docker

Looking at their content they reflect docker 19.x.x release series,
were last built in Dec 2020, and have 3 critical and 20 high rated
CVEs unfixed. This obsolete status is attested by this commit:

  606c63960a

The 'stable-dind' tag in particular appears buggy as it is unable to
resolve DNS for Fedora repos:

  - Curl error (6): Couldn't resolve host name for https://mirrors.fedoraproject.org/metalink?repo=fedora-37&arch=x86_64&countme=1 [getaddrinfo() thread failed to start]

We used the 'stable' tag previously at the recommendation of GitLab
docs, but those docs are wrong and pending a fix:

  https://gitlab.com/gitlab-org/gitlab/-/issues/409430

Fixes: 5f63a67adb
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Camilla Conte <cconte@redhat.com>
Message-Id: <20230531140654.1141145-1-berrange@redhat.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-05-31 10:29:14 -07:00
Camilla Conte
037a171a2f Add CI variable RUNNER_TAG
This allows to set a job tag dynamically.
We need this to be able to select the Kubernetes runner.
See https://wiki.qemu.org/Testing/CI/KubernetesRunners.

Signed-off-by: Camilla Conte <cconte@redhat.com>
Message-Id: <20230522174153.46801-5-cconte@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-05-24 12:26:40 -07:00
Camilla Conte
b105ce60ca Add loop over docker info
Wait for docker info to return successfuly to ensure that
the docker server (daemon) started.
This is needed for jobs running on Kubernetes.
See https://wiki.qemu.org/Testing/CI/KubernetesRunners.

Signed-off-by: Camilla Conte <cconte@redhat.com>
Message-Id: <20230522174153.46801-4-cconte@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-05-24 12:26:40 -07:00
Camilla Conte
5f63a67adb Use docker "stable" tag
Use the same tag in all jobs.

Signed-off-by: Camilla Conte <cconte@redhat.com>
Message-Id: <20230522174153.46801-3-cconte@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-05-24 12:26:40 -07:00
Camilla Conte
a878264d6c Remove redundant CI variables
These are not needed when using gitlab.com shared runners.

Signed-off-by: Camilla Conte <cconte@redhat.com>
Message-Id: <20230522174153.46801-2-cconte@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-05-24 12:26:40 -07:00
Richard Henderson
37246d54d6 * First batch of fixes to allow "make check" with "--without-default-devices"
* Enable the "bios bits" avocado test in the gitlab-CI
 * Another minor fix for the redundancy DMA blocker code
 -----BEGIN PGP SIGNATURE-----
 
 iQJFBAABCAAvFiEEJ7iIR+7gJQEY8+q5LtnXdP5wLbUFAmRrVhoRHHRodXRoQHJl
 ZGhhdC5jb20ACgkQLtnXdP5wLbUaiRAApPVveet6WPQ7Ag1448LtqHTGiwl8x2Ba
 jQ7FTKhqdTC5O+/BU7IQkvGmErPxCc8WPB7eoowwBVA/4dr8YIIBLKqO4RtP6LXs
 rtUkzsPI9ExW+iJjIMVOmHsp/shlRhuf+Tmlr8OsTObecCeA4Vbxc+RlvYXfCPhM
 8tOuLO8n6LQY/62fgXSzI5WlLQSzIo3aDSmCeWa1QHkPLf6itvGkwsNBytMJLoUT
 pXZnBNqlXiuyPtloLp+DMfRRkpq8AHB04+Sri7TVPxi7bJL28RMZiaAXpvHSFLz8
 JR2ApRrzBthiLMK1I6A0c2ZGCbVOAi1dhNDNqWCyx8ZBASEJj0XuT/+Qse81sKmG
 zNXr57x0CzWAJ59/taBM2hjUks10rJOmxHJYxS6i1JJR7u1zTuvii7toPMmf35zX
 bM7TYjKpYGa2HneHpw1eOjpTgUYZpgla/pVXZhKqoGdfmseBMlFU424MNl/xDRng
 bxuam3Ku+ClOeQlzXt8aceL/gTApJfvy5FAIAK5yUOQDTs6HjJJL2AfcOzss8kXb
 k6IMHgV1tnLed8B7K4iml2rzvk+RT3CPGvmaNwSAkdh8SnE5/bv1I6s4fHiXMlvC
 mmfvFSoWwdhcsD5r+XOFxfke8sGrOeQIXKefp6UL3hYVV7o2NUe89BytXZCzut/Y
 6ulR25HHtmI=
 =m1Px
 -----END PGP SIGNATURE-----

Merge tag 'pull-request-2023-05-22' of https://gitlab.com/thuth/qemu into staging

* First batch of fixes to allow "make check" with "--without-default-devices"
* Enable the "bios bits" avocado test in the gitlab-CI
* Another minor fix for the redundancy DMA blocker code

# -----BEGIN PGP SIGNATURE-----
#
# iQJFBAABCAAvFiEEJ7iIR+7gJQEY8+q5LtnXdP5wLbUFAmRrVhoRHHRodXRoQHJl
# ZGhhdC5jb20ACgkQLtnXdP5wLbUaiRAApPVveet6WPQ7Ag1448LtqHTGiwl8x2Ba
# jQ7FTKhqdTC5O+/BU7IQkvGmErPxCc8WPB7eoowwBVA/4dr8YIIBLKqO4RtP6LXs
# rtUkzsPI9ExW+iJjIMVOmHsp/shlRhuf+Tmlr8OsTObecCeA4Vbxc+RlvYXfCPhM
# 8tOuLO8n6LQY/62fgXSzI5WlLQSzIo3aDSmCeWa1QHkPLf6itvGkwsNBytMJLoUT
# pXZnBNqlXiuyPtloLp+DMfRRkpq8AHB04+Sri7TVPxi7bJL28RMZiaAXpvHSFLz8
# JR2ApRrzBthiLMK1I6A0c2ZGCbVOAi1dhNDNqWCyx8ZBASEJj0XuT/+Qse81sKmG
# zNXr57x0CzWAJ59/taBM2hjUks10rJOmxHJYxS6i1JJR7u1zTuvii7toPMmf35zX
# bM7TYjKpYGa2HneHpw1eOjpTgUYZpgla/pVXZhKqoGdfmseBMlFU424MNl/xDRng
# bxuam3Ku+ClOeQlzXt8aceL/gTApJfvy5FAIAK5yUOQDTs6HjJJL2AfcOzss8kXb
# k6IMHgV1tnLed8B7K4iml2rzvk+RT3CPGvmaNwSAkdh8SnE5/bv1I6s4fHiXMlvC
# mmfvFSoWwdhcsD5r+XOFxfke8sGrOeQIXKefp6UL3hYVV7o2NUe89BytXZCzut/Y
# 6ulR25HHtmI=
# =m1Px
# -----END PGP SIGNATURE-----
# gpg: Signature made Mon 22 May 2023 04:46:34 AM PDT
# gpg:                using RSA key 27B88847EEE0250118F3EAB92ED9D774FE702DB5
# gpg:                issuer "thuth@redhat.com"
# gpg: Good signature from "Thomas Huth <th.huth@gmx.de>" [undefined]
# gpg:                 aka "Thomas Huth <thuth@redhat.com>" [undefined]
# gpg:                 aka "Thomas Huth <th.huth@posteo.de>" [unknown]
# gpg:                 aka "Thomas Huth <huth@tuxfamily.org>" [undefined]
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg:          There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 27B8 8847 EEE0 2501 18F3  EAB9 2ED9 D774 FE70 2DB5

* tag 'pull-request-2023-05-22' of https://gitlab.com/thuth/qemu:
  memory: stricter checks prior to unsetting engaged_in_io
  acpi/tests/avocado/bits: enable bios bits avocado tests on gitlab CI pipeline
  .gitlab-ci.d/buildtest.yml: Run full "make check" with --without-default-devices
  tests/qemu-iotests/172: Run QEMU with -vga none and -nic none
  tests/qtest/meson.build: Run the net filter tests only with default devices
  tests/qtest: Check for the availability of virtio-ccw devices before using them
  tests/qtest/virtio-ccw-test: Remove superfluous tests
  tests/qtest/cdrom-test: Fix the test to also work without optional devices
  tests/qtest/usb-hcd-uhci-test: Skip test if UHCI controller is not available
  tests/qtest/readconfig-test: Check for the availability of USB controllers
  hw/sparc64/sun4u: Use MachineClass->default_nic and MachineClass->no_parallel
  hw/i386: Ignore the default parallel port if it has not been compiled into QEMU
  hw/char/parallel: Move TYPE_ISA_PARALLEL to the header file
  hw/sh4: Use MachineClass->default_nic in the sh4 r2d machine
  hw/s390x: Use MachineClass->default_nic in the s390x machine
  hw/ppc: Use MachineClass->default_nic in the ppc machines
  softmmu/vl.c: Disable default NIC if it has not been compiled into the binary
  hw: Move the default NIC machine class setting from the x86 to the generic one
  softmmu/vl.c: Check for the availability of the VGA device before using it
  hw/i386/Kconfig: ISAPC works fine without VGA_ISA

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-05-22 09:06:08 -07:00
Thomas Huth
bb6e473450 .gitlab-ci.d/buildtest.yml: Run full "make check" with --without-default-devices
Thanks to the fixes from the previous patches, we can now run
the full set of "make check" with all targets here.

Message-Id: <20230512124033.502654-19-thuth@redhat.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-05-22 09:44:48 +02:00
Richard Henderson
48727e5757 Revert last two patches
Unintentionally pushed.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-05-19 12:15:03 -07:00
Camilla Conte
ee327b23d1 Raise crash-test-debian timeout to 90 minutes
When running on the Kubernetes runner, this CI job is timing out.
Raise the limit to give the job enough time to run.

Signed-off-by: Camilla Conte <cconte@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20230407145252.32955-2-cconte@redhat.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-05-19 12:06:14 -07:00
Camilla Conte
e4da92d994 Add CI configuration for Kubernetes
Configure Gitlab CI to run on Kubernetes
according to the official documentation.
https://docs.gitlab.com/ee/ci/docker/using_docker_build.html#docker-in-docker-with-tls-enabled-in-kubernetes

These changes are needed because of the CI jobs
using Docker-in-Docker (dind).
As soon as Docker-in-Docker is replaced with Kaniko,
these changes can be reverted.

I documented what I did to set up the Kubernetes runner on the wiki:
https://wiki.qemu.org/Testing/CI/KubernetesRunners

Signed-off-by: Camilla Conte <cconte@redhat.com>
Message-Id: <20230407145252.32955-1-cconte@redhat.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-05-19 12:06:14 -07:00
Paolo Bonzini
9fd9f3952f gitlab: custom-runners: preserve more artifacts for debugging
Since custom runners are not generally available, make it possible to
debug the differences between a successful and a failing build by
comparing the logs and the build.ninja rules.

Acked-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-05-19 20:40:29 +02:00
Paolo Bonzini
dde001ef71 remove remaining traces of meson submodule
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-05-19 20:09:21 +02:00
John Snow
9c6692db55 tests: Use configure-provided pyvenv for tests
This patch changes how the avocado tests are provided, ever so
slightly. Instead of creating a new testing venv, use the
configure-provided 'pyvenv' instead, and install optional packages into
that.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20230511035435.734312-20-jsnow@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-05-18 08:53:51 +02:00
John Snow
66e2c6cbac configure: use 'mkvenv ensure meson' to bootstrap meson
This commit changes how we detect and install meson. It notably removes
'--meson='.

Currently, configure creates a lightweight Python virtual environment
unconditionally using the user's configured $python that inherits system
packages. Temporarily, we forced the use of meson source present via git
submodule or in the release tarball.

With this patch, we restore the ability to use a system-provided meson:

If Meson is installed in the build venv and meets our minimum version
requirements, we will use that Meson. This includes a system provided
meson, which would be visible via system-site packages inside the venv.

In the event that Meson is installed but *not for the chosen Python
interpreter*, not found, or of insufficient version, we will attempt to
install Meson from vendored source into the newly created Python virtual
environment. This vendored installation replaces both the git submodule
and tarball source mechanisms for sourcing meson.

As a result of this patch, the Python interpreter we use for both our
own build scripts *and* Meson extensions are always known to be the
exact same Python. As a further benefit, there will also be a symlink
available in the build directory that points to the correct, configured
python and can be used by e.g. manual tests to invoke the correct,
configured Python unambiguously.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20230511035435.734312-18-jsnow@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-05-18 08:53:51 +02:00
Ani Sinha
da9000784c tests/lcitool: Add mtools and xorriso and remove genisoimage as dependencies
Bios bits avocado tests need mformat (provided by the mtools package) and
xorriso tools in order to run within gitlab CI containers. Add those
dependencies within the Dockerfiles so that containers can be built with
those tools present and bios bits avocado tests can be run there.

xorriso package conflicts with genisoimage package on some distributions.
Therefore, it is not possible to have both the packages at the same time
in the container image uniformly for all distribution flavors. Further,
on some distributions like RHEL, both xorriso and genisoimage
packages provide /usr/bin/genisoimage and on some other distributions like
Fedora, only genisoimage package provides the same utility.
Therefore, this change removes the dependency on geninsoimage for building
container images altogether keeping only xorriso package. At the same time,
cdrom-test.c is updated to use and check for existence of only xorrisofs.

Signed-off-by: Ani Sinha <anisinha@redhat.com>
Message-Id: <20230504154611.85854-3-anisinha@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-05-16 09:14:18 +02:00
Alex Bennée
2cf72cb5eb gitlab: enable minimal device profile for aarch64 --disable-tcg
As this is likely the most common configuration people will want once
the --disable-tcg patches land.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Cc: Fabiano Rosas <farosas@suse.de>
Message-Id: <20230503091244.1450613-21-alex.bennee@linaro.org>
2023-05-10 16:02:06 +01:00
Alex Bennée
eb8d413f7f gitlab: add ubuntu-22.04-aarch64-without-defaults
This does a very minimal build without default devices or features. I
chose the aarch64 runner as it doesn't count towards CI minutes and is
a fairly under-utilised builder.

Message-Id: <20230503091244.1450613-20-alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2023-05-10 16:01:37 +01:00
Alex Bennée
c1924cde87 scripts/ci: clean-up the 20.04/22.04 confusion in ansible
We have a bunch of references to 20.04 (which s390x is still on)
although we are basically building on 22.04 now. Clean up the textual
references and use lcitool to generate the full package list to be
consistent.

We can drop "Install packages to build QEMU on Ubuntu on non-s390x" as
when we upgrade the s390x builder to 22.04 it won't need this
workaround.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230503091244.1450613-19-alex.bennee@linaro.org>
2023-05-10 16:01:00 +01:00
Thomas Huth
df1f50c3c4 .gitlab-ci.d/cirrus: Drop the CI job for compiling with FreeBSD 12
FreeBSD 13.0 has been released in April 2021:

 https://www.freebsd.org/releases/13.0R/announce/

According to QEMU's support policy, we stop supporting the previous
major release two years after the the new major release has been
published. So we can stop testing FreeBSD 12 in our CI now.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20230418160225.529172-1-thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Warner Losh <imp@bsdimp.com>
Message-Id: <20230424092249.58552-8-alex.bennee@linaro.org>
2023-04-27 14:58:23 +01:00
Thomas Huth
8b869aa591 scripts/device-crash-test: Add a parameter to run with TCG only
We're currently facing the problem that the device-crash-test script
runs twice as long in the CI when a runner supports KVM - which sometimes
results in a timeout of the CI job. To get a more deterministic runtime
here, add an option to the script that allows to run it with TCG only.

Reported-by: Eldon Stegall <eldon-qemu@eldondev.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20230414145845.456145-3-thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230424092249.58552-6-alex.bennee@linaro.org>
2023-04-27 14:58:11 +01:00
Thomas Huth
4d3bd91b26 gitlab-ci: Avoid to re-run "configure" in the device-crash-test jobs
After "make check-venv" had been added to these jobs, they started
to re-run "configure" each time since our logic in the makefile
thinks that some files are out of date here. Avoid it with the same
trick that we are using in buildtest-template.yml already by disabling
the up-to-date check via NINJA=":".

Fixes: 1d8cf47e5b ("tests: run 'device-crash-test' from tests/venv")
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20230414145845.456145-2-thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230424092249.58552-5-alex.bennee@linaro.org>
2023-04-27 14:58:11 +01:00
Alex Bennée
fc9988916a gitlab: fix typo
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20230403134920.2132362-11-alex.bennee@linaro.org>
2023-04-04 15:56:44 +01:00
Philippe Mathieu-Daudé
8f03c08506 gitlab-ci: Cover SPICE in the MSYS2 job
Include the mingw-w64-x86_64-spice package so SPICE is covered:

  C compiler for the host machine: cc -m64 -mcx16 (gcc 12.2.0 "cc (Rev10, Built by MSYS2 project) 12.2.0")
  ...
  Run-time dependency spice-protocol found: YES 0.14.4
  Run-time dependency spice-server found: YES 0.15.1

In particular this would have helped catching the build issue
reported as https://gitlab.com/qemu-project/qemu/-/issues/1553:

  [1851/5253] Compiling C object libcommon.fa.p/ui_spice-core.c.obj
  FAILED: libcommon.fa.p/ui_spice-core.c.obj
  ../ui/spice-core.c: In function 'watch_remove':
  ../ui/spice-core.c:152:5: error: implicit declaration of function 'qemu_close_to_socket' [-Werror=implicit-function-declaration]
  152 |     qemu_close_to_socket(watch->fd);
      |     ^~~~~~~~~~~~~~~~~~~~
  ../ui/spice-core.c:152:5: error: nested extern declaration of 'qemu_close_to_socket' [-Werror=nested-externs]

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20230322135721.61138-2-philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-03-24 12:04:06 +01:00
Alex Bennée
dbe9a9cdbb gitlab: update centos-8-stream job
A couple of clean-ups here:

  - inherit from the custom runners job for artefacts
  - call check-avocado directly
  - add some comments to the top about setup

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20230315174331.2959-6-alex.bennee@linaro.org>
2023-03-22 15:06:57 +00:00
Thomas Huth
d4f784c51b gitlab-ci.d/buildtest: Rework the target list of build-system-alpine
The target list of the build-system-alpine job is pretty much a copy
of the build-system-ubuntu job (apart from "aarch64-softmmu" which
has recently been removed from the ubuntu job in commit 6eda5ef5f8,
but aarch64-softmmu is still also tested in the opensuse jobs, so
we don't need to keep it here).

Let's stop wasting our CI minutes with such duplications, and focus
on testing targets instead that do not have such a great test coverage
yet: The "loongarch64-softmmu" target has never been added to our
build tests yet since it has been introduced, and the "mips64-softmmu"
target is so far only tested in jobs that lack the "avocado" testing
stage (only the little endian or 32-bit MIPS variants are tested in
jobs with avocado so far).

While we're at it, also move the avr-softmmu and mipsel-softmmu targets
from the Debian job to the alpine job, since the Debian job (and its
following test jobs) has already a long runtime compared to the others
jobs. With this movement, the runtimes should be more equally distributed
along the parallel running jobs now.

Message-Id: <20230309164850.109882-1-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-03-13 09:23:38 +01:00
Philippe Mathieu-Daudé
690ceb7193 gitlab-ci: Remove job building EDK2 firmware binaries
When we introduced this Gitlab-CI job in commit 71920809ce
("gitlab-ci.yml: Add jobs to build EDK2 firmware binaries"),
the naive plan was to have reproducible binaries by downloading
what this job would build, testing it and eventually committing
it. With retrospective, nothing happened 3 years later and this
job is just bitrotting:

  Step 1/3 : FROM ubuntu:18.04
  18.04: Pulling from library/ubuntu
  mediaType in manifest should be
  'application/vnd.docker.distribution.manifest.v2+json' not
  'application/vnd.oci.image.manifest.v1+json'

Remove this job to avoid wasting maintenance and CI ressources.

Reported-by: Palmer Dabbelt <palmer@rivosinc.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20230310133247.39268-1-philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-03-13 09:20:52 +01:00
Peter Maydell
dea644928d * Refine the distro support policy
* Deprecate 32-bit x86 and arm hosts for system emulation
 * Check bison version to be >= 3.0
 * Compile vnc test only if vnc is really enabled
 * Check docs/config/ich9-ehci-uhci.cfg via the readconfig-test
 * s390x: Add support for list-directed IPL from ECKD DASD
 -----BEGIN PGP SIGNATURE-----
 
 iQJFBAABCAAvFiEEJ7iIR+7gJQEY8+q5LtnXdP5wLbUFAmQIQD0RHHRodXRoQHJl
 ZGhhdC5jb20ACgkQLtnXdP5wLbX+EA/5ATc/Rz9Y8TZF36/sUKrmjOxQnX91hgRW
 zovgZejoHtek1AC4pLd9kcpo1JonXkOfmbkNRYTw6+1wY6Ipt2apkWAc+MqfvQ80
 dGznbLXnX/E2QHW7h5XqvAhRGvjog9b38zAQcR37PKZ9DkSfWBwMgTT2SXYpLUjR
 YWgqPGBYRq1ViG4q7hv2dHqi/idzh+DOWFGU5NAQC1fEBOmPYN9u6vfbX86dwZsy
 S9pqw16yN2MZKL2bgIViFIjdhPPMO3RFcv8NTPipzk3g/pHNIv58+xCE8r/Tp8LW
 JRQzdh6JbGvj1BXH68igXWGUFW6dhOXZWhgmFe4QEDaiCj91pHGvQCYgo2agx2bl
 rXaSQXSMhIcm+t9C9kO6UNxZf+1HohcM1ZlQXhvX6FcBPVMUx/52YEpiK+OtBKyq
 43AlKp4fp1xXLFSOyMNgSMR90sksng6CEoQqTr0jCZtF8H18wb+eHfb6dME3XRCw
 SKQeNGhkP1FxaQnlRAkeBxW7GkDE5YBjf9pSIaLZ/8VsLdyo3SK/DLkre5+qnu/V
 lksZlt1K7xDLxRFopBuVJ3MHeFSRazBDbWfrRFOALkdbGEEeZnnmEPFjD3mag+P4
 1y9WmBJHRLlvhlm195WA8babenUZoOfLkPrL0k7mNLs9sZyx1eIkZfOUgCWjFE2V
 9VBOWL8oi/w=
 =p13b
 -----END PGP SIGNATURE-----

Merge tag 'pull-request-2023-03-07' of https://gitlab.com/thuth/qemu into staging

* Refine the distro support policy
* Deprecate 32-bit x86 and arm hosts for system emulation
* Check bison version to be >= 3.0
* Compile vnc test only if vnc is really enabled
* Check docs/config/ich9-ehci-uhci.cfg via the readconfig-test
* s390x: Add support for list-directed IPL from ECKD DASD

# -----BEGIN PGP SIGNATURE-----
#
# iQJFBAABCAAvFiEEJ7iIR+7gJQEY8+q5LtnXdP5wLbUFAmQIQD0RHHRodXRoQHJl
# ZGhhdC5jb20ACgkQLtnXdP5wLbX+EA/5ATc/Rz9Y8TZF36/sUKrmjOxQnX91hgRW
# zovgZejoHtek1AC4pLd9kcpo1JonXkOfmbkNRYTw6+1wY6Ipt2apkWAc+MqfvQ80
# dGznbLXnX/E2QHW7h5XqvAhRGvjog9b38zAQcR37PKZ9DkSfWBwMgTT2SXYpLUjR
# YWgqPGBYRq1ViG4q7hv2dHqi/idzh+DOWFGU5NAQC1fEBOmPYN9u6vfbX86dwZsy
# S9pqw16yN2MZKL2bgIViFIjdhPPMO3RFcv8NTPipzk3g/pHNIv58+xCE8r/Tp8LW
# JRQzdh6JbGvj1BXH68igXWGUFW6dhOXZWhgmFe4QEDaiCj91pHGvQCYgo2agx2bl
# rXaSQXSMhIcm+t9C9kO6UNxZf+1HohcM1ZlQXhvX6FcBPVMUx/52YEpiK+OtBKyq
# 43AlKp4fp1xXLFSOyMNgSMR90sksng6CEoQqTr0jCZtF8H18wb+eHfb6dME3XRCw
# SKQeNGhkP1FxaQnlRAkeBxW7GkDE5YBjf9pSIaLZ/8VsLdyo3SK/DLkre5+qnu/V
# lksZlt1K7xDLxRFopBuVJ3MHeFSRazBDbWfrRFOALkdbGEEeZnnmEPFjD3mag+P4
# 1y9WmBJHRLlvhlm195WA8babenUZoOfLkPrL0k7mNLs9sZyx1eIkZfOUgCWjFE2V
# 9VBOWL8oi/w=
# =p13b
# -----END PGP SIGNATURE-----
# gpg: Signature made Wed 08 Mar 2023 07:58:53 GMT
# gpg:                using RSA key 27B88847EEE0250118F3EAB92ED9D774FE702DB5
# gpg:                issuer "thuth@redhat.com"
# gpg: Good signature from "Thomas Huth <th.huth@gmx.de>" [full]
# gpg:                 aka "Thomas Huth <thuth@redhat.com>" [full]
# gpg:                 aka "Thomas Huth <huth@tuxfamily.org>" [full]
# gpg:                 aka "Thomas Huth <th.huth@posteo.de>" [unknown]
# Primary key fingerprint: 27B8 8847 EEE0 2501 18F3  EAB9 2ED9 D774 FE70 2DB5

* tag 'pull-request-2023-03-07' of https://gitlab.com/thuth/qemu:
  pc-bios/s390-ccw: Update s390-ccw.img with the list-directed IPL fix
  pc-bios: Add support for List-Directed IPL from ECKD DASD
  docs/config: Set the "kvm" accelerator via "[accel]" section
  tests/qtest/readconfig: Test docs/config/ich9-ehci-uhci.cfg
  tests/qtest/readconfig: Rework test_object_rng_resp into a generic function
  gitlab-ci.d/crossbuilds: Drop the 32-bit arm system emulation jobs
  docs/about/deprecated: Deprecate 32-bit arm hosts for system emulation
  gitlab-ci.d/crossbuilds: Drop the i386 system emulation job
  docs/about/deprecated: Deprecate 32-bit x86 hosts for system emulation
  include/hw/i386: Clean up includes in x86.h
  test: Check vnc enable before compiling vnc test
  Hexagon (meson.build): define min bison version
  docs/about/build-platforms: Refine the distro support policy

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2023-03-09 15:19:30 +00:00
Thomas Huth
3f0760ea31 gitlab-ci.d/crossbuilds: Drop the 32-bit arm system emulation jobs
Hardly anybody still uses 32-bit arm environments for running QEMU,
so let's stop wasting our scarce CI minutes with these jobs.

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Wilfred Mallawa <wilfred.mallawa@wdc.com>
Message-Id: <20230306084658.29709-6-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-03-08 08:57:42 +01:00
Thomas Huth
4f9a8315e6 gitlab-ci.d/crossbuilds: Drop the i386 system emulation job
Hardly anybody still uses 32-bit x86 environments for running QEMU with
full system emulation, so let's stop wasting our scarce CI minutes with
this job.

(There are still the 32-bit MinGW and TCI jobs around for having
some compile test coverage on 32-bit)

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Wilfred Mallawa <wilfred.mallawa@wdc.com>
Message-Id: <20230306084658.29709-4-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-03-07 14:30:42 +01:00
Palmer Dabbelt
7bc1286b81
gitlab/opensbi: Move to docker:stable
The OpenSBI build has been using docker:19.03.1, which appears to be old
enough that v2 of the manifest is no longer supported.  Something has
started serving us those manifests, resulting in errors along the lines
of

    $ docker build --cache-from $IMAGE_TAG --tag $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA --tag $IMAGE_TAG .gitlab-ci.d/opensbi
    Step 1/7 : FROM ubuntu:18.04
    18.04: Pulling from library/ubuntu
    mediaType in manifest should be 'application/vnd.docker.distribution.manifest.v2+json' not 'application/vnd.oci.image.manifest.v1+json'

This moves to docker:stable, as was suggested by the template.  It also
adds the python3 package via apt, as OpenSBI requires that to build.

Reviewed-by: Bin Meng <bmeng@tinylab.org>
Message-ID: <20230303202448.11911-2-palmer@rivosinc.com>
Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-03-05 15:40:00 -08:00