Commit Graph

101 Commits

Author SHA1 Message Date
Paolo Bonzini
951f71ad67 dockerfiles: install bindgen from cargo on Ubuntu 22.04
Because Ubuntu 22.04 has a very old version of bindgen, that
does not have the important option --allowlist-file, it will
not be able to use --enable-rust out of the box.  Instead,
install the latest version of bindgen-cli via "cargo install"
in the container, following QEMU's own documentation.

Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-11-06 17:12:17 +01:00
Thomas Huth
9094f7c934 .gitlab-ci.d/cirrus: Remove the macos-15 job
Cirrus-CI stopped providing the possibility to run macOS 15 jobs.
Quoting https://cirrus-ci.org/guide/macOS/ :

 "Cirrus CI Cloud only allows ghcr.io/cirruslabs/macos-runner:sonoma image ..."

If you still try to run a Sequoia image, it gets automatically "upgraded"
to Sonoma instead. So the macos-15 job in the QEMU CI now does not
make sense anymore, thus let's remove it.

Message-ID: <20241021124722.139348-1-thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Acked-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-11-04 14:16:07 +01:00
Dorjoy Chowdhury
1ac32dc8ea tests/lcitool: Update libvirt-ci and add libcbor dependency
libcbor dependecy is necessary for adding virtio-nsm and nitro-enclave
machine support in the following commits. libvirt-ci has already been
updated with the dependency upstream and this commit updates libvirt-ci
submodule in QEMU to latest upstream. Also the libcbor dependency has
been added to tests/lcitool/projects/qemu.yml.

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Dorjoy Chowdhury <dorjoychy111@gmail.com>
Reviewed-by: Alexander Graf <graf@amazon.com>
Link: https://lore.kernel.org/r/20241008211727.49088-2-dorjoychy111@gmail.com
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-10-31 18:28:32 +01:00
Daniel P. Berrangé
548de8f8dc tests: add 'rust' and 'bindgen' to CI package list
Although we're not enabling rust by default yet, we can still add
rust and bindgen to the CI package list.

This demonstrates that we're not accidentally triggering unexpected
build behaviour merely from Rust being present. When we do dev work
to enable rust by default, this will show we're building correctly
on all platforms we target.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Link: https://lore.kernel.org/r/20241015133925.311587-2-berrange@redhat.com
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-10-30 16:30:56 +01:00
Daniel P. Berrangé
aa54f5be44 tests: update lcitool to fix freebsd py311-yaml rename
This addresses the py311-yaml -> py311-pyyaml rename in FreeBSD.

The change to the OpenSUSE dockerfile is something that will allow
QEMU to access rust bindgen in future.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-id: 20241014130255.10119-1-berrange@redhat.com
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2024-10-14 15:54:24 +01:00
Peter Maydell
b38d263bca * first commit for Rust support
* add CI job using Fedora + Rust nightly
 * fix detection of ATOMIC128 on x86_64
 * fix compilation with Sphinx 8.1.0
 -----BEGIN PGP SIGNATURE-----
 
 iQFIBAABCAAyFiEE8TM4V0tmI4mGbHaCv/vSX3jHroMFAmcJEKUUHHBib256aW5p
 QHJlZGhhdC5jb20ACgkQv/vSX3jHroOSZQf+LlvZm9npHR6lZ9DEruhu/uf0c1gO
 9+dBJiKQ1OWopSQOqEgOsLL0J123Ls4V8O3tzZwIDuuRofCB2+wKswad6CHoydJx
 4p9rRXv6MLlnTqqGxemm/dPZqJ7+6L0poHoDKW+s7AgfVDshhj1RSbQfs8Ujh41F
 f1sdi3DzopVWtK4CE+8/UeLy5Cxlixke9SKhYQrFHrdsANARP81gxQjczKApMc1z
 v9qkrLtkM06VUyuvbPps7CHSHDpzx9mXcmkkPgLqLX9MfbCztzi44aVSaS9HYk5G
 y54dSKdY7VJEuGhG916G+GMDJyow4nhT9Gk6tWtk63TQN5nExVsoZMOmdw==
 =PFGL
 -----END PGP SIGNATURE-----

Merge tag 'for-upstream' of https://gitlab.com/bonzini/qemu into staging

* first commit for Rust support
* add CI job using Fedora + Rust nightly
* fix detection of ATOMIC128 on x86_64
* fix compilation with Sphinx 8.1.0

# -----BEGIN PGP SIGNATURE-----
#
# iQFIBAABCAAyFiEE8TM4V0tmI4mGbHaCv/vSX3jHroMFAmcJEKUUHHBib256aW5p
# QHJlZGhhdC5jb20ACgkQv/vSX3jHroOSZQf+LlvZm9npHR6lZ9DEruhu/uf0c1gO
# 9+dBJiKQ1OWopSQOqEgOsLL0J123Ls4V8O3tzZwIDuuRofCB2+wKswad6CHoydJx
# 4p9rRXv6MLlnTqqGxemm/dPZqJ7+6L0poHoDKW+s7AgfVDshhj1RSbQfs8Ujh41F
# f1sdi3DzopVWtK4CE+8/UeLy5Cxlixke9SKhYQrFHrdsANARP81gxQjczKApMc1z
# v9qkrLtkM06VUyuvbPps7CHSHDpzx9mXcmkkPgLqLX9MfbCztzi44aVSaS9HYk5G
# y54dSKdY7VJEuGhG916G+GMDJyow4nhT9Gk6tWtk63TQN5nExVsoZMOmdw==
# =PFGL
# -----END PGP SIGNATURE-----
# gpg: Signature made Fri 11 Oct 2024 12:48:53 BST
# gpg:                using RSA key F13338574B662389866C7682BFFBD25F78C7AE83
# gpg:                issuer "pbonzini@redhat.com"
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>" [full]
# gpg:                 aka "Paolo Bonzini <pbonzini@redhat.com>" [full]
# Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4  E2F7 7E15 100C CD36 69B1
#      Subkey fingerprint: F133 3857 4B66 2389 866C  7682 BFFB D25F 78C7 AE83

* tag 'for-upstream' of https://gitlab.com/bonzini/qemu:
  docs: use consistent markup for footnotes
  docs: avoid footnotes consisting of just URLs
  docs: fix invalid footnote syntax
  gitlab-ci: add Rust-enabled CI job
  dockerfiles: add a Dockerfile using a nightly Rust toolchain
  meson: ensure -mcx16 is passed when detecting ATOMIC128
  meson: define qemu_isa_flags
  meson: fix machine option for x86_version
  rust: add PL011 device model
  rust: add utility procedural macro crate
  scripts/archive-source: find directory name for subprojects
  rust: add crate to expose bindings and interfaces
  meson.build: add HAVE_GLIB_WITH_ALIGNED_ALLOC flag
  .gitattributes: add Rust diff and merge attributes
  rust: add bindgen step as a meson dependency
  configure, meson: detect Rust toolchain
  build-sys: Add rust feature option
  Require meson version 1.5.0

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2024-10-11 18:19:37 +01:00
Paolo Bonzini
8105ca8512 dockerfiles: add a Dockerfile using a nightly Rust toolchain
This will be useful in order to test QEMU's Rust code with nightly
clippy.  It can also be used to check that the code builds, until the
minimum supported Rust version is lowered enough to allow enabling Rust
in other Dockerfiles too.

Use a separate container, instead of the Fedora one, to avoid that
CI breaks for everyone if for some reason the rustup build turns out
to be shaky.

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Acked-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-10-11 12:32:17 +02:00
Paolo Bonzini
07f0d32641 Require meson version 1.5.0
This is needed for Rust support.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Manos Pitsidianakis <manos.pitsidianakis@linaro.org>
Link: https://lore.kernel.org/r/74e1eb4b13717d061c5ad9c198bf56951fbfc14f.1727961605.git.manos.pitsidianakis@linaro.org
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-10-07 16:41:57 +02:00
Philippe Mathieu-Daudé
0522910d7d .gitlab-ci.d/cirrus: Add manual testing of macOS 15 (Sequoia)
Upgrade libvirt-ci so it covers macOS 15. Add a manual entry
(QEMU_JOB_OPTIONAL: 1) to test on Sequoia release. Refresh the
lci-tool generated files.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240917085058.1740-4-philmd@linaro.org>
[thuth: Fix image names in cirrus.yml and hash for tests/lcitool/libvirt-ci]
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-10-07 11:33:12 +02:00
Philippe Mathieu-Daudé
de11da6448 .gitlab-ci.d/cirrus: Drop support for macOS 13 (Ventura)
macOS 15 "Sequoia" was released on September 16, 2024 [1].

According to QEMU's support policy, we stop supporting
the previous major release two years after the the new
major release has been published. Time to remove support
for macOS 13 (Ventura, released on October 2022, [2]).

Promote the macOS 14 job, which was only built manually,
to be run by default.

[1] https://www.apple.com/newsroom/2024/09/macos-sequoia-is-available-today/
[2] https://www.apple.com/newsroom/2022/10/macos-ventura-is-now-available/

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240917085058.1740-3-philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-10-07 10:57:57 +02:00
Alex Bennée
c60473d292 testing: bump mips64el cross to bookworm and fix package list
The mips64el cross setup is very broken for bullseye which has now
entered LTS support so is unlikely to be fixed. While we still can't
build the container with all packages for bookworm due to a single
missing dependency that will hopefully get fixed in due course. For
the sake of keeping the CI green we disable the problematic packages
via the lcitool's mappings.yml file.

See also: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1081535

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
[thuth: Disable the problematic packages via lcitool's mappings.yml]
Message-ID: <20241002080333.127172-1-thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-10-02 10:21:16 +02:00
Peter Maydell
4b7ea33074 * Split --enable-sanitizers to --enable-{asan, ubsan}
* Build MSYS2 job using multiple CPUs
 * Fix "make distclean" wrt contrib/plugins/
 * Convert more Avocado tests to plain standalone functional tests
 * Fix bug that breaks "make check-functional" when tesseract is missing
 * Use builtin hashlib of Python in the functional tests
 * Update the FreeBSD CI jobs to 14.1
 -----BEGIN PGP SIGNATURE-----
 
 iQJFBAABCAAvFiEEJ7iIR+7gJQEY8+q5LtnXdP5wLbUFAmbhY4YRHHRodXRoQHJl
 ZGhhdC5jb20ACgkQLtnXdP5wLbU/aw/9HXl9H8BUDn8lnoEmxuuQSk8F19n/l5pt
 en3L8pMBt4dGFe/9KaGes2GFfid+cp2zlx+qQhA4HW35ntMJorF/qinOH/JGDtoM
 3O6RGZrQPn60zD9P2EbFVCrVYysVYCEu0U3Uglj6tf33bE0L7SJsQxqcbIciyIj5
 aq3Te0yMM2lqzCdMqNpWHGn3VMZRvbRaGBPDU4RLP8V2Bpz1iiRE+6HCH9Kg7HzS
 OmleeXtvcyInG+54onjfTcn4/XA27pl1UU04KFv5PrRPB3M2FspHn7oOT2yyQ+ls
 79mqIcd8PvycCT+3ch9p8KhVtbVBgZGmeemALLvk5FxysaWnl4KtSqmQNdqSvvpV
 waDDKlLaSnjEHDUse3bCJX0m4d7/vTBY5fOYxqZ4z5dl63csDtgPY4/VF4XR08sP
 tR1mW+2qEH9eygsxuKcBjx/j7Etpy+jL9pX2ii1V3ElhjjYuEnpEiURa+TaqPjpZ
 jmPtBEszzUdPbrD707tDkW3/ezT7VAnASQeYneJXB/JQG6K6Z//05iX6oCzCbRm3
 ceW/fem3UaeGYpzbMdoZToTuNlXEyS7NDcr39xJjH4LyRTPJAX4zeqUEdzces9g/
 u4Dw6rJ0Yhj4rscKxRvGl3/BH6CTI+8IAsbju2B/CnVLTqaABB0q/MDB90aB44xX
 bAVsl4P03Uk=
 =5TR0
 -----END PGP SIGNATURE-----

Merge tag 'pull-request-2024-09-11' of https://gitlab.com/thuth/qemu into staging

* Split --enable-sanitizers to --enable-{asan, ubsan}
* Build MSYS2 job using multiple CPUs
* Fix "make distclean" wrt contrib/plugins/
* Convert more Avocado tests to plain standalone functional tests
* Fix bug that breaks "make check-functional" when tesseract is missing
* Use builtin hashlib of Python in the functional tests
* Update the FreeBSD CI jobs to 14.1

# -----BEGIN PGP SIGNATURE-----
#
# iQJFBAABCAAvFiEEJ7iIR+7gJQEY8+q5LtnXdP5wLbUFAmbhY4YRHHRodXRoQHJl
# ZGhhdC5jb20ACgkQLtnXdP5wLbU/aw/9HXl9H8BUDn8lnoEmxuuQSk8F19n/l5pt
# en3L8pMBt4dGFe/9KaGes2GFfid+cp2zlx+qQhA4HW35ntMJorF/qinOH/JGDtoM
# 3O6RGZrQPn60zD9P2EbFVCrVYysVYCEu0U3Uglj6tf33bE0L7SJsQxqcbIciyIj5
# aq3Te0yMM2lqzCdMqNpWHGn3VMZRvbRaGBPDU4RLP8V2Bpz1iiRE+6HCH9Kg7HzS
# OmleeXtvcyInG+54onjfTcn4/XA27pl1UU04KFv5PrRPB3M2FspHn7oOT2yyQ+ls
# 79mqIcd8PvycCT+3ch9p8KhVtbVBgZGmeemALLvk5FxysaWnl4KtSqmQNdqSvvpV
# waDDKlLaSnjEHDUse3bCJX0m4d7/vTBY5fOYxqZ4z5dl63csDtgPY4/VF4XR08sP
# tR1mW+2qEH9eygsxuKcBjx/j7Etpy+jL9pX2ii1V3ElhjjYuEnpEiURa+TaqPjpZ
# jmPtBEszzUdPbrD707tDkW3/ezT7VAnASQeYneJXB/JQG6K6Z//05iX6oCzCbRm3
# ceW/fem3UaeGYpzbMdoZToTuNlXEyS7NDcr39xJjH4LyRTPJAX4zeqUEdzces9g/
# u4Dw6rJ0Yhj4rscKxRvGl3/BH6CTI+8IAsbju2B/CnVLTqaABB0q/MDB90aB44xX
# bAVsl4P03Uk=
# =5TR0
# -----END PGP SIGNATURE-----
# gpg: Signature made Wed 11 Sep 2024 10:31:50 BST
# gpg:                using RSA key 27B88847EEE0250118F3EAB92ED9D774FE702DB5
# gpg:                issuer "thuth@redhat.com"
# gpg: Good signature from "Thomas Huth <th.huth@gmx.de>" [full]
# gpg:                 aka "Thomas Huth <thuth@redhat.com>" [full]
# gpg:                 aka "Thomas Huth <huth@tuxfamily.org>" [full]
# gpg:                 aka "Thomas Huth <th.huth@posteo.de>" [unknown]
# Primary key fingerprint: 27B8 8847 EEE0 2501 18F3  EAB9 2ED9 D774 FE70 2DB5

* tag 'pull-request-2024-09-11' of https://gitlab.com/thuth/qemu: (24 commits)
  Update FreeBSD CI jobs FreeBSD 14.1
  tests/functional/qemu_test: Use Python hashlib instead of external programs
  tests/functional: Fix bad usage of has_cmd
  tests/functional: Convert the multiprocess avocado test into a standalone test
  tests/functional: Convert the or1k-sim Avocado test
  tests/functional: Convert the m68k MCF5208EVB Avocado test
  tests/functional: Convert the Alpha Clipper Avocado test
  tests/functional: Convert Aarch64 Raspi4 avocado tests
  tests/functional: Convert Aarch64 Raspi3 avocado tests
  tests/functional: Convert ARM Raspi2 avocado tests
  tests/functional: Convert mips32eb 4Kc Malta avocado tests
  tests/functional: Convert nanomips Malta avocado tests
  tests/functional: Convert mips32el Malta YAMON avocado test
  tests/functional: Convert mips64el 5KEc Malta avocado tests
  tests/functional: Convert mips64el I6400 Malta avocado tests
  tests/functional: Convert mips64el Fuloong2e avocado test (2/2)
  tests/functional: Convert the m68k Q800 Avocado test into a functional test
  tests/functional: Add the LinuxKernelTest for testing the Linux boot process
  MAINTAINERS: Remove myself from the Meson section
  MAINTAINERS: Remove myself as reviewer
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2024-09-11 19:28:23 +01:00
Thomas Huth
b4358ed4fd Update FreeBSD CI jobs FreeBSD 14.1
The current FreeBSD CI jobs are failing installation since the
"opencv" package is now missing there. Updating to 14.1 fixes
the issue.

Message-Id: <20240911090149.286257-1-thuth@redhat.com>
Reviewed-by: Li-Wen Hsu <lwhsu@FreeBSD.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-09-11 11:26:41 +02:00
Alex Bennée
19d2111059 tests/docker: update debian i686 and mipsel images to bookworm
Whatever issues there were which stopped these being updates when the
rest were have now been resolved. However mips64el continues to be
broken so don't update it here.

Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20240910173900.4154726-3-alex.bennee@linaro.org>
2024-09-10 23:32:58 +01:00
Alex Bennée
d0068b746a tests/docker: remove debian-armel-cross
As debian-11 transitions to LTS we are starting to have problems
building the image. While we could update to a later Debian building a
32 bit QEMU without modern floating point is niche host amongst the
few remaining 32 bit hosts we regularly build for. For now we still
have armhf-debian-cross-container which is currently built from the
more recent debian-12.

Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20240910173900.4154726-2-alex.bennee@linaro.org>
2024-09-10 23:32:51 +01:00
Daniel P. Berrangé
6d67a65f10 ci: add gtk-vnc to the deps
The gtk-vnc package is used by the vnc-display-test qtest
program. Technically only gvnc is needed, but since we
already pull in the gtk3 dep, it is harmless to depend
on gtk-vnc.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-ID: <20240718094159.902024-2-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-08-16 09:16:03 +02:00
Alex Bennée
d84ed5d2d4 testing: bump to latest libvirt-ci
This brings in the latest python mappings for the BSD updates.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20240718094523.1198645-2-alex.bennee@linaro.org>
2024-07-22 09:06:17 +01:00
Alex Bennée
515632d520 tests/lcitool: fix debian-i686-cross toolchain prefix
I guess we never noticed and tried to build with this cross image. Fix
the toolchain prefix so we actually build 32 bit images.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20240705084047.857176-2-alex.bennee@linaro.org>
2024-07-05 12:33:32 +01:00
zhenwei pi
c0cb5ccc35 test: Remove libibumad dependence
Remove libibumad dependence from the test environment.

Suggested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: zhenwei pi <pizhenwei@bytedance.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240611105427.61395-3-pizhenwei@bytedance.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-06-12 10:21:49 +02:00
Alex Bennée
1417704564 tests/lcitool: generate package lists for ansible
With the new ability to output YAML we can build the package list for
our ansible setup scripts. We will integrate them in the next commit.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20240603175328.3823123-12-alex.bennee@linaro.org>
2024-06-06 10:26:13 +01:00
Thomas Huth
61d1e3cbde tests/lcitool: Install mingw-w64-tools for the Windows cross-builds
Beside g++ we also need the mingw-w64-tools for properly building
the code in qga/vss-win32/ , so let's install that package now, too.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20240601070543.37786-5-thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20240603175328.3823123-11-alex.bennee@linaro.org>
2024-06-06 10:26:13 +01:00
Thomas Huth
06f3330bb0 tests/lcitool: Bump to latest libvirt-ci and update Fedora and Alpine version
Update to the latest version of lcitool. It dropped support for Fedora 38
and Alpine 3.18, so we have to update these to newer versions here, too.

Python 3.12 dropped the "imp" module which we still need for running
Avocado. Fortunately Fedora 40 still ships with a work-around package
that we can use until somebody updates our Avocado to a newer version.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20240601070543.37786-3-thuth@redhat.com>
[AJB: regen on rebase]
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20240603175328.3823123-10-alex.bennee@linaro.org>
2024-06-06 10:26:13 +01:00
Thomas Huth
8e3034914a tests/lcitool: Delete obsolete centos-stream-8.yml file
We've missed to delete this file when removing support for CentOS 8.
Since the current upstream version of the lcitool removed support
for CentOS 8 now, too, we have to remove the file before updating.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20240601070543.37786-2-thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20240603175328.3823123-8-alex.bennee@linaro.org>
2024-06-06 10:26:13 +01:00
Thomas Huth
d2f213cc06 tests/lcitool/projects/qemu.yml: Sort entries alphabetically again
Let's try to keep the entries in alphabetical order here!

Message-ID: <20240516084059.511463-5-thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-05-17 11:16:37 +02:00
Thomas Huth
fd77b25bbd tests/lcitool: Remove g++ from the containers (except for the MinGW one)
We don't need C++ for the normal QEMU builds anymore, so installing
g++ in each and every container seems to be a waste of time and disk
space. The only container that still needs it is the Fedora MinGW
container that builds the only remaining C++ code in ./qga/vss-win32/
and we can install it there with an extra project yml file instead.

Message-ID: <20240516084059.511463-4-thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-05-17 11:16:28 +02:00
Philippe Mathieu-Daudé
9ebe09e633 tests/lcitool: Remove 'xfsprogs' from QEMU
QEMU's commit a5730b8bd3 ("block/file-posix: Simplify the
XFS_IOC_DIOINFO handling") removed the need for the 'xfsprogs'
package.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
[thuth: Adjusted the patch from the lcitools repo to QEMU's repo]
Message-ID: <20240516084059.511463-3-thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-05-17 11:16:13 +02:00
Thomas Huth
4d8698e692 tests/lcitool/refresh: Treat the output of lcitool as text, not as bytes
In case lcitool fails (e.g. with a python backtrace), this makes
the output  of lcitool much more readable.

Suggested-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20240516084059.511463-2-thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-05-17 11:15:58 +02:00
Thomas Huth
641b1efe01 tests: Update our CI to use CentOS Stream 9 instead of 8
RHEL 9 (and thus also the derivatives) have been available since two
years now, so according to QEMU's support policy, we can drop the active
support for the previous major version 8 now.

Another reason for doing this is that Centos Stream 8 will go EOL soon:

https://blog.centos.org/2023/04/end-dates-are-coming-for-centos-stream-8-and-centos-linux-7/

  "After May 31, 2024, CentOS Stream 8 will be archived
   and no further updates will be provided."

Thus upgrade our CentOS Stream container to major version 9 now.

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20240418101056.302103-5-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-04-25 07:03:04 +02:00
Thomas Huth
c723d9d16f tests/lcitool/libvirt-ci: Update to the latest master branch
We need the latest fixes for the lcitool to be able to properly
update our CentOS docker file to CentOS Stream 9.

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-ID: <20240418101056.302103-3-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-04-25 07:03:04 +02:00
Thomas Huth
45070eb716 tests: Remove Ubuntu 20.04 container
Since Ubuntu 22.04 has now been available for more than two years, we
can stop actively supporting the previous LTS version of Ubuntu now.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240418101056.302103-2-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-04-25 07:03:04 +02:00
Peter Maydell
8b47ec7abe .gitlab-ci.d: Drop cross-win32-system job
We don't support 32-bit Windows any more, so we don't need to defend it
with this CI job.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: "Daniel P. Berrangé" <berrange@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-ID: <20240222130920.362517-3-peter.maydell@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2024-02-23 08:13:52 +01:00
Fabiano Rosas
7485508341 tests/docker: Add sqlite3 module to openSUSE Leap container
Avocado needs sqlite3:

  Failed to load plugin from module "avocado.plugins.journal":
  ImportError("Module 'sqlite3' is not installed.
  Use: sudo zypper install python311 to install it")

>From 'zypper info python311':
  "This package supplies rich command line features provided by
  readline, and sqlite3 support for the interpreter core, thus forming
  a so called "extended" runtime."

Include the appropriate package in the lcitool mappings which will
guarantee the dockerfile gets properly updated when lcitool is
run. Also include the updated dockerfile.

Signed-off-by: Fabiano Rosas <farosas@suse.de>
Suggested-by: Andrea Bolognani <abologna@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20240117164227.32143-1-farosas@suse.de>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20240207163812.3231697-2-alex.bennee@linaro.org>
2024-02-09 17:48:25 +00:00
Philippe Mathieu-Daudé
541069e653 .gitlab-ci.d/cirrus: Add manual testing of macOS 14 (Sonoma)
Upgrade libvirt-ci so it covers macOS 14. Add a manual entry
(QEMU_JOB_OPTIONAL: 1) to test on Sonoma release. Refresh the
lci-tool generated files.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20231109160504.93677-3-philmd@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-11-24 16:21:55 +01:00
Alex Bennée
8e721c3277 tests/docker: merge debian-native with debian-amd64
debian-native isn't really needed and suffers from the problem of
tracking a distros dependencies rather than the projects. With a
little surgery we can make the debian-amd64 container architecture
neutral and allow people to use it to build a native QEMU.

Rename it so it follows the same non-arch pattern of the other distro
containers.

Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Anders Roxell <anders.roxell@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231120150833.2552739-4-alex.bennee@linaro.org>
2023-11-23 14:10:04 +00:00
Philippe Mathieu-Daudé
7528ef7321 .gitlab-ci.d/cirrus: Upgrade macOS to 13 (Ventura)
macOS 14 "Sonoma" was released on September 2023 [1].

According to QEMU's support policy, we stop supporting the
previous major release two years after the the new major
release has been published. Replace the macOS 12 (Monterey)
testing by macOS 13 (Ventura, released on October 2022, [2]).

Refresh the generated files by running:

  $ make lcitool-refresh

[1] https://www.apple.com/newsroom/2023/09/macos-sonoma-is-available-today/
[2] https://www.apple.com/newsroom/2022/10/macos-ventura-is-now-available/

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20231108162022.76189-1-philmd@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231120150833.2552739-3-alex.bennee@linaro.org>
2023-11-23 14:10:02 +00:00
Daniel P. Berrangé
aa5730b07e tests/docker: replace fedora-i386 with debian-i686
Fedora is gradually killing off i386 packages in its repos, via a
death-by-1000-cuts process. Thus Debian looks like a better long
term bet for i686 build testing. It has the added advantage that
we can generate it via lcitool too.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20231107164109.1449014-1-berrange@redhat.com>
[AJB: tweak commit msg, set correct prefix]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231120150833.2552739-2-alex.bennee@linaro.org>
2023-11-23 11:55:32 +00:00
Alex Bennée
3e3df0d84f tests/lcitool: add swtpm to the package list
We need this to test some TPM stuff.

Reviewed-by: "Daniel P. Berrangé" <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231009164104.369749-4-alex.bennee@linaro.org>
2023-10-11 08:46:23 +01:00
Alex Bennée
ca9cde57ab tests: update most Debian images to Bookworm
Bookworm has been out a while now. Time to update our containers to
the current stable. This requires the latest lcitool repo so update
the sub-module too.

For some reason the MIPs containers won't build so skip those for now.
We also have to skip the armel builds due to a stuck libc update.

Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230914155422.426639-2-alex.bennee@linaro.org>
2023-09-20 15:06:26 +01:00
Ilya Maximets
cb039ef3d9 net: add initial support for AF_XDP network backend
AF_XDP is a network socket family that allows communication directly
with the network device driver in the kernel, bypassing most or all
of the kernel networking stack.  In the essence, the technology is
pretty similar to netmap.  But, unlike netmap, AF_XDP is Linux-native
and works with any network interfaces without driver modifications.
Unlike vhost-based backends (kernel, user, vdpa), AF_XDP doesn't
require access to character devices or unix sockets.  Only access to
the network interface itself is necessary.

This patch implements a network backend that communicates with the
kernel by creating an AF_XDP socket.  A chunk of userspace memory
is shared between QEMU and the host kernel.  4 ring buffers (Tx, Rx,
Fill and Completion) are placed in that memory along with a pool of
memory buffers for the packet data.  Data transmission is done by
allocating one of the buffers, copying packet data into it and
placing the pointer into Tx ring.  After transmission, device will
return the buffer via Completion ring.  On Rx, device will take
a buffer form a pre-populated Fill ring, write the packet data into
it and place the buffer into Rx ring.

AF_XDP network backend takes on the communication with the host
kernel and the network interface and forwards packets to/from the
peer device in QEMU.

Usage example:

  -device virtio-net-pci,netdev=guest1,mac=00:16:35:AF:AA:5C
  -netdev af-xdp,ifname=ens6f1np1,id=guest1,mode=native,queues=1

XDP program bridges the socket with a network interface.  It can be
attached to the interface in 2 different modes:

1. skb - this mode should work for any interface and doesn't require
         driver support.  With a caveat of lower performance.

2. native - this does require support from the driver and allows to
            bypass skb allocation in the kernel and potentially use
            zero-copy while getting packets in/out userspace.

By default, QEMU will try to use native mode and fall back to skb.
Mode can be forced via 'mode' option.  To force 'copy' even in native
mode, use 'force-copy=on' option.  This might be useful if there is
some issue with the driver.

Option 'queues=N' allows to specify how many device queues should
be open.  Note that all the queues that are not open are still
functional and can receive traffic, but it will not be delivered to
QEMU.  So, the number of device queues should generally match the
QEMU configuration, unless the device is shared with something
else and the traffic re-direction to appropriate queues is correctly
configured on a device level (e.g. with ethtool -N).
'start-queue=M' option can be used to specify from which queue id
QEMU should start configuring 'N' queues.  It might also be necessary
to use this option with certain NICs, e.g. MLX5 NICs.  See the docs
for examples.

In a general case QEMU will need CAP_NET_ADMIN and CAP_SYS_ADMIN
or CAP_BPF capabilities in order to load default XSK/XDP programs to
the network interface and configure BPF maps.  It is possible, however,
to run with no capabilities.  For that to work, an external process
with enough capabilities will need to pre-load default XSK program,
create AF_XDP sockets and pass their file descriptors to QEMU process
on startup via 'sock-fds' option.  Network backend will need to be
configured with 'inhibit=on' to avoid loading of the program.
QEMU will need 32 MB of locked memory (RLIMIT_MEMLOCK) per queue
or CAP_IPC_LOCK.

There are few performance challenges with the current network backends.

First is that they do not support IO threads.  This means that data
path is handled by the main thread in QEMU and may slow down other
work or may be slowed down by some other work.  This also means that
taking advantage of multi-queue is generally not possible today.

Another thing is that data path is going through the device emulation
code, which is not really optimized for performance.  The fastest
"frontend" device is virtio-net.  But it's not optimized for heavy
traffic either, because it expects such use-cases to be handled via
some implementation of vhost (user, kernel, vdpa).  In practice, we
have virtio notifications and rcu lock/unlock on a per-packet basis
and not very efficient accesses to the guest memory.  Communication
channels between backend and frontend devices do not allow passing
more than one packet at a time as well.

Some of these challenges can be avoided in the future by adding better
batching into device emulation or by implementing vhost-af-xdp variant.

There are also a few kernel limitations.  AF_XDP sockets do not
support any kinds of checksum or segmentation offloading.  Buffers
are limited to a page size (4K), i.e. MTU is limited.  Multi-buffer
support implementation for AF_XDP is in progress, but not ready yet.
Also, transmission in all non-zero-copy modes is synchronous, i.e.
done in a syscall.  That doesn't allow high packet rates on virtual
interfaces.

However, keeping in mind all of these challenges, current implementation
of the AF_XDP backend shows a decent performance while running on top
of a physical NIC with zero-copy support.

Test setup:

2 VMs running on 2 physical hosts connected via ConnectX6-Dx card.
Network backend is configured to open the NIC directly in native mode.
The driver supports zero-copy.  NIC is configured to use 1 queue.

Inside a VM - iperf3 for basic TCP performance testing and dpdk-testpmd
for PPS testing.

iperf3 result:
 TCP stream      : 19.1 Gbps

dpdk-testpmd (single queue, single CPU core, 64 B packets) results:
 Tx only         : 3.4 Mpps
 Rx only         : 2.0 Mpps
 L2 FWD Loopback : 1.5 Mpps

In skb mode the same setup shows much lower performance, similar to
the setup where pair of physical NICs is replaced with veth pair:

iperf3 result:
  TCP stream      : 9 Gbps

dpdk-testpmd (single queue, single CPU core, 64 B packets) results:
  Tx only         : 1.2 Mpps
  Rx only         : 1.0 Mpps
  L2 FWD Loopback : 0.7 Mpps

Results in skb mode or over the veth are close to results of a tap
backend with vhost=on and disabled segmentation offloading bridged
with a NIC.

Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> (docker/lcitool)
Signed-off-by: Jason Wang <jasowang@redhat.com>
2023-09-18 14:36:13 +08:00
Ilya Maximets
a6f376e9ba tests: bump libvirt-ci for libasan and libxdp
This pulls in the fixes for libasan version as well as support for
libxdp that will be used for af-xdp netdev in the next commits.

Signed-off-by: Ilya Maximets <i.maximets@ovn.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Jason Wang <jasowang@redhat.com>
2023-09-18 14:36:13 +08:00
Paolo Bonzini
7ace219303 tests/docker: add python3-tomli dependency to containers
Instead of having CI pick tomli from the vendored wheel at configure
time, place it in the containers.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-08-28 09:55:48 +02:00
Paolo Bonzini
dcb8541b0b lcitool: bump libvirt-ci submodule and regenerate
This brings in a newer version of the pipewire mapping, so rename it.

Python 3.9 and 3.10 do not seem to work in OpenSUSE LEAP 15.5 (weird,
because 3.9 persisted from 15.3 to 15.4) so bump the Python runtime
version to 3.11.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-08-28 09:55:47 +02:00
Philippe Mathieu-Daudé
03e596de17 tests/lcitool: Generate distribution packages list in JSON format
Add the generate_pkglist() helper to generate a list of packages
required by a distribution to build QEMU.

Since we can not add a "THIS FILE WAS AUTO-GENERATED" comment in
JSON, create the files under tests/vm/generated/ sub-directory;
add a README mentioning the files are generated.

Suggested-by: Erik Skultety <eskultet@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Warner Losh <imp@bsdimp.com>
Message-Id: <20230711144922.67491-2-philmd@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-07-18 09:36:28 +02:00
Marc-André Lureau
62259d816c tests/lcitool: add pipewire
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20230506163735.3481387-3-marcandre.lureau@redhat.com>
2023-07-17 15:22:56 +04:00
Marc-André Lureau
6200e0ff5f libvirt-ci: update submodule to cover pipewire
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20230506163735.3481387-2-marcandre.lureau@redhat.com>
2023-07-17 15:22:56 +04:00
Alex Bennée
0101dd71b0 tests/docker: convert riscv64-cross to lcitool
We still need to base this on Debian Sid until riscv64 is promoted to
a release architecture (or another distro provides a full cross
compile target). We use the new qemu-minimal project description to
avoid bringing in all the extra dependencies because every extra
package is another chance for sid to fail.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230630180423.558337-16-alex.bennee@linaro.org>
2023-07-03 12:51:53 +01:00
Alex Bennée
b911b9001e tests/lcitool: introduce qemu-minimal
This is a very bare bones set of dependencies for a minimal build of
QEMU. This will be useful for minimal cross-compile sanity check based
on things like Debian Sid where stuff isn't always in sync.

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230630180423.558337-15-alex.bennee@linaro.org>
2023-07-03 12:51:50 +01:00
Alex Bennée
690be80412 tests/lcitool: add an explicit gcc-native package
We need a native compiler to build the hexagon codegen tools. In our
current images we already have a gcc as a side effect of a broken
dependency between gcovr and lcov but this will be fixed when we move
to bookworm. See
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=987818 for details.

Update the packages while we are at it.

Acked-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230630180423.558337-14-alex.bennee@linaro.org>
2023-07-03 12:51:47 +01:00
Erik Skultety
76eb439c0f tests/lcitool: Bump fedora container versions
Fedora 37 -> 38

Signed-off-by: Erik Skultety <eskultet@redhat.com>
Acked-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <c9b00e573a7a80fc6ce5c68595382f5c916a9195.1685528076.git.eskultet@redhat.com>
[AJB: Dropped alpine (in prev commit), reflow commit msg]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230630180423.558337-13-alex.bennee@linaro.org>
2023-07-03 12:51:40 +01:00
Alex Bennée
c7374742d8 tests/lcitool: update to latest version
We need this for the riscv64 and gcc-native mappings. As the older
alpine release has been dropped from the mappings we also need to bump
the version of alpine we use.

Acked-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230630180423.558337-12-alex.bennee@linaro.org>
2023-07-03 12:51:37 +01:00