Flatten into a single dockerfile and update to match the rest of the
test cross compile dockerfiles.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220914155950.804707-23-alex.bennee@linaro.org>
It's becoming harder to maintain a cross-compiler to test this host
architecture as the old stable Debian 10 ("Buster") moved into LTS
which supports fewer architectures. For now:
- mark it's deprecation in the docs
- downgrade the containers to build TCG tests only
- drop the cross builds from our CI
Users with an appropriate toolchain and user-space can still take
their chances building it.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Huacai Chen <chenhuacai@kernel.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220914155950.804707-22-alex.bennee@linaro.org>
The custom runner is now using 22.04 so we can drop our hacks to deal
with broken libssh and glusterfs. The provisioning scripts will be
updated in a separate commit.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20220914155950.804707-21-alex.bennee@linaro.org>
The project has reached the magic size at which we see
/usr/aarch64-linux-gnu/lib/libc.a(init-first.o): in function `__libc_init_first':
(.text+0x10): relocation truncated to fit: R_AARCH64_LD64_GOTPAGE_LO15 against \
symbol `__environ' defined in .bss section in /usr/aarch64-linux-gnu/lib/libc.a(environ.o)
/usr/bin/ld: (.text+0x10): warning: too many GOT entries for -fpic, please recompile with -fPIC
The bug has been reported upstream, but in the meantime there is
nothing we can do except build a non-pie executable.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220823210329.1969895-1-richard.henderson@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20220914155950.804707-20-alex.bennee@linaro.org>
We missed removing this dependency when we flattened the build.
Fixes: 39ce923732 (gitlab: enable a very minimal build with the tricore container)
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20220914155950.804707-16-alex.bennee@linaro.org>
Flatten into a single dockerfile. We really don't need the rest of the
stuff from the QEMU base image just to compile test images.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20220914155950.804707-15-alex.bennee@linaro.org>
Update to the latest stable Debian. While we are at it flatten into a
single dockerfile. We really don't need the rest of the stuff from
the QEMU base image just to compile test images.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20220914155950.804707-14-alex.bennee@linaro.org>
Update to the latest stable Debian. While we are at it flatten into a
single dockerfile. We really don't need the rest of the stuff from
the QEMU base image just to compile test images.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20220914155950.804707-13-alex.bennee@linaro.org>
Update to the latest stable Debian. While we are at it flatten into a
single dockerfile. We really don't need the rest of the stuff from
the QEMU base image just to compile test images.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20220914155950.804707-12-alex.bennee@linaro.org>
Update to the latest stable Debian. While we are at it flatten into a
single dockerfile. We really don't need the rest of the stuff from
the QEMU base image just to compile test images.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20220914155950.804707-11-alex.bennee@linaro.org>
Update to the latest stable Debian. While we are at it flatten into a
single dockerfile. We really don't need the rest of the stuff from
the QEMU base image just to compile test images.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20220914155950.804707-10-alex.bennee@linaro.org>
Update to the latest stable Debian. While we are at it flatten into a
single dockerfile. We really don't need the rest of the stuff from
the QEMU base image just to compile test images.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20220914155950.804707-9-alex.bennee@linaro.org>
We already limit the scope of the cross system build to reduce the
cross build times. With the recent addition of more targets we are
also running into timeout issues for some of the cross user builds.
I've selected a few of those linux-user targets which are less likely
to be in common use as distros don't have pre-built rootfs for them.
I've also added the same CROSS_SKIP_TARGETS variable as is
occasionally used to further limit cross system builds.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220914155950.804707-2-alex.bennee@linaro.org>
The FreeBSD 13 job in our CI started failing since the python port
stopped working after 13.1 has been released. Thus update our CI
job to FreeBSD 13.1 to get it working again.
Suggested-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220920102041.45067-1-thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
The sed processing of build/config-host.mak seems to be no longer
needed, and there is no such in the 32-bit build too. Drop it.
Signed-off-by: Bin Meng <bin.meng@windriver.com>
Message-Id: <20220908132817.1831008-5-bmeng.cn@gmail.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
The Windows jobs are currently aborting at weird places - and
there's the suspicion that it's due to memory constraints in
the Windows containers. Let's switch to single-threaded compilation
to decrease the pressure on the memory load, and to make the
job more deterministic for further investigations.
Message-Id: <20220825193323.104768-1-thuth@redhat.com>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
Message-Id: <20220728200422.1502-1-luoyonggang@gmail.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
The following error message was seen during the configure:
"ln: failed to create symbolic link
'x86_64-softmmu/qemu-system-x86_64.exe': No such file or directory"
By default the MSYS environment variable is not defined, so the runtime
behavior of winsymlinks is: if <target> does not exist, 'ln -s' fails.
At the configure phase, the qemu-system-x86_64.exe has not been built
so creation of the symbolic link fails hence the error message.
Set winsymlinks to 'native' whose behavior is most similar to the
behavior of 'ln -s' on *nix, that is:
a) if native symlinks are enabled, and whether <target> exists
or not, creates <destination> as a native Windows symlink;
b) else if native symlinks are not enabled, and whether <target>
exists or not, 'ln -s' creates as a Windows shortcut file.
Signed-off-by: Bin Meng <bin.meng@windriver.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220725123000.807608-1-bmeng.cn@gmail.com>
Message-Id: <20220725140520.515340-6-alex.bennee@linaro.org>
Since we express dependencies via a 'needs' clause, we don't need to
split container builds into separate stages. GitLab happily lets jobs
depend on other jobs in the same stage and will run them when possible.
Acked-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220722130431.2319019-4-berrange@redhat.com>
[AJB: fix typo]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20220725140520.515340-4-alex.bennee@linaro.org>
When tests fail meson just displays a summary and tells you to look at
the testlog.txt file for details. The native jobs on shared runners
publish testlog.txt as an artifact. For the Cirrus jobs and custom
runner jobs this is not currently possible. The best we can do is cat
the log contents on failure, to give maintainers a fighting chance
of diagnosing the problem.
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220722130431.2319019-3-berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220725140520.515340-3-alex.bennee@linaro.org>
Notable changes:
- libvirt-ci source tree was re-arranged, so the script we
run now lives in a bin/ sub-dir
- opensuse 15.2 is replaced by opensuse 15.3
- libslirp is temporarily dropped on opensuse as the
libslirp-version.h is broken
https://bugzilla.opensuse.org/show_bug.cgi?id=1201551
- The incorrectly named python3-virtualenv module was
changed to python3-venv, but most distros don't need
any package as 'venv' is a standard part of python
- glibc-static was renamed to libc-static, to reflect
fact that it isn't going to be glibc on all distros
- The cmocka/json-c deps that were manually added to
the centos dockerfile and are now consistently added
to all targets
Acked-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220722130431.2319019-2-berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Acked-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220725140520.515340-2-alex.bennee@linaro.org>
This reverts commit 309df6acb2.
With Ilya's 'multifd: Copy pages before compressing them with zlib'
in the latest migration series, this shouldn't be a problem any more.
Suggested-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
If we don't need to emulate any target, we certainly don't need TCG.
This should also help to compile again with
".../configure --enable-tools --disable-system --disable-user"
on systems that do not have a TCG backend.
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
[thuth: Re-arranged the code, remove check-softfloat from buildtest.yml]
Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220706153816.768143-1-thuth@redhat.com>
FreeBSD has stopped shipping python 3.8, causing our cirrus
builds to fail immediately. Upstream lcitool has an update
to address this, but has also reorganized its source tree so
additional changes are required for 'make lcitool-update'.
In the meantime, fix the build.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Recent runs have been taking just over the 60m default.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220606182436.410053-1-richard.henderson@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
To preserve contributor CI credits we don't want jobs to run by default
unless the QEMU_CI variable is set. For most jobs we can achieve this
using the base template, but the edk2/opensbi jobs are a little special
as they have some complex conditions we can't easily model in the base
template.
We duplicate existing rules and put them under control of QEMU_CI
variable, such that QEMU_CI=1 creates manual jobs and QEMU_CI=2
immediately runs jobs.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220629170638.520630-4-berrange@redhat.com>
[thuth: Fixed "on_success" <-> "manual" copy-n-paste bug]
Signed-off-by: Thomas Huth <thuth@redhat.com>
Get rid of comments stating the obvious and re-arrange remaining
comments. The opensbi split of rules for file matches is also
merged into one rule.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220629170638.520630-3-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
The edk2/opensbi gitlab CI config was using single space indents
which is not consistent with the rest of the gitlab CI config
files.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220629170638.520630-2-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
add the libvfio-user library as a submodule. build it as a meson
subproject.
libvfio-user is distributed with BSD 3-Clause license and
json-c with MIT (Expat) license
Signed-off-by: Elena Ufimtseva <elena.ufimtseva@oracle.com>
Signed-off-by: John G Johnson <john.g.johnson@oracle.com>
Signed-off-by: Jagannathan Raman <jag.raman@oracle.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: c2adec87958b081d1dc8775d4aa05c897912f025.1655151679.git.jag.raman@oracle.com
[Changed submodule URL to QEMU's libvfio-user mirror on GitLab. The QEMU
project mirrors its dependencies so that it can provide full source code
even in the event that its dependencies become unavailable. Note that
the mirror repo is manually updated, so please contact me to make newer
libvfio-user commits available. If I become a bottleneck we can set up a
cronjob.
Updated scripts/meson-buildoptions.sh to match the meson_options.txt
change. Failure to do so can result in scripts/meson-buildoptions.sh
being modified by the build system later on and you end up with a dirty
working tree.
--Stefan]
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Running on all 80 cores of our aarch64 runner does occasionally
trigger a race condition which fails the build. However the CI system
is not the time and place to play with much heisenbugs so turn down
the nproc to "only" use 40 cores in the build.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Tested-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20220613171258.1905715-8-alex.bennee@linaro.org>
The GitLab variable comparisons don't have shell like semantics where
an unset variable compares equal to empty string. We need to explicitly
test against 'null' to detect an unset variable.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Tested-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20220608160651.248781-1-berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220613171258.1905715-7-alex.bennee@linaro.org>
The job definitions recently got a second "variables:" section by
accident and thus are failing now if one tries to run them. Merge
the two sections into one again to fix the issue.
And while we're at it, bump the timeout here (70 minutes are currently
not enough for the aarch64 job). The jobs are marked as manual anyway,
so if the user starts them, they want to see their result for sure and
then it's annoying if the job timeouts too early.
Fixes: e312d1fdbb ("gitlab: convert build/container jobs to .base_job_template")
Signed-off-by: Thomas Huth <thuth@redhat.com>
Acked-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220603124809.70794-1-thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220613171258.1905715-4-alex.bennee@linaro.org>
Remove the sys.path hacking from device-crash-test, and add in a little
user-friendly message for anyone who was used to running this script
directly from the source tree.
Modify the GitLab job recipes to create the tests/venv first, then run
device-crash-test from that venv.
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20220526000921.1581503-10-jsnow@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
To preserve CI shared runner credits we don't want to run
pipelines on every push.
This sets up the config so that pipelines are never created
for contributors by default. To override this the QEMU_CI
variable can be set to a non-zero value. If set to 1, the
pipeline will be created but all jobs will remain manually
started. The contributor can selectively run jobs that they
care about. If set to 2, the pipeline will be created and
all jobs will immediately start.
This behavior can be controlled using push variables
git push -o ci.variable=QEMU_CI=1
To make this more convenient define an alias
git config --local alias.push-ci "push -o ci.variable=QEMU_CI=1"
git config --local alias.push-ci-now "push -o ci.variable=QEMU_CI=2"
Which lets you run
git push-ci
to create the pipeline, or
git push-ci-now
to create and run the pipeline
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220526110705.59952-6-berrange@redhat.com>
[AJB: fix typo, replicate alias tips in ci.rst]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20220527153603.887929-33-alex.bennee@linaro.org>
This converts the main build and container jobs to use the
base job rules, defining the following new variables
- QEMU_JOB_SKIPPED - jobs that are known to be currently
broken and should not be run. Can still be manually
launched if desired.
- QEMU_JOB_AVOCADO - jobs that run the Avocado integration
test harness.
- QEMU_JOB_PUBLISH - jobs that publish content after the
branch is merged upstream
As build-tools-and-docs runs on master we declare the requirement of
building amd64-debian-container optional as it should already exits
once we merge.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220526110705.59952-5-berrange@redhat.com>
[AJB: fix upstream typo, mention optional container req]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20220527153603.887929-32-alex.bennee@linaro.org>
This folds the static checks into using the base job
template rules, introducing one new variable
- QEMU_JOB_ONLY_FORKS - a job that should never run
on an upstream pipeline. The information it reports
is only applicable to contributors in a pre-submission
scenario, not time of merge.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220526110705.59952-4-berrange@redhat.com>
[AJB: fix typo]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20220527153603.887929-31-alex.bennee@linaro.org>
This folds the Cirrus job rules into the base job
template, introducing two new variables
- QEMU_JOB_CIRRUS - identifies the job as making
use of Cirrus CI via cirrus-run
- QEMU_JOB_OPTIONAL - identifies the job as one
that is not run by default, primarily due to
resource constraints. It can be manually invoked
by users if they wish to validate that scenario.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220526110705.59952-3-berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20220527153603.887929-30-alex.bennee@linaro.org>
Currently job rules are spread across the various templates
and jobs, making it hard to understand exactly what runs in
what scenario. This leads to inconsistency in the rules and
increased maint burden.
The intent is that we introduce a common '.base_job_template'
which will have a general purpose 'rules:' block. No other
template or job should define 'rules:', but instead they must
rely on the inherited rules. To allow behaviour to be tweaked,
rules will be influenced by a number of variables with the
naming scheme 'QEMU_JOB_nnnn'.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220526110705.59952-2-berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20220527153603.887929-29-alex.bennee@linaro.org>
The one minor wrinkle we need to account for is the netmap support
still requires building from source. We also include cscope and GNU
global as they are used in one of the builds.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Cc: Philippe Mathieu-Daudé <f4bug@amsat.org>
Cc: Luigi Rizzo <rizzo@iet.unipi.it>
Cc: Giuseppe Lettieri <g.lettieri@iet.unipi.it>
Cc: Vincenzo Maffione <v.maffione@gmail.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220527153603.887929-12-alex.bennee@linaro.org>
Use lcitool to update debian-ppc64el-cross to a Debian 11 based system.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220527153603.887929-11-alex.bennee@linaro.org>
Use lcitool to update debian-mips64el-cross to a Debian 11 based system.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220527153603.887929-10-alex.bennee@linaro.org>
Use lcitool to update debian-mipsel-cross to a Debian 11 based system.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220527153603.887929-9-alex.bennee@linaro.org>
Use lcitool to update debian-armel-cross to a Debian 11 based system.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220527153603.887929-8-alex.bennee@linaro.org>
Use lcitool to update debian-armhf-cross to a Debian 11 based system.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220527153603.887929-7-alex.bennee@linaro.org>
This allows the gitlab UI to show the test results in different ways,
see doc:
https://docs.gitlab.com/ee/ci/unit_test_reports.html#how-it-works
Previous we only reports avocado test results (.avocado_test_job_template),
with this change, the qemu/meson tests are also covered.
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20220525173411.612224-1-marcandre.lureau@redhat.com>
[AJB: expand the commit description]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20220527153603.887929-4-alex.bennee@linaro.org>
The cross-i386-tci job uses the fedora-i386-cross image, so we should make sure
that the corresponding job that builds it (the i386-fedora-cross-container job)
has finished before we start the TCI job.
Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220524092600.89997-1-thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220527153603.887929-3-alex.bennee@linaro.org>
The "riscv64-debian-cross-container" job does not depend on any other
container job from the first stage, so we can move it to the first
stage, too.
The "riscv64-debian-test-cross-container" job needs the debian11
container, so we should add a proper "needs:" statement here.
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20220524093141.91012-1-thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Message-Id: <20220527153603.887929-2-alex.bennee@linaro.org>
Now that we allow compiling with Capstone v3.0.5 again, all our supported
build hosts should provide at least this version of the disassembler
library, so we do not need to ship this as a submodule anymore.
Message-Id: <20220516145823.148450-4-thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
According to
https://lore.kernel.org/qemu-devel/20200921174118.39352-1-richard.henderson@linaro.org/
there was an issue with Capstone 3.0.4 from Ubuntu 18, which was the reason
for bumping our minimum Capstone requirement to version 4.0. And indeed,
compiling with that version 3.0.4 from Ubuntu 18.04 still fails (after
allowing it with a hack in meson.build). But now that we've dropped support
for Ubuntu 18.04, that issue is not relevant anymore. Compiling with Capstone
version 3.0.5 (e.g. used in Ubuntu 20.04) seems to work fine, so let's allow
that version again.
Message-Id: <20220516145823.148450-3-thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
According to our "Supported build platforms" policy, we now do not support
Ubuntu 18.04 anymore. Remove the related container files and entries from
our CI.
Message-Id: <20220516115912.120951-1-thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
The 'check-patch' and 'check-dco' jobs only need Python and git for
checking the patches, so it's not really necessary to use a container
here that has all the other build dependencies installed. By using a
lightweight Alpine container, we can improve the runtime here quite a
bit, cutting it down from ca. 1:30 minutes to ca. 45 seconds.
Suggested-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220516082310.33876-1-thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
When running 'make check' we only get a summary of progress on the
console. Fortunately meson/ninja have saved the raw test output to a
logfile. Exposing this log will make it easier to debug failures that
happen in CI.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220509124134.867431-3-berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
The perl test harness is not necessary anymore since commit 3d2f73ef75
("build: use "meson test" as the test harness"). Thus remove it from
tests/lcitool/projects/qemu.yml, run "make lcitool-refresh" and manually
clean the remaining docker / vm files that are not managed by lcitool yet.
Message-Id: <20220329102808.423681-1-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Without linking it in it won't be presented on the UI. Also while
doing that fix the misnamed job from 20.40 to 20.04.
Fixes: cc44a16002 ("gitlab: add a new aarch32 custom runner definition")
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20220315121954.2283887-1-alex.bennee@linaro.org>
There appears to be a bug in the s390 hardware-accelerated version of
zlib distributed with Ubuntu 20.04, which makes our test
/i386/migration/multifd/tcp/zlib hit an assertion perhaps one time in
10. Fortunately zlib provides an escape hatch where we can disable the
hardware-acceleration entirely by setting the environment variable
DFLTCC to 0. Do this on all our CI which runs on s390 hosts, both our
custom gitlab runner and also the Travis hosts.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Acked-by: Cornelia Huck <cohuck@redhat.com>
Message-id: 20220321161151.3654386-1-alex.bennee@linaro.org
Cc: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
* VSS header fixes (Marc-André)
* 5-level EPT support (Vitaly)
* AMX support (Jing Liu & Yang Zhong)
* Bundle changes to MSI routes (Longpeng)
* More precise emulation of #SS (Gareth)
* Disable ASAN testing
-----BEGIN PGP SIGNATURE-----
iQFIBAABCAAyFiEE8TM4V0tmI4mGbHaCv/vSX3jHroMFAmIwb5QUHHBib256aW5p
QHJlZGhhdC5jb20ACgkQv/vSX3jHroOOUQf8DiNcq8XVVMdX946Qwa4pSxc4ZJtF
X+RkNsscluuLJ2vGEFKwPVps6c6UPqAhXUruZOQmcLmma511MsyJrxyfd4iRgPD2
tL1+n4RpfsbnTEGT8c6TFWWMEIOjLTbKmR/SIxuxpeVG3xlk6tlCevykrIdc90gP
vQIByTGFx3GwiPyDo0j92mA/CsWLnfq6zQ2Tox1xCyt8R+QDimqG0KGLc5RAyiyC
ZmilN2yaqizDfkIzinwHG6gP1NGwVUsrUNl4X9C4mwEMFnsXiyKP5n/BlDZ7w4Wb
QXalFpPg1hJxRGGvyta6OF9VmCfmK9Q0FNVWm1lPE5adn3ECHFo6FJKvfg==
=LVgf
-----END PGP SIGNATURE-----
Merge tag 'for-upstream' of https://gitlab.com/bonzini/qemu into staging
* whpx fixes in preparation for GDB support (Ivan)
* VSS header fixes (Marc-André)
* 5-level EPT support (Vitaly)
* AMX support (Jing Liu & Yang Zhong)
* Bundle changes to MSI routes (Longpeng)
* More precise emulation of #SS (Gareth)
* Disable ASAN testing
# gpg: Signature made Tue 15 Mar 2022 10:51:00 GMT
# gpg: using RSA key F13338574B662389866C7682BFFBD25F78C7AE83
# gpg: issuer "pbonzini@redhat.com"
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>" [full]
# gpg: aka "Paolo Bonzini <pbonzini@redhat.com>" [full]
# Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4 E2F7 7E15 100C CD36 69B1
# Subkey fingerprint: F133 3857 4B66 2389 866C 7682 BFFB D25F 78C7 AE83
* tag 'for-upstream' of https://gitlab.com/bonzini/qemu: (22 commits)
gitlab-ci: do not run tests with address sanitizer
KVM: SVM: always set MSR_AMD64_TSC_RATIO to default value
i386: Add Icelake-Server-v6 CPU model with 5-level EPT support
x86: Support XFD and AMX xsave data migration
x86: add support for KVM_CAP_XSAVE2 and AMX state migration
x86: Add AMX CPUIDs enumeration
x86: Add XFD faulting bit for state components
x86: Grant AMX permission for guest
x86: Add AMX XTILECFG and XTILEDATA components
x86: Fix the 64-byte boundary enumeration for extended state
linux-headers: include missing changes from 5.17
target/i386: Throw a #SS when loading a non-canonical IST
target/i386: only include bits in pg_mode if they are not ignored
kvm/msi: do explicit commit when adding msi routes
kvm-irqchip: introduce new API to support route change
update meson-buildoptions.sh
qga/vss: update informative message about MinGW
qga/vss-win32: check old VSS SDK headers
meson: fix generic location of vss headers
vmxcap: Add 5-level EPT bit
...
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Upstream CI uses ubuntu 18.04 too, so pick
that version (instead of something newer).
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
python2 is not supported any more,
so go install python3 instead.
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
The new s390x machine has more of everything including the OS. As
18.04 will soon be going we might as well get onto something moderately
modern.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Christian Borntraeger <borntraeger@de.ibm.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Acked-by: Cornelia Huck <cohuck@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Cc: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220225172021.3493923-16-alex.bennee@linaro.org>
Although running on aarch64 hardware we can still target 32bit builds
with a cross compiler and run the resulting binaries.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220225172021.3493923-10-alex.bennee@linaro.org>
Cross building QEMU for riscv64 still involves messing about with sid
and ports. However for building tests we can have a slimmer compiler
only container which should be more stable.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220225172021.3493923-7-alex.bennee@linaro.org>
A later compiler is needed for some upcomming tests so we might as
well migrate to an lcitool generated docker file.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Cc: David Hildenbrand <david@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220225172021.3493923-6-alex.bennee@linaro.org>
Using lcitool update debian-arm64-cross to a Debian 11 based system.
As a result we can drop debian-arm64-test-cross just for building
tests.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220225172021.3493923-5-alex.bennee@linaro.org>
It's likely broken, and nobody cared for picking it up again
during the deprecation phase, so let's remove this now.
Since this is the last entry in deprecated_targets_list, remove
the related code in the configure script, too.
Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Acked-by: Cédric Le Goater <clg@kaod.org>
Acked-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20211215084958.185214-1-thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220112112722.3641051-32-alex.bennee@linaro.org>
Personal repos may not have release tags (v6.0.0, v6.1.0, etc) and this
causes cross_system_build_job to fail when pretty-printing a unique
qemu-setup-*.exe name:
version="$(git describe --match v[0-9]*)";
^^^^^^^^^^ fails ^^^^^^^^^^^
mv -v qemu-setup*.exe qemu-setup-${version}.exe;
Fall back to the short commit hash if necessary. This fixes CI failures
that Greg Kurz and I experienced in our personal repos.
Cc: Greg Kurz <groug@kaod.org>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Greg Kurz <groug@kaod.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20220125173454.10381-1-stefanha@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220204204335.1689602-14-alex.bennee@linaro.org>
lcitool doesn't support MSYS2 targets, so manually remove
this now unnecessary library.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220121154134.315047-7-f4bug@amsat.org>
Message-Id: <20220204204335.1689602-11-alex.bennee@linaro.org>
The previous commit removed all uses of libxml2.
Refresh lcitool submodule, update qemu.yml and refresh the generated
files by running:
$ make lcitool-refresh
Note: This refreshment also removes libudev dependency on Fedora
and CentOS due to libvirt-ci commit 18bfaee ("mappings: Improve
mapping for libudev"), since "The udev project has been absorbed
by the systemd project", and lttng-ust on FreeBSD runners due to
libvirt-ci commit 6dd9b6f ("guests: drop lttng-ust from FreeBSD
platform").
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220121154134.315047-6-f4bug@amsat.org>
Message-Id: <20220204204335.1689602-10-alex.bennee@linaro.org>
Now that all RISC-V machines can use OpenSBI BIN images, we remove
OpenSBI ELF images and also exclude these images from BIOS build.
Signed-off-by: Anup Patel <apatel@ventanamicro.com>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
Note, since libtasn1 was fixed in 12.3 [*], this commit re-enables GnuTLS.
[*] https://gitlab.com/gnutls/libtasn1/-/merge_requests/71
Signed-off-by: Brad Smith <brad@comstyle.com>
Tested-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Warner Losh <imp@bsdimp.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <YdUCQLVe5JSWZByQ@humpty.home.comstyle.com>
Message-Id: <20220105135009.1584676-31-alex.bennee@linaro.org>
The current Cirrus CI variables files were previously generated by using
lcitool. This change wires them up to the refresh script to make that
link explicit.
This changes the package list because libvirt-ci now knows about the
mapping for dtc on FreeBSD and macOS platforms.
The variables are also now emit in sorted order for stability across
runs.
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20211215141949.3512719-15-berrange@redhat.com>
Message-Id: <20220105135009.1584676-15-alex.bennee@linaro.org>
This duplicates the ubuntu2004 container but with an inconsistent set of
packages.
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20211215141949.3512719-14-berrange@redhat.com>
Message-Id: <20220105135009.1584676-14-alex.bennee@linaro.org>
The block I/O tests don't work on Alpine because their alternative libc
impl emits different strings for errnos, which breaks the expected
output matching. e.g.
=== IO: pattern 102
wrote 512/512 bytes at offset 512
512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-qemu-img: Error while reading offset 0 of blkdebug:TEST_DIR/blkdebug.conf:TEST_DIR/t.IMGFMT: Input/output error
+qemu-img: Error while reading offset 0 of blkdebug:TEST_DIR/blkdebug.conf:TEST_DIR/t.IMGFMT: I/O error
4
Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824
Formatting 'TEST_DIR/t.IMGFMT.2', fmt=IMGFMT size=0
Currently the I/O tests are skipped as a side effect of the Alpine image
containing a minimal busybox 'sed' binary, rather than GNU sed. This is
a fragile assumption that will be invalidated when the dockerfile is
changed to be autogenerated from a standardized package list that
includes GNU sed.
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20211215141949.3512719-6-berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20220105135009.1584676-6-alex.bennee@linaro.org>
We just ran into a problem that the docs don't build on RHEL8 / CentOS 8
anymore. Seems like these distros are using one of the oldest Sphinx
versions that we still have to support. Thus enable the docs build in
the CI on CentOS so that such bugs don't slip in so easily again.
Message-Id: <20220104091240.160867-1-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Disable check-python-tox
Fix emulation of hppa STBY insn
-----BEGIN PGP SIGNATURE-----
iQFRBAABCgA7FiEEekgeeIaLTbaoWgXAZN846K9+IV8FAmHTMwQdHHJpY2hhcmQu
aGVuZGVyc29uQGxpbmFyby5vcmcACgkQZN846K9+IV9Rhgf/fFVZTVKscFhrnjzH
T7PjP3c6TeVpFU8uIb6h1NC0Ugyo4rBds2Gg/qIE+/X8jvDIHa6/aW1HKacWq6Uq
3Qea+FU56Oc8BpH1CQZEMR/U2ChDofvtcXf/PGHklnpgYCqZctkpNAFPdWlpT9Fx
jy2Y29rGijV1Q2GEXqD8H6Ij6SX+QZBO5DYaiqGsLSWMS8W2a58rfahBBgslvL82
UkpJ7TMzvERN8XPBlFOzvqzgSUX4CObAxn2/EiFT4rXukRbVuf5PbDtl2nNRtcS4
Rfu59GsrHz0tgl3JTRZ/5ab7Na/w8ewknCUPtNygf0k8kKZn8IhDDMhT0ekepoZ9
dsynQg==
=WrNf
-----END PGP SIGNATURE-----
Merge tag 'pull-misc-20220103' of https://gitlab.com/rth7680/qemu into staging
Fix some meson conversion breakage
Disable check-python-tox
Fix emulation of hppa STBY insn
# gpg: Signature made Mon 03 Jan 2022 09:31:48 AM PST
# gpg: using RSA key 7A481E78868B4DB6A85A05C064DF38E8AF7E215F
# gpg: issuer "richard.henderson@linaro.org"
# gpg: Good signature from "Richard Henderson <richard.henderson@linaro.org>" [ultimate]
* tag 'pull-misc-20220103' of https://gitlab.com/rth7680/qemu:
gitlab: Disable check-python-tox
target/hppa: Fix atomic_store_3 for STBY
tests/tcg: Unconditionally use 90 second timeout
tests/tcg: Use $cpu in configure.sh
meson: Unify mips and mips64 in host_arch
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Set this test to be manually run, until failures can be fixed.
Suggested-by: John Snow <jsnow@redhat.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
The philmd@redhat.com email address will stop working on
2022-01-01, change it to my personal email address.
Update .mailmap in case anyone wants to send me an email
because of some past commit I authored.
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20211231000759.707519-1-philmd@redhat.com>
The new msys2-64bit job is often running for more than 50 minutes - and
if the CI is currently loaded, it times out after 60 minutes. The job
has been declared with a bigger timeout, but seems like this is getting
ignored on the shared Gitlab-CI Windows runners, so we're currently
seeing a lot of failures with this job. Thus we have to reduce the time
it takes to finish this job. Since we want to test compiling the WHPX
and HAX accelerator code with this job, switching to another target CPU
is not really a good option, so let's reduce the amount of code that we
have to compile with the --without-default-devices switch instead.
Message-Id: <20211216082253.43899-1-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Gitlab also provides runners with Windows, we can use them to
test compilation with MSYS2, in both, 64-bit and 32-bit.
However, it takes quite a long time to set up the VM, so to stay
in a reasonable time frame, we can only compile and check one
target here.
Message-Id: <20211115140623.104116-1-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Cirrus-CI provides KVM in their Linux containers, so we can also run
our VM-based NetBSD and OpenBSD build jobs there.
Since the VM installation might take a while, we only run the "help"
target on the first invocation to avoid timeouts, and then only check
the build during the next run, once the base image has been cached.
For the the build tests, we also only use very a limited set of target
CPUs since compiling in these VMs is not very fast (especially the
build on OpenBSD seems to be incredibly slow).
The jobs are marked as "manual" only, since this double-indirect setup
(with the cirrus-run script and VMs in the Cirrus-CI containers) might
fail more often than the other jobs, and since we can trigger a limited
amount of Cirrus-CI jobs at a time anyway (due to the restrictions in
the free tier of Cirrus). Thus these jobs are rather added as convenience
for contributors who would like to run the NetBSD/OpenBSD tests without
the need of downloading and installing the corresponding VM images on
their local machines.
Message-Id: <20211209103124.121942-1-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
The device-crash-test script has been quite neglected in the past,
so that it bit-rot quite often. Let's add CI jobs that run this
script for at least some targets, so that this script does not
regress that easily anymore.
Message-Id: <20211126162724.1162049-1-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
The jobs on Cirrus-CI sometimes get delayed quite a bit, waiting to
be scheduled, so while the build test itself finishes within 60 minutes,
the total run time of the jobs can be longer due to this waiting time.
Thus let's increase the timeout on the gitlab side a little bit, so
that these jobs are not marked as failing just because of the delay.
Message-Id: <20211116163309.246602-1-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
On the primary QEMU repository we want the CI jobs to run on the staging
branch as a gating CI test.
Cirrus CI has very limited job concurrency, so if there are too many
jobs triggered they'll queue up and hit the GitLab CI job timeout before
they complete on Cirrus.
If we let Cirrus jobs run again on the master branch immediately after
merging from staging, that just increases the chances jobs will get
queued and subsequently timeout.
The same applies for merges to the stable branches.
User forks meanwhile should be allowed to run Cirrus CI jobs freely.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20211116112757.1909176-1-berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
To ease maintenance, add the custom-runners/ directory and
split custom-runners.yml in 3 files, all included by the
current custom-runners.yml:
- ubuntu-18.04-s390x.yml
- ubuntu-20.04-aarch64.yml
- centos-stream-8-x86_64.yml
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20211115095608.2436223-1-philmd@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20211115142915.3797652-7-alex.bennee@linaro.org>
This introduces three different parts of a job designed to run
on a custom runner managed by Red Hat. The goals include:
a) propose a model for other organizations that want to onboard
their own runners, with their specific platforms, build
configuration and tests.
b) bring awareness to the differences between upstream QEMU and the
version available under CentOS Stream, which is "A preview of
upcoming Red Hat Enterprise Linux minor and major releases".
c) because of b), it should be easier to identify and reduce the gap
between Red Hat's downstream and upstream QEMU.
The components of this custom job are:
I) OS build environment setup code:
- additions to the existing "build-environment.yml" playbook
that can be used to set up CentOS/EL 8 systems.
- a CentOS Stream 8 specific "build-environment.yml" playbook
that adds to the generic one.
II) QEMU build configuration: a script that will produce binaries with
features as similar as possible to the ones built and packaged on
CentOS stream 8.
III) Scripts that define the minimum amount of testing that the
binaries built with the given configuration (point II) under the
given OS build environment (point I) should be subjected to.
IV) Job definition: GitLab CI jobs that will dispatch the build/test
jobs (see points #II and #III) to the machine specifically
configured according to #I.
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Tested-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20211111160501.862396-2-crosa@redhat.com>
Message-Id: <20211115142915.3797652-6-alex.bennee@linaro.org>
In the discussion about renaming the `tests/acceptance` [1], the
conclusion was that the folders inside `tests` are related to the
framework running the tests and not directly related to the type of
the tests.
This changes the folder to `tests/avocado` and adjusts the MAKEFILE, the
CI related files and the documentation.
[1] https://lists.gnu.org/archive/html/qemu-devel/2021-05/msg06553.html
Reviewed-by: Niek Linnenbank <nieklinnenbank@gmail.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20211105155354.154864-3-willianr@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Allows edk2 detect virtio-mmio devices and pcie ecam.
See comment in hw/i386/microvm-dt.c for more details.
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Sergio Lopez <slp@redhat.com>
Message-Id: <20211014193617.2475578-1-kraxel@redhat.com>
The EDK2 firmware images built to test QEMU do not require
the following submodules:
- MdeModulePkg/Universal/RegularExpressionDxe/oniguruma
- UnitTestFrameworkPkg/Library/CmockaLib/cmocka
The only submodules required are:
- ArmPkg/Library/ArmSoftFloatLib/berkeley-softfloat-3
- BaseTools/Source/C/BrotliCompress/brotli
- CryptoPkg/Library/OpensslLib/openssl
- MdeModulePkg/Library/BrotliCustomDecompressLib/brotli
Adapt the buildsys machinery to only initialize the required
submodules.
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Gerd Hoffmann <kraxel@redhat.com>
Message-Id: <20211018105816.2663195-3-philmd@redhat.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
A typo meant the substitution would not work, and the placeholder in the
target file didn't even exist.
The result was that tests were never run on the FreeBSD and macOS jobs,
only a basic build.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Acked-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20210915125452.1704899-3-berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210917162332.3511179-10-alex.bennee@linaro.org>
The check-patch job is intended to be used by contributors or
subsystem maintainers to see if there are style mistakes. The
false positive rate is too high to be used in a gating scenario
so should not run it on the upstream repo ever.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Acked-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20210915125452.1704899-2-berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210917162332.3511179-9-alex.bennee@linaro.org>
To be able to cross build QEMU itself we need to include a few more
libraries. These are only available in Debian's unstable ports repo
for now so we need to base the riscv64 image on sid with the the
minimal libs needed to build QEMU (glib/pixman).
The result works but is not as clean as using build-dep to bring in
more dependencies. However sid is by definition a shifting pile of
sand and by keeping the list of libs minimal we reduce the chance of
having an image we can't build. It's good enough for a basic cross
build testing of TCG.
Cc: "Daniel P. Berrangé" <berrange@redhat.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20210914185830.1378771-2-richard.henderson@linaro.org>
[AJB: tweak allow_failure]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210917162332.3511179-5-alex.bennee@linaro.org>
If a gitlab CI job is marked as manual-only but is not marked
as allow_failure, then gitlab considers that the pipeline is
"blocked" until the job has been manually triggered. We need
to mark these manual-only jobs as also allow_failure: true
so that gitlab doesn't insist that they have run before it
will consider the pipeline to be complete.
Fixes: 4c9af1ea14
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-id: 20210915123412.8232-1-peter.maydell@linaro.org
Acked-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Currently we define a lot of jobs for our custom runners:
for both aarch64 and s390x we have
- all-linux-static
- all
- alldbg
- clang (manual)
- tci
- notcg (manual)
This is overkill. The main reason to run on these hosts is to get
coverage for the host architecture; we can leave the handling of
differences like debug vs non-debug to the x86 CI jobs.
The jobs are also generally running OK; they occasionally fail due to
timeouts, which is likely because we're overloading the machine by
asking it to run 4 CI jobs at once plus the ad-hoc CI.
Remove the 'allow_failure' tag from all these jobs, and switch the
s390x-alldbg, aarch64-all, s390x-tci and aarch64-tci jobs to manual.
(We keep -all on s390x and -alldbg on aarch64 just for diversity
of coverage.)
This will let us make the switch for s390x and aarch64 hosts from
the ad-hoc CI to gitlab.
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Acked-by: Thomas Huth <thuth@redhat.com>
Message-id: 20210913101948.12600-1-peter.maydell@linaro.org
libfdt in Debian is too old to be usable for QEMU. So far we were
silently falling back to the internal dtc submodule, but since
this is wrong, let's remove the --enable-fdt=system switch here now.
Message-Id: <20210827151718.178988-1-thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Acked-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
- fix typo in execlog plugin
- clean-up and document gitlab FOO_RUNNER_AVAILABLE vars
- fix plugin build issue on OSX and modules
- add multi-core support to cache modelling plugin
- clean-ups for plugin arg=FOO handling
-----BEGIN PGP SIGNATURE-----
iQEzBAABCgAdFiEEZoWumedRZ7yvyN81+9DbCVqeKkQFAmEwqF4ACgkQ+9DbCVqe
KkTCcAf/fCHK1hhXJIaxpFyMGvCkhYzUgKlpL4b05ofXFyQJ4JWp+0OadQzBcVIN
PwsVVsXik/5ibKvQiud8wt3/kbr9gj5RZdwVITZMEpod6hy9Nt5oaUhxEZpiVs3f
XZG17zaK8huwTwU6EtHWOnvIWMCuvBXpw8cM/6jF6pEiq162VJbiRo9L/aQ+n0Io
TZTy9YqEbrZUTj+XrvgPV0TewjM6T/zTujXFJtyCzYJ3P3kx9z5a3/KzfyJI9qkk
TYw0MX9MY8J5/sXcP6OjOqXqppsRe+G5Uaz0BoW9SvOmR/0nNGy/N8mVCZTD6mqs
zMsWw5RyIZsZCyWEYj4TYWEORv5Kiw==
=CBs1
-----END PGP SIGNATURE-----
Merge remote-tracking branch 'remotes/stsquad/tags/pull-for-6.2-020921-1' into staging
Testing and plugin updates:
- fix typo in execlog plugin
- clean-up and document gitlab FOO_RUNNER_AVAILABLE vars
- fix plugin build issue on OSX and modules
- add multi-core support to cache modelling plugin
- clean-ups for plugin arg=FOO handling
# gpg: Signature made Thu 02 Sep 2021 11:33:02 BST
# gpg: using RSA key 6685AE99E75167BCAFC8DF35FBD0DB095A9E2A44
# gpg: Good signature from "Alex Bennée (Master Work Key) <alex.bennee@linaro.org>" [full]
# Primary key fingerprint: 6685 AE99 E751 67BC AFC8 DF35 FBD0 DB09 5A9E 2A44
* remotes/stsquad/tags/pull-for-6.2-020921-1: (22 commits)
docs/devel: be consistent about example plugin names
docs/deprecated: deprecate passing plugin args through `arg=`
tests/plugins/syscalls: adhere to new arg-passing scheme
tests/plugins/mem: introduce "track" arg and make args not positional
tests/plugins/insn: made arg inline not positional and parse it as bool
tests/plugins/bb: adapt to the new arg passing scheme
docs/tcg-plugins: new passing parameters scheme for cache docs
plugins/howvec: adapting to the new argument passing scheme
plugins/hwprofile: adapt to the new plugin arguments scheme
plugins/lockstep: make socket path not positional & parse bool arg
plugins/hotblocks: Added correct boolean argument parsing
plugins/hotpages: introduce sortby arg and parsed bool args correctly
plugins/api: added a boolean parsing plugin api
plugins: allow plugin arguments to be passed directly
docs/devel/tcg-plugins: added cores arg to cache plugin
plugins: sort exported symbol list
plugins/cache: supported multicore cache modelling
plugins: do not limit exported symbols if modules are active
gitlab-ci: Fix ..._RUNNER_AVAILABLE variables and document them
gitlab-ci: Remove superfluous "dnf install" statement
...
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
The patch that recently introduced the S390X_RUNNER_AVAILABLE variable
in custom-runners.yml missed that the bottom half of the file is rather
about aarch64 than s390x. Thus rename the S390X_RUNNER_AVAILABLE to
AARCH64_RUNNER_AVAILABLE in those jobs.
Finally mention both variables in our CI documentation, too.
Fixes: c5dd0f0342 ("Improve rules for the staging branch")
Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210730143809.717079-4-thuth@redhat.com>
[AJB: moved due to docu changes]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210806141015.2487502-5-alex.bennee@linaro.org>
The container already features meson and ninja, so there is no need
to try to install it with dnf again.
Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210730143809.717079-3-thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210806141015.2487502-4-alex.bennee@linaro.org>
Both jobs are testing more or less the same thing (building QEMU with
features disabled), so we are wasting precious CI cycles here by doing
this twice. Merge the jobs by using --without-default-features by default
and just adding some additional --disable-... switches which are not
covered by the generic switch (yet). And while we're at it, also test
compilation with "--disable-fdt" (which forces us to change the list
of targets in this job, though, since some targets do not work without
fdt).
Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210730143809.717079-2-thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210806141015.2487502-3-alex.bennee@linaro.org>
Give a good name to test file.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210824083856.17408-29-vsementsov@virtuozzo.com>
[hreitz: Adjust .gitlab-ci.d/buildtest.yml]
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
The windows cross builds still take way too long in gitlab CI, so need
more targets to be skipped. We don't want to hurt coverage of other
cross builds more though, so we let jobs fine tune with a new env
variale $CROSS_SKIP_TARGETS.
We take the set of targets that are considered relatively niche or
very old architectures, and skip approx half of them in win32 builds
and the other half of them in win64.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210810140653.3969823-3-berrange@redhat.com>
[thuth: Swapped the "CROSS_SKIP_TARGETS:" lines as suggested by philmd]
Signed-off-by: Thomas Huth <thuth@redhat.com>
We need to cut down compile time by excluding more targets. Both these
targets still have their 64-bit variant enabled, so the loss of coverage
is mitigated to some degree.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210810140653.3969823-2-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
If maintainers are currently pushing to a branch called "staging"
in their repository, they are ending up with some stuck jobs - unless
they have a s390x CI runner machine available. That's ugly, we should
make sure that the related jobs are really only started if such a
runner is available. So let's only run these jobs if it's the
"staging" branch of the main repository of the QEMU project (where
we can be sure that the s390x runner is available), or if the user
explicitly set a S390X_RUNNER_AVAILABLE variable in their CI configs
to declare that they have such a runner available, too.
Fixes: 4799c21023 ("Jobs based on custom runners: add job definitions ...")
Message-Id: <20210728173857.497523-1-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
These two jobs are currently failing very often - the linker seems to
get killed due to out-of-memory problems. Since apparently nobody has
currently an idea how to fix that nicely, let's mark the jobs as manual
for the time being until someone comes up with a proper fix.
Message-Id: <20210728075141.400816-1-thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
The iotests 197 and 215 are occasionally failing in the gitlab-CI now.
According to the log, the failure is "./common.rc: Killed" which might
be an indication that the process has been killed due to out-of-memory
reasons. Both tests are doing a big read with 2G that likely causes
this issue. It used to work fine in the gitlab-CI in the past, but
either the program is now requiring more free memory, or the the CI
containers have changed, so that the OOM condition now sometimes occurs.
Anyway, these two tests are not really suitable for CI containers if
they are doing things like huge reads (which is likely also the reason
why they haven't been added to the "auto" group in the past), so let's
simply disable them in the gitlab-CI now, too.
Message-Id: <20210727162542.318882-1-thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Jobs depending on another should not use the 'when: always'
condition, because if a dependency failed we should not keep
running jobs depending on it. The correct condition is
'when: on_success'.
Fixes: c6fc0fc1a7 ("gitlab-ci.yml: Add jobs to build OpenSBI firmware binaries")
Reported-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210727142431.1672530-5-philmd@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Jobs depending on another should not use the 'when: always'
condition, because if a dependency failed we should not keep
running jobs depending on it. The correct condition is
'when: on_success'.
Fixes: 71920809ce ("gitlab-ci.yml: Add jobs to build EDK2 firmware binaries")
Reported-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20210727142431.1672530-4-philmd@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Jobs depending on another should not use the 'when: always'
condition, because if a dependency failed we should not keep
running jobs depending on it. The correct condition is
'when: on_success'.
Fixes: f56bf4caf7 ("gitlab: Run Avocado tests manually (except mainstream CI)")
Reported-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20210727142431.1672530-3-philmd@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
- git ignore some file editor detritus
- add overview on device emulation terminology
- remove needless if leg in configure custom devices logic
- numerous gitdm/mailmap updates
- fix plugin_exit race for linux-user
- fix a few bugs in cache modelling plugin
- fix plugin calculation of physical address
- handle pure assembler/linker tcg tests outside of docker
- add tricore build to gitlab
- remove superfluous MacOSX task
- generalise the OpenBSI gitlab rules
-----BEGIN PGP SIGNATURE-----
iQEzBAABCgAdFiEEZoWumedRZ7yvyN81+9DbCVqeKkQFAmD67ioACgkQ+9DbCVqe
KkTKcwf/UlEl2wlYrvymoJy7caasD/UrlNwpaK90zRTxuofJjG2bXX/58EGBlgxe
cjbX3841702HXAFDxzswmWX3yeXQVgNiXrEy5FapB7L3h/qC/BDVS4pz987TITNW
b4lw/d05bQVGavcdJXPnX0CdV8IK4hpieDkdh/LGj5+CSAC39jvyVGTYBpfLKl8D
cps5qcJcJuQjz4hatB1haZOV1WHLyw8KI0srfcEqkh3IImZFZ09frs3qqAxrYRqE
Y62d7SIN5TX1XgWuTaf1mmaMVmbNi9DTdCxWpGYr4ZBLhhzKeljQvKbF45ZGQ0z9
eBVBdxGJTX/WAKrimcbHxLo43pCo2Q==
=7vzj
-----END PGP SIGNATURE-----
Merge remote-tracking branch 'remotes/stsquad/tags/pull-for-6.1-rc1-230721-1' into staging
Doc, metadata, plugin and testing updates for 6.1-rc1:
- git ignore some file editor detritus
- add overview on device emulation terminology
- remove needless if leg in configure custom devices logic
- numerous gitdm/mailmap updates
- fix plugin_exit race for linux-user
- fix a few bugs in cache modelling plugin
- fix plugin calculation of physical address
- handle pure assembler/linker tcg tests outside of docker
- add tricore build to gitlab
- remove superfluous MacOSX task
- generalise the OpenBSI gitlab rules
# gpg: Signature made Fri 23 Jul 2021 17:28:26 BST
# gpg: using RSA key 6685AE99E75167BCAFC8DF35FBD0DB095A9E2A44
# gpg: Good signature from "Alex Bennée (Master Work Key) <alex.bennee@linaro.org>" [full]
# Primary key fingerprint: 6685 AE99 E751 67BC AFC8 DF35 FBD0 DB09 5A9E 2A44
* remotes/stsquad/tags/pull-for-6.1-rc1-230721-1: (28 commits)
gitlab-ci: Extract OpenSBI job rules to reusable section
gitlab-ci: Remove the second superfluous macos task
gitlab: enable a very minimal build with the tricore container
tests/tcg/configure.sh: add handling for assembler only builds
plugins: Fix physical address calculation for IO regions
plugins/cache: Fixed "function decl. is not a prototype" warnings
plugins/cache: limited the scope of a mutex lock
plugins/cache: Fixed a bug with destroying FIFO metadata
tcg/plugins: implement a qemu_plugin_user_exit helper
contrib/gitdm: add more individual contributor entries.
contrib/gitdm: add a new interns group-map for GSoC/Outreachy work
contrib/gitdm: add an explicit academic entry for BU
contrib/gitdm: add group-map for Netflix
contrib/gitdm: add domain-map for NVIDIA
contrib/gitdm: add domain-map for Crudebyte
contrib/gitdm: un-ironically add a mapping for LWN
contrib/gitdm: add domain-map/group-map for Wind River
contrib/gitdm: add domain-map for Eldorado
contrib/gitdm: add domain-map/group-map mappings for Samsung
gitdm.config: sort the corporate GroupMap entries
...
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
All jobs depending on 'docker-opensbi' job must use at most all
the rules that triggers it. The simplest way to ensure that
is to always use the same rules. Extract all the rules to a
reusable section, and include this section (with the 'extends'
keyword) in both 'docker-opensbi' and 'build-opensbi' jobs.
The problem was introduced in commit c6fc0fc1a7 ("gitlab-ci.yml:
Add jobs to build OpenSBI firmware binaries"), but was revealed in
commit 91e9c47e50 ("docker: OpenSBI build job depends on OpenSBI
container").
This fix is similar to the one used with the EDK2 firmware job in
commit ac0595cf6b ("gitlab-ci: Extract EDK2 job rules to reusable
section").
Reported-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210720164829.3949558-1-philmd@redhat.com>
Message-Id: <20210720232703.10650-30-alex.bennee@linaro.org>
While there might have been bigger differnces between the -base and
the -xcode images in the beginning, they almost vanished in the
current builds, e.g. when comparing the output of the "configure"
step after cleaning up the differences due to temporary path names,
I only get:
$ diff -u /tmp/base.txt /tmp/xcode.txt
--- /tmp/base.txt 2021-07-16 09:16:24.211427940 +0200
+++ /tmp/xcode.txt 2021-07-16 09:16:43.029684274 +0200
@@ -19,14 +19,14 @@
Build type: native build
Project name: qemu
Project version: 6.0.50
-C compiler for the host machine: cc (clang 12.0.0 "Apple clang version 12.0.0 (clang-1200.0.32.29)")
+C compiler for the host machine: cc (clang 12.0.0 "Apple clang version 12.0.0 (clang-1200.0.32.28)")
C linker for the host machine: cc ld64 609.8
Host machine cpu family: x86_64
Host machine cpu: x86_64
Program sh found: YES (/bin/sh)
Program python3 found: YES (/usr/local/opt/python@3.9/bin/python3.9)
Program bzip2 found: YES (/usr/bin/bzip2)
-C++ compiler for the host machine: c++ (clang 12.0.0 "Apple clang version 12.0.0 (clang-1200.0.32.29)")
+C++ compiler for the host machine: c++ (clang 12.0.0 "Apple clang version 12.0.0 (clang-1200.0.32.28)")
C++ linker for the host machine: c++ ld64 609.8
Objective-C compiler for the host machine: clang (clang 12.0.0)
Objective-C linker for the host machine: clang ld64 609.8
Since we're not using Xcode itself at all, it seems like it does not
make much sense anymore to waste compute cycles with two images here.
Thus let's delete the -xcode job now.
[AJB: fix up commit formatting which trips up b4]
Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20210719073051.1559348-1-thuth@redhat.com>
Message-Id: <20210720232703.10650-29-alex.bennee@linaro.org>
Rather than base of the shared Debian 10 container which would require
us to bring in even more dependencies just bring in what is needed for
building tricore-softmmu in GitLab. We don't even remove the container
from the DOCKER_PARTIAL_IMAGES lest we cause more confusion.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210720232703.10650-28-alex.bennee@linaro.org>
GitLab will happily publish pages generated by the latest CI pipeline
from any branch:
https://docs.gitlab.com/ee/user/project/pages/introduction.html
"Remember that GitLab Pages are by default branch/tag agnostic
and their deployment relies solely on what you specify in
.gitlab-ci.yml. You can limit the pages job with the only
parameter, whenever a new commit is pushed to a branch used
specifically for your pages."
The current "pages" job is not limited, so it is happily publishing
docs content from any branch/tag in qemu.git that gets pushed to.
This means we're potentially publishing from the "staging" branch
or worse from outdated "stable-NNN" branches
This change restricts it to only publish from the default branch
in the main repository. For contributor forks, however, we allow
it to publish from any branch, since users will have arbitrarily
named topic branches in flight at any time.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210723113051.2792799-1-berrange@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Build windows installer for qemu in gitlab CI,
store the result as artifact.
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20210623091137.1156959-2-kraxel@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
- custom runner playbooks for configuring GitLab runners
- integrate Cirrus jobs into GitLab via cirrus-run
- clean-up docker package lists
- bump NetBSD to 9.2
- bump OpenBSD to 6.9
- make test-mmap more hexagon friendly
- fixup handling of hostaddr for plugins
- disallow some incompatible plugin configurations
- fix handling of -ldl for BSDs
- remove some old unused symbols from the plugin symbol map
- enable plugins by default for most TCG builds
- honour main build -Wall settings for plugins
- new execlog plugin
- new cache modelling plugin
- fix io_uring build regression
- disable modular TCG on Darwin
-----BEGIN PGP SIGNATURE-----
iQEzBAABCgAdFiEEZoWumedRZ7yvyN81+9DbCVqeKkQFAmDu+xsACgkQ+9DbCVqe
KkQ2XAf+MRBeSMX9J091iloiqSjpuobAcVuoIFciG7gEqWd2iTQVPDqJNgBJUANQ
vTA7f89ljxj/FZjSZwLgR6cqY6X0ufXI1BLEIt7s78LJtVp14sHVo89GwNBzmRwo
615T49KG8b5EBBU5YlVcAW/m8DlfgI4b1ufS/qHldOukKegu+haoCDjGG6RNpYNx
mmXgLOBJiB/p2u2S73KTIa+3AEIIlsTJZpdLPus8hby+/Q4qB9t8YbdHaweyM9qs
NjiojczvlbHLFd/IQSl6fqBS9QI+KWf4+oFd6ZB4THhr9/GUYTiMeLjRKucBgDQG
TwGtd1RN3gAPI5aEJ0xlgCL/crz1qg==
=dQl6
-----END PGP SIGNATURE-----
Merge remote-tracking branch 'remotes/stsquad/tags/pull-testing-and-plugins-140721-5' into staging
Testing and plugin updates:
- custom runner playbooks for configuring GitLab runners
- integrate Cirrus jobs into GitLab via cirrus-run
- clean-up docker package lists
- bump NetBSD to 9.2
- bump OpenBSD to 6.9
- make test-mmap more hexagon friendly
- fixup handling of hostaddr for plugins
- disallow some incompatible plugin configurations
- fix handling of -ldl for BSDs
- remove some old unused symbols from the plugin symbol map
- enable plugins by default for most TCG builds
- honour main build -Wall settings for plugins
- new execlog plugin
- new cache modelling plugin
- fix io_uring build regression
- disable modular TCG on Darwin
# gpg: Signature made Wed 14 Jul 2021 15:56:27 BST
# gpg: using RSA key 6685AE99E75167BCAFC8DF35FBD0DB095A9E2A44
# gpg: Good signature from "Alex Bennée (Master Work Key) <alex.bennee@linaro.org>" [full]
# Primary key fingerprint: 6685 AE99 E751 67BC AFC8 DF35 FBD0 DB09 5A9E 2A44
* remotes/stsquad/tags/pull-testing-and-plugins-140721-5: (44 commits)
MAINTAINERS: Added myself as a reviewer for TCG Plugins
docs/devel: Added cache plugin to the plugins docs
plugins/cache: Added FIFO and LRU eviction policies
plugins/cache: Enable cache parameterization
plugins: Added a new cache modelling plugin
docs/devel: tcg-plugins: add execlog plugin description
contrib/plugins: add execlog to log instruction execution and memory access
contrib/plugins: enable -Wall for building plugins
tcg/plugins: enable by default for most TCG builds
configure: stop user enabling plugins on Windows for now
configure: add an explicit static and plugins check
configure: don't allow plugins to be enabled for a non-TCG build
tcg/plugins: remove some stale entries from the symbol list
meson.build: relax the libdl test to one for the function dlopen
meson.build: move TCG plugin summary output
plugins: fix-up handling of internal hostaddr for 32 bit
tests/tcg: make test-mmap a little less aggressive
tests/vm: update openbsd to release 6.9
tests/vm: update NetBSD to 9.2
tests/docker: expand opensuse-leap package list
...
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
All jobs depending on 'docker-edk2' job must use at most all
the rules that triggers it. The simplest way to ensure that
is to always use the same rules. Extract all the rules to a
reusable section, and include this section (with the 'extends'
keyword) in both 'docker-edk2' and 'build-edk2' jobs.
The problem was introduced in commit 71920809ce ("gitlab-ci.yml:
Add jobs to build EDK2 firmware binaries"), but was revealed in
commit 1925468ddb ("docker: EDK2 build job depends on EDK2
container") and eventually failed on CI:
https://gitlab.com/qemu-project/qemu/-/pipelines/335995843
Reported-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210714101003.3113726-1-philmd@redhat.com>
Aside from a minor bloat to file size the ability to have TCG plugins
has no real impact on performance unless a plugin is actively loaded.
Even then the libempty.so plugin shows only a minor degradation in
performance caused by the extra book keeping the TCG has to do to keep
track of instructions. As it's a useful feature lets just enable it by
default and reduce our testing matrix a little.
We need to move our linker testing earlier so we can be sure we can
enable the loader module required. As we have ruled out static &
plugins in an earlier patch we can also reduce the indent a little.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20210709143005.1554-33-alex.bennee@linaro.org>
This adds support for running 4 jobs via Cirrus CI runners:
* FreeBSD 12
* FreeBSD 13
* macOS 11 with default XCode
* macOS 11 with latest XCode
The gitlab job uses a container published by the libvirt-ci
project (https://gitlab.com/libvirt/libvirt-ci) that contains
the 'cirrus-run' command. This accepts a short yaml file that
describes a single Cirrus CI job, runs it using the Cirrus CI
REST API, and reports any output to the console.
In this way Cirrus CI is effectively working as an indirect
custom runner for GitLab CI pipelines. The key benefit is that
Cirrus CI job results affect the GitLab CI pipeline result and
so the user only has look at one CI dashboard.
[AJB: remove $TEMPORARILY_DISABLED condition, s/py37/py38/]
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210625172211.451010-3-berrange@redhat.com>
Message-Id: <20210709143005.1554-8-alex.bennee@linaro.org>
The QEMU project has two machines (aarch64 and s390x) that can be used
for jobs that do build and run tests. This introduces those jobs,
which are a mapping of custom scripts used for the same purpose.
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20210630012619.115262-5-crosa@redhat.com>
Message-Id: <20210709143005.1554-5-alex.bennee@linaro.org>
As described in the included documentation, the "custom runner" jobs
extend the GitLab CI jobs already in place. One of their primary
goals of catching and preventing regressions on a wider number of host
systems than the ones provided by GitLab's shared runners.
This sets the stage in which other community members can add their own
machine configuration documentation/scripts, and accompanying job
definitions. As a general rule, those newly added contributed jobs
should run as "non-gating", until their reliability is verified (AKA
"allow_failure: true").
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20210630012619.115262-2-crosa@redhat.com>
Message-Id: <20210709143005.1554-2-alex.bennee@linaro.org>
Currently tox tests against the installed interpreters, however if any
supported interpreter is absent then it will return fail. It seems not
reasonable to expect developers to have all supported interpreters
installed on their systems. Luckily tox can be configured to skip
missing interpreters.
This changed the tox setup so that missing interpreters are skipped by
default. On the CI, however, we still want to enforce it tests
against all supported. This way on CI the
--skip-missing-interpreters=false option is passed to tox.
Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20210630184546.456582-1-wainersm@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Well, Cleber was right, this is a better name.
In preparation for adding a different kind of virtual environment check
(One that simply uses whichever version of Python you happen to have),
rename this test 'check-pipenv' so that it matches the CI job
'check-python-pipenv'.
Remove the "If you don't know which test to run" hint, because it's not
actually likely you have Python 3.6 installed to be able to run the
test. It's still the test I'd most prefer you to run, but it's not the
test you are most likely to be able to run.
Rename the 'venv' target to 'pipenv' as well, and move the more
pertinent help text under the 'check-pipenv' target.
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-id: 20210629214323.1329806-8-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
Rather than relying on external tweaks lets just do it inside
checkpatch's direct commitish handling which is QEMU specific code
anyway.
Suggested-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20210623102749.25686-4-alex.bennee@linaro.org>
This job is hitting the 70min limit, so split it in 2 tasks.
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210525082556.4011380-7-f4bug@amsat.org>
This copies the behaviour of patchew's configuration to make the diff
algorithm generate a minimal diff.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20210602153247.27651-1-alex.bennee@linaro.org>
* Bump minimum versions of some requirements after removing CentOS 7 support
-----BEGIN PGP SIGNATURE-----
iQJFBAABCAAvFiEEJ7iIR+7gJQEY8+q5LtnXdP5wLbUFAmC3L1IRHHRodXRoQHJl
ZGhhdC5jb20ACgkQLtnXdP5wLbU8wA//aRAIn0Qa3xoZdsT434P99GHauCZ5ePq3
meY4co69c+TfkQZ/b0xlyvT3+7bd9Ni92CQL7n/LtX5bs3pIhRjHNIsm50e4x6Da
0jb422cKWfRIltXlCfdL/dnFRtJluH83M0seRGMvmhveuWPZ19oSIasEiQjeg//e
KMNK5z4HEnJ1czb3Bf8p38bmmY/O/QEAA5wAqk7iJkJHz6T/GlqImLYYwpFPlHj6
JQttm0aWsHrsqEXxnuV0/DT1yHyXDB6S4iuAvABZWhv/M/nCaXo0ib0gW5NPtRPo
Yf3HO163F9/fewJCc1AUsBe1C/2UwmSWRhEtxpr9uuW2Mv9qEl3hkJwd4k6sEOvh
U4i+GONC4eElPcmECKUfHA9EP+7faDs6xnM6Ev/PIEp+cPJ2QRfklZv4qpMUWYtb
3KkADchOyVZAsdB8cGmnznDEVmno1Dt0adVUq8CF6uW6MwD3pb4838arrfwIfwOp
g4yTI1AQQykkxxOaR7GSNoxWRti7TH4fzLhx/xXDd9TKIOWuOepiyuhB7+Q48WAJ
6EW/JUIzOe7k3GsI0iBsk+y67ED2tpATiRKWWw4QS1BwIhNEMUqgZkucFfKE6Ze1
B1Xw+Di+1CzrLHV6lCjtdLVmqWqcGcO43HqhSK4sodE87BDGGwy9UxE6oBVt2C1x
HPQ6/omDlFk=
=RAkI
-----END PGP SIGNATURE-----
Merge remote-tracking branch 'remotes/thuth-gitlab/tags/pull-request-2021-06-02' into staging
* Update the references to some doc files (use *.rst instead of *.txt)
* Bump minimum versions of some requirements after removing CentOS 7 support
# gpg: Signature made Wed 02 Jun 2021 08:12:18 BST
# gpg: using RSA key 27B88847EEE0250118F3EAB92ED9D774FE702DB5
# gpg: issuer "thuth@redhat.com"
# gpg: Good signature from "Thomas Huth <th.huth@gmx.de>" [full]
# gpg: aka "Thomas Huth <thuth@redhat.com>" [full]
# gpg: aka "Thomas Huth <huth@tuxfamily.org>" [full]
# gpg: aka "Thomas Huth <th.huth@posteo.de>" [unknown]
# Primary key fingerprint: 27B8 8847 EEE0 2501 18F3 EAB9 2ED9 D774 FE70 2DB5
* remotes/thuth-gitlab/tags/pull-request-2021-06-02:
configure: bump min required CLang to 6.0 / XCode 10.0
configure: bump min required GCC to 7.5.0
configure: bump min required glib version to 2.56
tests/docker: drop CentOS 7 container
tests/vm: convert centos VM recipe to CentOS 8
crypto: drop used conditional check
crypto: bump min gnutls to 3.5.18, dropping RHEL-7 support
crypto: bump min gcrypt to 1.8.0, dropping RHEL-7 support
crypto: drop back compatibility typedefs for nettle
crypto: bump min nettle to 3.4, dropping RHEL-7 support
patchew: move quick build job from CentOS 7 to CentOS 8 container
block/ssh: Bump minimum libssh version to 0.8.7
docs: fix references to docs/devel/s390-dasd-ipl.rst
docs: fix references to docs/specs/tpm.rst
docs: fix references to docs/devel/build-system.rst
docs: fix references to docs/devel/atomics.rst
docs: fix references to docs/devel/tracing.rst
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
It has been over two years since RHEL-8 was released, and thus per the
platform build policy, we no longer need to support RHEL-7 as a build
target.
Reviewed-by: Thomas Huth <thuth@redhat.com>
Acked-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20210514120415.1368922-10-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
It has been over two years since RHEL-8 was released, and thus per the
platform build policy, we no longer need to support RHEL-7 as a build
target. This lets us increment the minimum required gnutls version
Per repology, current shipping versions are:
RHEL-8: 3.6.14
Debian Buster: 3.6.7
openSUSE Leap 15.2: 3.6.7
Ubuntu LTS 18.04: 3.5.18
Ubuntu LTS 20.04: 3.6.13
FreeBSD: 3.6.15
Fedora 33: 3.6.16
Fedora 34: 3.7.1
OpenBSD: 3.6.15
macOS HomeBrew: 3.6.15
Ubuntu LTS 18.04 has the oldest version and so 3.5.18 is the new minimum.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20210514120415.1368922-7-berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
[thuth: rebased to use .gitlab-ci.d/buildtest.yml]
Signed-off-by: Thomas Huth <thuth@redhat.com>
It has been over two years since RHEL-8 was released, and thus per the
platform build policy, we no longer need to support RHEL-7 as a build
target. This lets us increment the minimum required gcrypt version and
assume that HMAC is always supported
Per repology, current shipping versions are:
RHEL-8: 1.8.5
Debian Buster: 1.8.4
openSUSE Leap 15.2: 1.8.2
Ubuntu LTS 18.04: 1.8.1
Ubuntu LTS 20.04: 1.8.5
FreeBSD: 1.9.2
Fedora 33: 1.8.6
Fedora 34: 1.9.3
OpenBSD: 1.9.3
macOS HomeBrew: 1.9.3
Ubuntu LTS 18.04 has the oldest version and so 1.8.0 is the new minimum.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20210514120415.1368922-6-berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
[thuth: rebased to use .gitlab-ci.d/buildtest.yml]
Signed-off-by: Thomas Huth <thuth@redhat.com>
It has been over two years since RHEL-8 was released, and thus per the
platform build policy, we no longer need to support RHEL-7 as a build
target. This lets us increment the minimum required nettle version and
drop a lot of backwards compatibility code for 2.x series of nettle.
Per repology, current shipping versions are:
RHEL-8: 3.4.1
Debian Buster: 3.4.1
openSUSE Leap 15.2: 3.4.1
Ubuntu LTS 18.04: 3.4
Ubuntu LTS 20.04: 3.5.1
FreeBSD: 3.7.2
Fedora 33: 3.5.1
Fedora 34: 3.7.2
OpenBSD: 3.7.2
macOS HomeBrew: 3.7.2
Ubuntu LTS 18.04 has the oldest version and so 3.4 is the new minimum.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20210514120415.1368922-4-berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
[thuth: rebased to use .gitlab-ci.d/buildtest.yml]
Signed-off-by: Thomas Huth <thuth@redhat.com>
Add a Python container that has just enough juice for us to run the
Python code quality analysis tools. Base this container on Fedora,
because Fedora has very convenient packaging for testing multiple Python
versions.
We need python3, pip (for pulling packages), pipenv and virtualenv for
creating virtual environments, and tox for running tests. make is needed
for running 'make check-tox' and 'make venv-check' targets. Python3.10
is needed explicitly because the tox package only pulls in 3.6-3.9, but
we wish to test the forthcoming release of Python as well to help
predict any problems. Lastly, we need gcc to compile PyPI packages that
may not have a binary distribution available.
Add two tests:
check-python-pipenv uses pipenv to test a frozen, very explicit set of
packages against our minimum supported python version, Python 3.6. This
test is not allowed to fail. The dependencies this test uses do not
change unless python/Pipfile.lock is changed.
check-python-tox uses tox to install the latest versions of required
python dependencies against a wide array of Python versions from 3.6 to
3.9, even including the yet-to-be-released Python 3.10. This test is
allowed to fail with a warning.
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Message-id: 20210527211715.394144-32-jsnow@redhat.com
[Fix rebase conflict over .gitlab-ci.yml --js]
Signed-off-by: John Snow <jsnow@redhat.com>
Per GitLab documentation [*]:
"rules replaces only/except and they can’t be used together
in the same job."
Since the 'rules' syntax is more powerful and we are already using
it, convert the check-dco/check-patch jobs so no job use the 'only/
except' syntax.
[*] https://docs.gitlab.com/ee/ci/yaml/#rules
Inspired-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210525132418.4133235-3-f4bug@amsat.org>
We want to skip the checkpatch and DCO signoff jobs when
pushing to the default branch. Currently this branch is
called 'master', but we don't need to hardcode this in
the CI configuration, because the $CI_DEFAULT_BRANCH
env variable exposes it.
References:
- https://sfconservancy.org/news/2020/jun/23/gitbranchname/
- https://about.gitlab.com/blog/2021/03/10/new-git-default-branch-name/
Suggested-by: Savitoj Singh <savsingh@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210525153826.4174157-1-philmd@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Due to a design problem and misunderstanding between the Avocado
framework and QEMU, Avocado is fetching many asset artifacts it
shouldn't be fetching, exhausting the jobs CI timeout.
Since Avocado artifacts are cached, this is not an issue with old
forks, which already have populated the cache and do not need to
download new artifacts to run the tests.
However this is very confusing to new contributors who start to
fork the project and keep having failing CI pipelines.
As a temporary kludge, add the QEMU_CI_AVOCADO_TESTING variable
to allow old forks to keep running the Avocado tests, while still
allowing new forks to use the mainstream set of CI tests.
Keep the tests enabled by default on the mainstream namespace
which is old enough to have a populated cache, hoping we will
keep this cache long enough until the Avocado/QEMU design issue
is fixed.
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210525082556.4011380-9-f4bug@amsat.org>
Sometimes pull requests are merged during the week-end, triggering
a CI pipeline. Currently if such pipeline fails, the Avocado reports
are available for 2 days. For the reviewers working on the project
during office hours, the reports are already discarded when they
want to look at them. Increase this time to 1 week, which should
give reviewers enough time.
Only keep the reports on failure, which is the only case we'll
look at them.
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210525082556.4011380-8-f4bug@amsat.org>
Extract the jobs preparing the cross containers into a new file
(container-cross.yml).
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210525082556.4011380-5-f4bug@amsat.org>
To allow forks to easily decide which jobs they want to run,
but without disrupting the current default, move the current
set of jobs to a new file corresponding to the jobs run by
the mainstream project CI:
https://gitlab.com/qemu-project/qemu/-/pipelines
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210511072952.2813358-11-f4bug@amsat.org>
Extract the build/test jobs run by default on the mainstream
CI into a new file (buildtest.yml).
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210511072952.2813358-9-f4bug@amsat.org>
It is not possible to use the previously extracted templates
without this set of core containers. Extract them into a new
file (container-core.yml) to be able to build them without
having to build all the other containers by default.
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210519185504.2198573-10-f4bug@amsat.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
To be able to reuse the mainstream build/test jobs templates,
extract them into a new file (buildtest-template.yml).
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210519185504.2198573-8-f4bug@amsat.org>
[thuth: Keep the "acceptance_test_job_template" name for now]
Signed-off-by: Thomas Huth <thuth@redhat.com>
Extract the build stages used by our job templates to a new file
(stages.yml) to be able to include it with the other templates,
without having to run all the jobs included in the default
.gitlab-ci.yml, which are mainly useful for mainstream CI.
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210519185504.2198573-7-f4bug@amsat.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Extract the DCO / checkpatch jobs to a new file (static_checks.yml)
to be able to run them without having to run all the jobs included
in the default .gitlab-ci.yml, which are mainly useful for the
mainstream CI.
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210519185504.2198573-6-f4bug@amsat.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Extract the crossbuild job templates to a new file
(crossbuild-template.yml) to be able to reuse them
without having to run all the jobs included, which
are mainly useful for mainstream CI.
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210519185504.2198573-5-f4bug@amsat.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Extract the container job template to a new file
(container-template.yml) to be able to reuse it
without having to run all the jobs included, which
are mainly useful for mainstream CI.
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210519185504.2198573-4-f4bug@amsat.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
We're currently only testing TCI with a 64-bit host -- also test
with a 32-bit host. Enable a selection of softmmu and user-only
targets, 32-bit LE, 64-bit LE, 32-bit BE, as there are ifdefs for each.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Acked-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20210502235727.1979457-27-richard.henderson@linaro.org>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
The hexagon container is always manually built but of course not
everyone will be building it themselves and pushing to their
registries. We still need to create a "local" registry copy for the
actual gitlab tests to run. We don't build it in this case, just pull
it across from the upstream registry. We disable this rule from
running on the qemu-project itself so it doesn't accidentally wipe out
our master copy.
Fixes: 910c40ee94 ("gitlab: add build-user-hexagon test")
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Cornelia Huck <cohuck@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Cc: Cornelia Huck <cohuck@redhat.com>
Message-Id: <20210520174303.12310-4-alex.bennee@linaro.org>
Since c8e6793903 ("containers.yml: build with docker.py tooling") we
don't need to manually pull stuff from the upstream repository. Just
set the -r field to explicitly use that rather than the current
registry.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20210520174303.12310-3-alex.bennee@linaro.org>
A newer compiler is needed to build tests for Power10 instructions. As
done for arm64 on c729a99d27, a new '-test-cross' image is created for
ppc64 and ppc64le. As done on 936fda4d77, a test for compiler support
is added to verify that the toolchain in use has '-mpower10'. Finally,
Unused images (docker-power-cross and docker-ppc64-cross) are removed.
Signed-off-by: Matheus Ferst <matheus.ferst@eldorado.org.br>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210423205757.1752480-2-matheus.ferst@eldorado.org.br>
Message-Id: <20210512102051.12134-27-alex.bennee@linaro.org>
'extends' is an alternative to using YAML anchors
and is a little more flexible and readable. See:
https://docs.gitlab.com/ee/ci/yaml/#extends
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210418233448.1267991-2-f4bug@amsat.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Add a new job to cross-build the mips64el target without
the TCG accelerator (IOW: only KVM accelerator enabled).
Only build the mips64el target which is known to work
and has users.
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Acked-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20210428170410.479308-31-f4bug@amsat.org>
In d0f26e68a0 ("gitlab: force enable docs build in Fedora, Ubuntu,
Debian") we made sure we can build the documents on more than one
system. However we don't want to build documents all the time as it's
a waste of cycles (and energy). So lets reduce the total amount of
documentation we build while still keeping coverage of at least one
build on each supported target.
Fixes: a8a3abe0b3 ("gitlab: move docs and tools build across from Travis")
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20210323165308.15244-23-alex.bennee@linaro.org>
This allows the build jobs to start running as soon as their respective
container image is ready, instead of waiting for all container builds
to finish.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210216132954.295906-3-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Currently we attempt to skip building container images if the commits do
not involve changes to the dockerfiles or gitlab CI definitions.
Conceptually this makes sense, but there is a challenge in the real
world implementation of this in gitlab.
In the case of a CI pipeline triggered from a merge request, GitLab
knows the common ancestor of the merge request and the main git repo,
so it can trivially determine if any of the commits associated with
the MR change the dockerfiles.
In the case of a CI pipeline triggered from a push to a branch, it is
much more difficult. There is no concept of a common ancestor in this
case. Instead GitLab looks at the set of commits in the git push event.
On the surface this may sound reasonable, but it doesn't take into
account that a push event does not always contain the full set of
patches from a branch.
For example, consider pushing 5 commits, one of which contains a
dockerfile change. This will trigger a CI pipeline for the
containers. Now consider you do some more work on the branch and push 3
further commits, so you now have a branch of 8 commits. For the second
push GitLab will only look at the 3 most recent commits, the other 5
were already present. Thus GitLab will not realize that the branch has
dockerfile changes that need to trigger the container build.
This can cause real world problems:
- Push 5 commits to branch "foo", including a dockerfile change
=> rebuilds the container images with content from "foo"
=> build jobs runs against containers from "foo"
- Refresh your master branch with latest upstream master
=> rebuilds the container images with content from "master"
=> build jobs runs against containers from "master"
- Push 3 more commits to branch "foo", with no dockerfile change
=> no container rebuild triggers
=> build jobs runs against containers from "master"
The "changes" conditional in gitlab is OK, *provided* your build
jobs are not relying on any external state from previous builds.
This is NOT the case in QEMU, because we are building container
images and these are cached. This is a scenario in which the
"changes" conditional is not usuable.
The only other way to avoid this problem would be to use the git
branch name as the container image tag, instead of always using
"latest". The downside of this approach is that the user's gitlab
registry will grow significantly until it starts to trigger
GitLab's automatic deletion policy. Every time the user starts
a new branch they will have to trigger a rebuild of the container
images. Given this, we might as well just drop the conditional
and always build the container images. Most of the time docker
will be able to use the layer cache to avoid the most expensive
part of the rebuild process (installing all the RPMs/debs/etc)
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20210216132954.295906-2-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Add build-system-opensuse jobs and opensuse-leap.docker dockerfile.
Use openSUSE Leap 15.2 container image in the gitlab-CI.
Signed-off-by: Cho, Yu-Chen <acho@suse.com>
Tested-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20201229085046.8536-1-acho@suse.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
After adding some missing packages, it's possible to check 32-bit
builds and tests with the fedora-i386-cross container in the gitlab-CI,
too. Unfortunately, the code in subprojects/ ignores the --extra-cflags
(on purpose), so the vhost-user part has to be disabled for this.
While we're at it, update the container to Fedora 31. Unfortunately the
gcc from the later versions emits some very dubious format-truncation
warnings, so Fedora 32 and 33 are currently unsuitable for this job.
Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20201215083451.92322-1-thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Otherwise we miss coverage of KVM support in the cross build. To
balance it out add arm-softmmu (no kvm, subset of aarch64),
cris-softmmu and ppc-softmmu to the exclude list which do get coverage
elsewhere.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20201210190417.31673-3-alex.bennee@linaro.org>
Cross-build ARM and X86 targets with only Xen accelerator enabled.
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20201207131503.3858889-6-philmd@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Cross-build s390x target with only KVM accelerator enabled.
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20201207131503.3858889-5-philmd@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Introduce a job template to cross-build accelerator specific
jobs (enable a specific accelerator, disabling the others).
The specific accelerator is selected by the $ACCEL environment
variable (default to KVM).
Extra options such disabling other accelerators are passed
via the $ACCEL_CONFIGURE_OPTS environment variable.
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20201207131503.3858889-4-philmd@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
'extends' is an alternative to using YAML anchors
and is a little more flexible and readable. See:
https://docs.gitlab.com/ee/ci/yaml/#extends
More importantly it allows exploding YAML jobs.
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20201207131503.3858889-3-philmd@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Update containers.yml to use the $CI_REGISTRY variable as other files
such as edk2.yml do.
Signed-off-by: Rebecca Cran <rebecca@nuviainc.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20201113172519.31056-1-rebecca@nuviainc.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
We don't need running commentary for the CI logs and by keeping it
short we might just see the problem on the first page. While we are at
it flush the previous line so order is maintained between script and
sub process.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20201113174404.19608-1-alex.bennee@linaro.org>
If the current branch is synced to the current upstream git master,
there are no commits that need checking. This causes checkpatch.pl
to print an error that it found no commits. We need to avoid calling
checkpatch.pl in this case.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Acked-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20201019143537.283094-2-berrange@redhat.com>
Message-Id: <20201021163136.27324-5-alex.bennee@linaro.org>
While checkpatch.pl can validate DCO sign off that job must always be
advisory only since it is expected that certain patches will fail some
code style rules.
We require the DCO sign off to be mandatory for all commits though, so
it benefits from being validated in a standalone job.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20200918132903.1848939-3-berrange@redhat.com>
[thuth: Use "stage: build" to let it run earlier]
Signed-off-by: Thomas Huth <thuth@redhat.com>
This job is advisory since it is expected that certain patches will fail
the style checks and checkpatch.pl provides no way to mark exceptions to
the rules.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20200918132903.1848939-2-berrange@redhat.com>
[thuth: Use "stage: build" to let it run earlier]
Signed-off-by: Thomas Huth <thuth@redhat.com>
Some of the cross-compiler builds (the mips build and the win64 build
for example) are quite slow and sometimes hit the 1h time limit.
Increase the limit a little bit to make sure that we do not get failures
in the CI runs just because of some few minutes.
Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200921174320.46062-7-thuth@redhat.com>
Message-Id: <20200925154027.12672-15-alex.bennee@linaro.org>
We do not support Debian 9 in QEMU anymore, and the Debian 9 containers
are now no longer used in the gitlab-CI. Time to remove them.
Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20200921174320.46062-6-thuth@redhat.com>
Message-Id: <20200925154027.12672-14-alex.bennee@linaro.org>
We do not support Debian 9 anymore, thus update the Tricore container
to Debian 10 now.
Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20200921174320.46062-4-thuth@redhat.com>
Message-Id: <20200925154027.12672-12-alex.bennee@linaro.org>
According to our support policy, Debian 9 is not supported by the
QEMU project anymore. Since we now switched the MinGW cross-compiler
builds to Fedora, we do not need these Debian9-based containers
in the gitlab-CI anymore, and can now also get rid of the "layer3"
container build stage this way.
Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20200921174320.46062-3-thuth@redhat.com>
Message-Id: <20200925154027.12672-11-alex.bennee@linaro.org>
According to our support policy, we do not support Debian 9 in QEMU
anymore, and we only support building the Windows binaries with a
very recent version of the MinGW toolchain. So we should not test
the MinGW cross-compilation with Debian 9 anymore, but switch to
something newer like Fedora. To do this, we need a separate Fedora
container for each build that provides the QEMU_CONFIGURE_OPTS
environment variable.
Unfortunately, the MinGW 64-bit compiler seems to be a little bit
slow, so we also have to disable some features like "capstone" in the
build here to make sure that the CI pipelines still finish within a
reasonable amount of time.
Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200921174320.46062-2-thuth@redhat.com>
Message-Id: <20200925154027.12672-10-alex.bennee@linaro.org>
Now that we can use all our QEMU test containers in the gitlab-CI, we can
easily add some jobs that test cross-compilation for various architectures.
There is just only small ugliness: Since the shared runners on gitlab.com
are single-threaded, we have to split each compilation job into two parts
(--disable-user and --disable-system), and exclude some additional targets,
to avoid that the jobs are running too long and hitting the timeout of 1 h.
Message-Id: <20200823111757.72002-8-thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
This updates the GitLab CI opensbi job to build opensbi bios images
for the generic platform.
Signed-off-by: Bin Meng <bin.meng@windriver.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Message-Id: <1596439832-29238-7-git-send-email-bmeng.cn@gmail.com>
Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
The tests/docker/* wildcard seems to only match the files that are directly
in the tests/docker folder - but changes to the files in the directory
tests/docker/dockerfiles are currently ignored. Seems like we need a
separate entry to match the files in that folder. With this wildcard added,
the stages now get re-run successfully when something in the dockerfiles
has been changed.
Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200713182235.30379-1-thuth@redhat.com>
Instead of building the docker files directly use the same docker.py
scripting as we do for building locally. This should help ensure we
use the exact same steps and allow us to cache properly when building
locally.
To get this working you have to have a fairly recent docker binary
otherwise you will see the error message:
=> ERROR importing cache manifest from registry.gitlab....
So far docker 19.03.12 works (from the docker apt repos) but 18.09.1,
build 4c52b90 which is packaged in Debian Buster fails.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200701135652.1366-39-alex.bennee@linaro.org>
Most of the time we are just rebuilding the same things. We can skip
this although currently there is no mechanism for picking up new
distro releases.
Rather than try to be too fine grained allow any change to trigger all
the images being rebuilt.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200701135652.1366-38-alex.bennee@linaro.org>
We happily use all the cross images for both cross-building QEMU as
well as building the linux-user tests. However calling docker from
within docker seems not to work. As we can build in Debian anyway why
not include an image that has all the compilers available for
non-docker invocation.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200701135652.1366-33-alex.bennee@linaro.org>
According to the documentation to be able to use --cache-from for
remote registries you need to enable both buildkit and inline the
metadata. We want to do this to support pulling from gitlab when users
build their local docker images.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200701135652.1366-24-alex.bennee@linaro.org>
We have a number of container images in tests/docker/dockerfiles
that are intended to provide well defined environments for doing
test builds. We want our CI system to use these containers too.
This introduces builds of all of them as the first stage in the
CI, so that the built containers are available for later build
jobs. The containers are setup to use the GitLab container
registry as the cache, so we only pay the penalty of the full
build when the dockerfiles change. The main qemu-project/qemu
repo is used as a second cache, so that users forking QEMU will
see a fast turnaround time on their CI jobs.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20200622153318.751107-3-berrange@redhat.com>
[AJB: tweak the tag format]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20200701135652.1366-22-alex.bennee@linaro.org>
If no stage is listed, jobs get put in an implicit "test" stage.
Some jobs which create container images to be used by later stages
are currently listed as in a "build" stages.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Acked-by: Laszlo Ersek <lersek@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200622153318.751107-2-berrange@redhat.com>
Message-Id: <20200701135652.1366-21-alex.bennee@linaro.org>
The edk2.yml and opensbi.yml files have recently been moved/renamed,
but the change has not been reflected in the rules in the YML files
yet.
Fixes: 922febe2af ("Move edk2 and opensbi YAML files to .gitlab-ci.d folder")
Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20200625151627.24986-1-thuth@redhat.com>
Message-Id: <20200701135652.1366-20-alex.bennee@linaro.org>
We have a dedicated folder for the gitlab-ci - so there is no need
to clutter the top directory with these .yml files.
Message-Id: <20200525131823.715-5-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Add two GitLab jobs to build the OpenSBI firmware binaries.
The first job builds a Docker image with the packages requisite
to build OpenSBI, and stores this image in the GitLab registry.
The second job pulls the image from the registry and builds the
OpenSBI firmware binaries.
The docker image is only rebuilt if the GitLab YAML or the
Dockerfile is updated. The second job is only built when the
roms/opensbi/ submodule is updated, when a git-ref starts with
'opensbi' or when the last commit contains 'OpenSBI'. The files
generated are archived in the artifacts.zip file.
With OpenSBI v0.6, it took 2 minutes 56 seconds to build
the docker image, and 1 minute 24 seconds to generate the
artifacts.zip with the firmware binaries (filesize: 111KiB).
See: https://gitlab.com/lbmeng/qemu/pipelines/120520138
Suggested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
Add two GitLab job to build the EDK2 firmware binaries.
The first job build a Docker image with the packages requisite
to build EDK2, and store this image in the GitLab registry.
The second job pull the image from the registry and build the
EDK2 firmware binaries.
The docker image is only rebuilt if the GitLab YAML or the
Dockerfile is updated.
The second job is only built when the roms/edk2/ submodule is
updated, when a git-ref starts with 'edk2' or when the last
commit contains 'EDK2'. The files generated are archived in
the artifacts.zip file.
With edk2-stable201905, it took 2 minutes 52 seconds to build
the docker image, and 36 minutes 28 seconds to generate the
artifacts.zip with the firmware binaries (filesize: 10MiB).
See: https://gitlab.com/philmd/qemu/pipelines/107553178
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Acked-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
