The lcitool created containers save the full distro package list
details into /packages.txt. The idea is that build jobs will 'cat'
this file, so that the build log has a record of what packages
were used. This is important info, because when it comes to debug
failures, the original container is often lost.
This extends the manually written dockerfiles to also create the
/packages.txt file.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20240724095505.33544-2-berrange@redhat.com>
Acked-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20240729144414.830369-2-alex.bennee@linaro.org>
All the lcitool generated containers define a "MAKE" env. It will be
convenient for later patches if all containers do this.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-ID: <20240513111551.488088-2-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
This update includes support for privileged instructions.
Signed-off-by: Brian Cain <bcain@quicinc.com>
Reviewed-by: Matheus Tavares Bernardino <quic_mathbern@quicinc.com>
Tested-by: Matheus Tavares Bernardino <quic_mathbern@quicinc.com>
Message-Id: <20240114232354.4109231-1-bcain@quicinc.com>
The `ccache` tool can be very effective at reducing compilation times
when re-running pipelines with only minor changes each time. For example
a fresh 'build-system-fedora' job will typically take 20 minutes on the
gitlab.com shared runners. With ccache this is reduced to as little as
6 minutes.
Normally meson would auto-detect existance of ccache in $PATH and use
it automatically, but the way we wrap meson from configure breaks this,
as we're passing in an config file with explicitly set compiler paths.
Thus we need to add $CCACHE_WRAPPERSPATH to the front of $PATH. For
unknown reasons if doing this in msys though, gcc becomes unable to
invoke 'cc1' when run from meson. For msys we thus set CC='ccache gcc'
before invoking 'configure' instead.
A second problem with msys is that cache misses are incredibly
expensive, so enabling ccache massively slows down the build when
the cache isn't well populated. This is suspected to be a result of
the cost of spawning processes under the msys architecture. To deal
with this we set CCACHE_DEPEND=1 which enables ccache's 'depend_only'
strategy. This avoids extra spawning of the pre-processor during
cache misses, with the downside that is it less likely ccache will
find a cache hit after semantically benign compiler flag changes.
This is the lesser of two evils, as otherwise we can't use ccache
at all under msys and remain inside the job time limit.
If people are finding ccache to hurt their pipelines, it can be
disabled by setting the 'CCACHE_DISABLE=1' env variable against
their gitlab fork CI settings.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20230804111054.281802-2-berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230829161528.2707696-2-alex.bennee@linaro.org>
Instead of having CI pick tomli from the vendored wheel at configure
time, place it in the containers.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Several debian-based tests need the python3-venv dependency as a
consequence of Debian debundling the "ensurepip" module normally
included with Python.
As mkvenv.py stands as of this commit, Debian requires EITHER:
(A) setuptools and pip, or
(B) ensurepip
mkvenv is a few seconds faster if you have setuptools and pip, so
developers should prefer the first requirement. For the purposes of CI,
the time-save is a wash; it's only a matter of who is responsible for
installing pip and when; the timing is about the same.
Arbitrarily, I chose adding ensurepip to the test configuration because
it is normally part of the Python stdlib, and always having it allows us
a more consistent cross-platform environment.
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20230511035435.734312-12-jsnow@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
These are flat but not generated by lcitool so we need to manually
update them with the `useradd` stanza.
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20230228190653.1602033-20-alex.bennee@linaro.org>
The current docker image for cross compiling hexagon guests
is manually built since it takes >2 hours to build from source.
This patch:
1. Solves the above issue by using the prebuilt clang
toolchain hosted on CodeLinaro [1] and maintained by QUIC [2].
2. The dockerfile is also switched from multi-stage to single stage
build to allow the CI docker engine to reuse the layer cache.
3. Re-enables the hexagon-cross-container job to be always run in
CI and makes it a non-optional dependency for the
build-user-hexagon job.
The changes for 1 & 2 together bring down the build time to
~3 minutes in GitLab CI when cache is reused and ~9 minutes
when cache cannot be reused.
[1]: https://github.com/CodeLinaro/hexagon-builder
[2]: https://github.com/quic/toolchain_for_hexagon/releases/
Signed-off-by: Mukilan Thiyagarajan <quic_mthiyaga@quicinc.com>
[AJB: also tweak MAINTAINERS, remove QEMU_JOB_ONLY_FORKS and comment]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20221219144354.11659-1-quic_mthiyaga@quicinc.com>
Message-Id: <20221221090411.1995037-6-alex.bennee@linaro.org>
debian-hexagon-cross contains two images, one to build the toolchain
used for building the Hexagon tests themselves, and one image to build
QEMU and run the tests.
This commit adds flex/bison to the final image that builds QEMU so that
it can also build idef-parser.
Note: This container is not built by the CI and needs to be rebuilt and
updated manually.
Signed-off-by: Anton Johansson <anjo@rev.ng>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20221014223642.147845-1-anjo@rev.ng>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20221027183637.2772968-8-alex.bennee@linaro.org>
Update to the latest stable Debian. While we are at it flatten into a
single dockerfile as we do not some of the extraneous packages from
the base image to build the toolchain.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220914155950.804707-28-alex.bennee@linaro.org>
This image is a little special because it takes a long time to build.
As such most users don't want to be doing that and just pull random
binaries from the ether as intended by the container gods. This
involves someone with credentials and a beefy machine running:
make docker-image-debian-hexagon-cross V=1 NOCACHE=1 J=30
docker tag qemu/debian-hexagon-cross registry.gitlab.com/qemu-project/qemu/qemu/debian-hexagon-cross
docker push registry.gitlab.com/qemu-project/qemu/qemu/debian-hexagon-cross
With a suitable binary in the "cloud" a normal user will run:
make docker-image-debian-hexagon-cross
or have it run for them through the dependency mechanism of our
over-engineered makefiles and get the binary they wanted. There are a
few wrinkles of course including needing to tweak the final image to
have the credentials of the user so we can actually do our cross
compiles.
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Cc: Alessandro Di Federico <ale@rev.ng>
Cc: Philippe Mathieu-Daudé <f4bug@amsat.org>
Cc: Brian Cain <bcain@quicinc.com>
Message-Id: <20210512102051.12134-7-alex.bennee@linaro.org>
