mirrors/qemu
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
274afbb121
qemu/.gitlab-ci.d/containers.yml

265 lines
6.2 KiB
YAML
Raw Normal View History

gitlab: build all container images during CI We have a number of container images in tests/docker/dockerfiles that are intended to provide well defined environments for doing test builds. We want our CI system to use these containers too. This introduces builds of all of them as the first stage in the CI, so that the built containers are available for later build jobs. The containers are setup to use the GitLab container registry as the cache, so we only pay the penalty of the full build when the dockerfiles change. The main qemu-project/qemu repo is used as a second cache, so that users forking QEMU will see a fast turnaround time on their CI jobs. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-Id: <20200622153318.751107-3-berrange@redhat.com> [AJB: tweak the tag format] Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Acked-by: Thomas Huth <thuth@redhat.com> Message-Id: <20200701135652.1366-22-alex.bennee@linaro.org>
2020-07-01 16:56:33 +03:00
.container_job_template: &container_job_definition
image: docker:stable
stage: containers
services:
- docker:dind
before_script:
- export TAG="$CI_REGISTRY_IMAGE/qemu/$NAME:latest"
- export COMMON_TAG="$CI_REGISTRY/qemu-project/qemu/$NAME:latest"
containers.yml: build with docker.py tooling Instead of building the docker files directly use the same docker.py scripting as we do for building locally. This should help ensure we use the exact same steps and allow us to cache properly when building locally. To get this working you have to have a fairly recent docker binary otherwise you will see the error message: => ERROR importing cache manifest from registry.gitlab.... So far docker 19.03.12 works (from the docker apt repos) but 18.09.1, build 4c52b90 which is packaged in Debian Buster fails. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20200701135652.1366-39-alex.bennee@linaro.org>
2020-07-01 16:56:50 +03:00
- apk add python3
gitlab: build all container images during CI We have a number of container images in tests/docker/dockerfiles that are intended to provide well defined environments for doing test builds. We want our CI system to use these containers too. This introduces builds of all of them as the first stage in the CI, so that the built containers are available for later build jobs. The containers are setup to use the GitLab container registry as the cache, so we only pay the penalty of the full build when the dockerfiles change. The main qemu-project/qemu repo is used as a second cache, so that users forking QEMU will see a fast turnaround time on their CI jobs. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-Id: <20200622153318.751107-3-berrange@redhat.com> [AJB: tweak the tag format] Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Acked-by: Thomas Huth <thuth@redhat.com> Message-Id: <20200701135652.1366-22-alex.bennee@linaro.org>
2020-07-01 16:56:33 +03:00
- docker info
- docker login registry.gitlab.com -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD"
script:
containers.yml: build with docker.py tooling Instead of building the docker files directly use the same docker.py scripting as we do for building locally. This should help ensure we use the exact same steps and allow us to cache properly when building locally. To get this working you have to have a fairly recent docker binary otherwise you will see the error message: => ERROR importing cache manifest from registry.gitlab.... So far docker 19.03.12 works (from the docker apt repos) but 18.09.1, build 4c52b90 which is packaged in Debian Buster fails. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20200701135652.1366-39-alex.bennee@linaro.org>
2020-07-01 16:56:50 +03:00
- echo "TAG:$TAG"
- echo "COMMON_TAG:$COMMON_TAG"
gitlab: build all container images during CI We have a number of container images in tests/docker/dockerfiles that are intended to provide well defined environments for doing test builds. We want our CI system to use these containers too. This introduces builds of all of them as the first stage in the CI, so that the built containers are available for later build jobs. The containers are setup to use the GitLab container registry as the cache, so we only pay the penalty of the full build when the dockerfiles change. The main qemu-project/qemu repo is used as a second cache, so that users forking QEMU will see a fast turnaround time on their CI jobs. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-Id: <20200622153318.751107-3-berrange@redhat.com> [AJB: tweak the tag format] Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Acked-by: Thomas Huth <thuth@redhat.com> Message-Id: <20200701135652.1366-22-alex.bennee@linaro.org>
2020-07-01 16:56:33 +03:00
- docker pull "$TAG" || docker pull "$COMMON_TAG" || true
containers.yml: build with docker.py tooling Instead of building the docker files directly use the same docker.py scripting as we do for building locally. This should help ensure we use the exact same steps and allow us to cache properly when building locally. To get this working you have to have a fairly recent docker binary otherwise you will see the error message: => ERROR importing cache manifest from registry.gitlab.... So far docker 19.03.12 works (from the docker apt repos) but 18.09.1, build 4c52b90 which is packaged in Debian Buster fails. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20200701135652.1366-39-alex.bennee@linaro.org>
2020-07-01 16:56:50 +03:00
- ./tests/docker/docker.py --engine docker build
-t "qemu/$NAME" -f "tests/docker/dockerfiles/$NAME.docker"
-r $CI_REGISTRY_IMAGE
- docker tag "qemu/$NAME" "$TAG"
gitlab: build all container images during CI We have a number of container images in tests/docker/dockerfiles that are intended to provide well defined environments for doing test builds. We want our CI system to use these containers too. This introduces builds of all of them as the first stage in the CI, so that the built containers are available for later build jobs. The containers are setup to use the GitLab container registry as the cache, so we only pay the penalty of the full build when the dockerfiles change. The main qemu-project/qemu repo is used as a second cache, so that users forking QEMU will see a fast turnaround time on their CI jobs. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-Id: <20200622153318.751107-3-berrange@redhat.com> [AJB: tweak the tag format] Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Acked-by: Thomas Huth <thuth@redhat.com> Message-Id: <20200701135652.1366-22-alex.bennee@linaro.org>
2020-07-01 16:56:33 +03:00
- docker push "$TAG"
after_script:
- docker logout
gitlab: limit re-builds of the containers Most of the time we are just rebuilding the same things. We can skip this although currently there is no mechanism for picking up new distro releases. Rather than try to be too fine grained allow any change to trigger all the images being rebuilt. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-Id: <20200701135652.1366-38-alex.bennee@linaro.org>
2020-07-01 16:56:49 +03:00
rules:
- changes:
- .gitlab-ci.d/containers.yml
- tests/docker/*
gitlab-ci/containers: Add missing wildcard where we should look for changes The tests/docker/* wildcard seems to only match the files that are directly in the tests/docker folder - but changes to the files in the directory tests/docker/dockerfiles are currently ignored. Seems like we need a separate entry to match the files in that folder. With this wildcard added, the stages now get re-run successfully when something in the dockerfiles has been changed. Signed-off-by: Thomas Huth <thuth@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20200713182235.30379-1-thuth@redhat.com>
2020-07-13 21:22:35 +03:00
- tests/docker/dockerfiles/*
gitlab: limit re-builds of the containers Most of the time we are just rebuilding the same things. We can skip this although currently there is no mechanism for picking up new distro releases. Rather than try to be too fine grained allow any change to trigger all the images being rebuilt. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-Id: <20200701135652.1366-38-alex.bennee@linaro.org>
2020-07-01 16:56:49 +03:00
- if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'
- if: '$CI_COMMIT_REF_NAME == "testing/next"'
gitlab: build all container images during CI We have a number of container images in tests/docker/dockerfiles that are intended to provide well defined environments for doing test builds. We want our CI system to use these containers too. This introduces builds of all of them as the first stage in the CI, so that the built containers are available for later build jobs. The containers are setup to use the GitLab container registry as the cache, so we only pay the penalty of the full build when the dockerfiles change. The main qemu-project/qemu repo is used as a second cache, so that users forking QEMU will see a fast turnaround time on their CI jobs. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-Id: <20200622153318.751107-3-berrange@redhat.com> [AJB: tweak the tag format] Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Acked-by: Thomas Huth <thuth@redhat.com> Message-Id: <20200701135652.1366-22-alex.bennee@linaro.org>
2020-07-01 16:56:33 +03:00
amd64-centos7-container:
<<: *container_job_definition
variables:
NAME: centos7
amd64-centos8-container:
<<: *container_job_definition
variables:
NAME: centos8
amd64-debian10-container:
<<: *container_job_definition
variables:
NAME: debian10
amd64-debian11-container:
<<: *container_job_definition
variables:
NAME: debian11
amd64-debian9-container:
<<: *container_job_definition
variables:
NAME: debian9
amd64-debian9-mxe-container:
<<: *container_job_definition
stage: containers-layer2
needs: ['amd64-debian9-container']
variables:
NAME: debian9-mxe
alpha-debian-cross-container:
<<: *container_job_definition
stage: containers-layer2
needs: ['amd64-debian10-container']
variables:
NAME: debian-alpha-cross
amd64-debian-cross-container:
<<: *container_job_definition
stage: containers-layer2
needs: ['amd64-debian10-container']
variables:
NAME: debian-amd64-cross
tests/docker: add a linux-user testing focused image We happily use all the cross images for both cross-building QEMU as well as building the linux-user tests. However calling docker from within docker seems not to work. As we can build in Debian anyway why not include an image that has all the compilers available for non-docker invocation. Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-Id: <20200701135652.1366-33-alex.bennee@linaro.org>
2020-07-01 16:56:44 +03:00
amd64-debian-user-cross-container:
<<: *container_job_definition
stage: containers-layer2
needs: ['amd64-debian10-container']
variables:
NAME: debian-all-test-cross
gitlab: build all container images during CI We have a number of container images in tests/docker/dockerfiles that are intended to provide well defined environments for doing test builds. We want our CI system to use these containers too. This introduces builds of all of them as the first stage in the CI, so that the built containers are available for later build jobs. The containers are setup to use the GitLab container registry as the cache, so we only pay the penalty of the full build when the dockerfiles change. The main qemu-project/qemu repo is used as a second cache, so that users forking QEMU will see a fast turnaround time on their CI jobs. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Message-Id: <20200622153318.751107-3-berrange@redhat.com> [AJB: tweak the tag format] Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Acked-by: Thomas Huth <thuth@redhat.com> Message-Id: <20200701135652.1366-22-alex.bennee@linaro.org>
2020-07-01 16:56:33 +03:00
amd64-debian-container:
<<: *container_job_definition
stage: containers-layer2
needs: ['amd64-debian10-container']
variables:
NAME: debian-amd64
arm64-debian-cross-container:
<<: *container_job_definition
stage: containers-layer2
needs: ['amd64-debian10-container']
variables:
NAME: debian-arm64-cross
arm64-test-debian-cross-container:
<<: *container_job_definition
stage: containers-layer2
needs: ['amd64-debian11-container']
variables:
NAME: debian-arm64-test-cross
armel-debian-cross-container:
<<: *container_job_definition
stage: containers-layer2
needs: ['amd64-debian10-container']
variables:
NAME: debian-armel-cross
armhf-debian-cross-container:
<<: *container_job_definition
stage: containers-layer2
needs: ['amd64-debian10-container']
variables:
NAME: debian-armhf-cross
hppa-debian-cross-container:
<<: *container_job_definition
stage: containers-layer2
needs: ['amd64-debian10-container']
variables:
NAME: debian-hppa-cross
m68k-debian-cross-container:
<<: *container_job_definition
stage: containers-layer2
needs: ['amd64-debian10-container']
variables:
NAME: debian-m68k-cross
mips64-debian-cross-container:
<<: *container_job_definition
stage: containers-layer2
needs: ['amd64-debian10-container']
variables:
NAME: debian-mips64-cross
mips64el-debian-cross-container:
<<: *container_job_definition
stage: containers-layer2
needs: ['amd64-debian10-container']
variables:
NAME: debian-mips64el-cross
mips-debian-cross-container:
<<: *container_job_definition
stage: containers-layer2
needs: ['amd64-debian10-container']
variables:
NAME: debian-mips-cross
mipsel-debian-cross-container:
<<: *container_job_definition
stage: containers-layer2
needs: ['amd64-debian10-container']
variables:
NAME: debian-mipsel-cross
powerpc-debian-cross-container:
<<: *container_job_definition
stage: containers-layer2
needs: ['amd64-debian10-container']
variables:
NAME: debian-powerpc-cross
ppc64-debian-cross-container:
<<: *container_job_definition
stage: containers-layer2
needs: ['amd64-debian10-container']
variables:
NAME: debian-ppc64-cross
ppc64el-debian-cross-container:
<<: *container_job_definition
stage: containers-layer2
needs: ['amd64-debian10-container']
variables:
NAME: debian-ppc64el-cross
riscv64-debian-cross-container:
<<: *container_job_definition
stage: containers-layer2
needs: ['amd64-debian10-container']
variables:
NAME: debian-riscv64-cross
s390x-debian-cross-container:
<<: *container_job_definition
stage: containers-layer2
needs: ['amd64-debian10-container']
variables:
NAME: debian-s390x-cross
sh4-debian-cross-container:
<<: *container_job_definition
stage: containers-layer2
needs: ['amd64-debian10-container']
variables:
NAME: debian-sh4-cross
sparc64-debian-cross-container:
<<: *container_job_definition
stage: containers-layer2
needs: ['amd64-debian10-container']
variables:
NAME: debian-sparc64-cross
tricore-debian-cross-container:
<<: *container_job_definition
stage: containers-layer2
needs: ['amd64-debian9-container']
variables:
NAME: debian-tricore-cross
win32-debian-cross-container:
<<: *container_job_definition
stage: containers-layer3
needs: ['amd64-debian9-mxe-container']
variables:
NAME: debian-win32-cross
win64-debian-cross-container:
<<: *container_job_definition
stage: containers-layer3
needs: ['amd64-debian9-mxe-container']
variables:
NAME: debian-win64-cross
xtensa-debian-cross-container:
<<: *container_job_definition
variables:
NAME: debian-xtensa-cross
cris-fedora-cross-container:
<<: *container_job_definition
variables:
NAME: fedora-cris-cross
amd64-fedora-container:
<<: *container_job_definition
variables:
NAME: fedora
i386-fedora-cross-container:
<<: *container_job_definition
variables:
NAME: fedora-i386-cross
amd64-ubuntu1804-container:
<<: *container_job_definition
variables:
NAME: ubuntu1804
amd64-ubuntu2004-container:
<<: *container_job_definition
variables:
NAME: ubuntu2004
amd64-ubuntu-container:
<<: *container_job_definition
variables:
NAME: ubuntu
Reference in New Issue Copy Permalink