qemu/.gitlab-ci.d/windows.yml

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

142 lines
5.3 KiB
YAML
Raw Normal View History

.shared_msys2_builder:
extends: .base_job_template
tags:
- shared-windows
- windows
- windows-1809
cache:
gitlab: enable ccache for many build jobs The `ccache` tool can be very effective at reducing compilation times when re-running pipelines with only minor changes each time. For example a fresh 'build-system-fedora' job will typically take 20 minutes on the gitlab.com shared runners. With ccache this is reduced to as little as 6 minutes. Normally meson would auto-detect existance of ccache in $PATH and use it automatically, but the way we wrap meson from configure breaks this, as we're passing in an config file with explicitly set compiler paths. Thus we need to add $CCACHE_WRAPPERSPATH to the front of $PATH. For unknown reasons if doing this in msys though, gcc becomes unable to invoke 'cc1' when run from meson. For msys we thus set CC='ccache gcc' before invoking 'configure' instead. A second problem with msys is that cache misses are incredibly expensive, so enabling ccache massively slows down the build when the cache isn't well populated. This is suspected to be a result of the cost of spawning processes under the msys architecture. To deal with this we set CCACHE_DEPEND=1 which enables ccache's 'depend_only' strategy. This avoids extra spawning of the pre-processor during cache misses, with the downside that is it less likely ccache will find a cache hit after semantically benign compiler flag changes. This is the lesser of two evils, as otherwise we can't use ccache at all under msys and remain inside the job time limit. If people are finding ccache to hurt their pipelines, it can be disabled by setting the 'CCACHE_DISABLE=1' env variable against their gitlab fork CI settings. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Message-Id: <20230804111054.281802-2-berrange@redhat.com> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20230829161528.2707696-2-alex.bennee@linaro.org>
2023-08-29 19:15:17 +03:00
key: "$CI_JOB_NAME"
paths:
- msys64/var/cache
gitlab: enable ccache for many build jobs The `ccache` tool can be very effective at reducing compilation times when re-running pipelines with only minor changes each time. For example a fresh 'build-system-fedora' job will typically take 20 minutes on the gitlab.com shared runners. With ccache this is reduced to as little as 6 minutes. Normally meson would auto-detect existance of ccache in $PATH and use it automatically, but the way we wrap meson from configure breaks this, as we're passing in an config file with explicitly set compiler paths. Thus we need to add $CCACHE_WRAPPERSPATH to the front of $PATH. For unknown reasons if doing this in msys though, gcc becomes unable to invoke 'cc1' when run from meson. For msys we thus set CC='ccache gcc' before invoking 'configure' instead. A second problem with msys is that cache misses are incredibly expensive, so enabling ccache massively slows down the build when the cache isn't well populated. This is suspected to be a result of the cost of spawning processes under the msys architecture. To deal with this we set CCACHE_DEPEND=1 which enables ccache's 'depend_only' strategy. This avoids extra spawning of the pre-processor during cache misses, with the downside that is it less likely ccache will find a cache hit after semantically benign compiler flag changes. This is the lesser of two evils, as otherwise we can't use ccache at all under msys and remain inside the job time limit. If people are finding ccache to hurt their pipelines, it can be disabled by setting the 'CCACHE_DISABLE=1' env variable against their gitlab fork CI settings. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Message-Id: <20230804111054.281802-2-berrange@redhat.com> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20230829161528.2707696-2-alex.bennee@linaro.org>
2023-08-29 19:15:17 +03:00
- ccache
when: always
needs: []
stage: build
gitlab: enable ccache for many build jobs The `ccache` tool can be very effective at reducing compilation times when re-running pipelines with only minor changes each time. For example a fresh 'build-system-fedora' job will typically take 20 minutes on the gitlab.com shared runners. With ccache this is reduced to as little as 6 minutes. Normally meson would auto-detect existance of ccache in $PATH and use it automatically, but the way we wrap meson from configure breaks this, as we're passing in an config file with explicitly set compiler paths. Thus we need to add $CCACHE_WRAPPERSPATH to the front of $PATH. For unknown reasons if doing this in msys though, gcc becomes unable to invoke 'cc1' when run from meson. For msys we thus set CC='ccache gcc' before invoking 'configure' instead. A second problem with msys is that cache misses are incredibly expensive, so enabling ccache massively slows down the build when the cache isn't well populated. This is suspected to be a result of the cost of spawning processes under the msys architecture. To deal with this we set CCACHE_DEPEND=1 which enables ccache's 'depend_only' strategy. This avoids extra spawning of the pre-processor during cache misses, with the downside that is it less likely ccache will find a cache hit after semantically benign compiler flag changes. This is the lesser of two evils, as otherwise we can't use ccache at all under msys and remain inside the job time limit. If people are finding ccache to hurt their pipelines, it can be disabled by setting the 'CCACHE_DISABLE=1' env variable against their gitlab fork CI settings. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Message-Id: <20230804111054.281802-2-berrange@redhat.com> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20230829161528.2707696-2-alex.bennee@linaro.org>
2023-08-29 19:15:17 +03:00
timeout: 100m
variables:
# This feature doesn't (currently) work with PowerShell, it stops
# the echo'ing of commands being run and doesn't show any timing
FF_SCRIPT_SECTIONS: 0
artifacts:
name: "$CI_JOB_NAME-$CI_COMMIT_REF_SLUG"
expire_in: 7 days
paths:
- build/meson-logs/testlog.txt
reports:
junit: "build/meson-logs/testlog.junit.xml"
before_script:
- Write-Output "Acquiring msys2.exe installer at $(Get-Date -Format u)"
- If ( !(Test-Path -Path msys64\var\cache ) ) {
mkdir msys64\var\cache
}
- Invoke-WebRequest
"https://repo.msys2.org/distrib/msys2-x86_64-latest.sfx.exe.sig"
-outfile "msys2.exe.sig"
- if ( Test-Path -Path msys64\var\cache\msys2.exe.sig ) {
Write-Output "Cached installer sig" ;
if ( ((Get-FileHash msys2.exe.sig).Hash -ne (Get-FileHash msys64\var\cache\msys2.exe.sig).Hash) ) {
Write-Output "Mis-matched installer sig, new installer download required" ;
Remove-Item -Path msys64\var\cache\msys2.exe.sig ;
if ( Test-Path -Path msys64\var\cache\msys2.exe ) {
Remove-Item -Path msys64\var\cache\msys2.exe
}
} else {
Write-Output "Matched installer sig, cached installer still valid"
}
} else {
Write-Output "No cached installer sig, new installer download required" ;
if ( Test-Path -Path msys64\var\cache\msys2.exe ) {
Remove-Item -Path msys64\var\cache\msys2.exe
}
}
- if ( !(Test-Path -Path msys64\var\cache\msys2.exe ) ) {
Write-Output "Fetching latest installer" ;
Invoke-WebRequest
"https://repo.msys2.org/distrib/msys2-x86_64-latest.sfx.exe"
-outfile "msys64\var\cache\msys2.exe" ;
Copy-Item -Path msys2.exe.sig -Destination msys64\var\cache\msys2.exe.sig
} else {
Write-Output "Using cached installer"
}
- Write-Output "Invoking msys2.exe installer at $(Get-Date -Format u)"
- msys64\var\cache\msys2.exe -y
- ((Get-Content -path .\msys64\etc\\post-install\\07-pacman-key.post -Raw)
-replace '--refresh-keys', '--version') |
Set-Content -Path ${CI_PROJECT_DIR}\msys64\etc\\post-install\\07-pacman-key.post
- .\msys64\usr\bin\bash -lc "sed -i 's/^CheckSpace/#CheckSpace/g' /etc/pacman.conf"
- .\msys64\usr\bin\bash -lc 'pacman --noconfirm -Syuu' # Core update
- .\msys64\usr\bin\bash -lc 'pacman --noconfirm -Syuu' # Normal update
- taskkill /F /FI "MODULES eq msys-2.0.dll"
script:
- Write-Output "Installing mingw packages at $(Get-Date -Format u)"
- .\msys64\usr\bin\bash -lc "pacman -Sy --noconfirm --needed
bison diffutils flex
git grep make sed
$MINGW_TARGET-binutils
$MINGW_TARGET-capstone
gitlab: enable ccache for many build jobs The `ccache` tool can be very effective at reducing compilation times when re-running pipelines with only minor changes each time. For example a fresh 'build-system-fedora' job will typically take 20 minutes on the gitlab.com shared runners. With ccache this is reduced to as little as 6 minutes. Normally meson would auto-detect existance of ccache in $PATH and use it automatically, but the way we wrap meson from configure breaks this, as we're passing in an config file with explicitly set compiler paths. Thus we need to add $CCACHE_WRAPPERSPATH to the front of $PATH. For unknown reasons if doing this in msys though, gcc becomes unable to invoke 'cc1' when run from meson. For msys we thus set CC='ccache gcc' before invoking 'configure' instead. A second problem with msys is that cache misses are incredibly expensive, so enabling ccache massively slows down the build when the cache isn't well populated. This is suspected to be a result of the cost of spawning processes under the msys architecture. To deal with this we set CCACHE_DEPEND=1 which enables ccache's 'depend_only' strategy. This avoids extra spawning of the pre-processor during cache misses, with the downside that is it less likely ccache will find a cache hit after semantically benign compiler flag changes. This is the lesser of two evils, as otherwise we can't use ccache at all under msys and remain inside the job time limit. If people are finding ccache to hurt their pipelines, it can be disabled by setting the 'CCACHE_DISABLE=1' env variable against their gitlab fork CI settings. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Message-Id: <20230804111054.281802-2-berrange@redhat.com> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20230829161528.2707696-2-alex.bennee@linaro.org>
2023-08-29 19:15:17 +03:00
$MINGW_TARGET-ccache
$MINGW_TARGET-curl
$MINGW_TARGET-cyrus-sasl
$MINGW_TARGET-dtc
$MINGW_TARGET-gcc
$MINGW_TARGET-glib2
$MINGW_TARGET-gnutls
$MINGW_TARGET-gtk3
$MINGW_TARGET-libgcrypt
$MINGW_TARGET-libjpeg-turbo
$MINGW_TARGET-libnfs
$MINGW_TARGET-libpng
$MINGW_TARGET-libssh
$MINGW_TARGET-libtasn1
$MINGW_TARGET-libusb
$MINGW_TARGET-lzo2
$MINGW_TARGET-nettle
$MINGW_TARGET-ninja
$MINGW_TARGET-pixman
$MINGW_TARGET-pkgconf
$MINGW_TARGET-python
$MINGW_TARGET-SDL2
$MINGW_TARGET-SDL2_image
$MINGW_TARGET-snappy
$MINGW_TARGET-spice
$MINGW_TARGET-usbredir
$MINGW_TARGET-zstd "
- Write-Output "Running build at $(Get-Date -Format u)"
- $env:CHERE_INVOKING = 'yes' # Preserve the current working directory
- $env:MSYS = 'winsymlinks:native' # Enable native Windows symlink
gitlab: enable ccache for many build jobs The `ccache` tool can be very effective at reducing compilation times when re-running pipelines with only minor changes each time. For example a fresh 'build-system-fedora' job will typically take 20 minutes on the gitlab.com shared runners. With ccache this is reduced to as little as 6 minutes. Normally meson would auto-detect existance of ccache in $PATH and use it automatically, but the way we wrap meson from configure breaks this, as we're passing in an config file with explicitly set compiler paths. Thus we need to add $CCACHE_WRAPPERSPATH to the front of $PATH. For unknown reasons if doing this in msys though, gcc becomes unable to invoke 'cc1' when run from meson. For msys we thus set CC='ccache gcc' before invoking 'configure' instead. A second problem with msys is that cache misses are incredibly expensive, so enabling ccache massively slows down the build when the cache isn't well populated. This is suspected to be a result of the cost of spawning processes under the msys architecture. To deal with this we set CCACHE_DEPEND=1 which enables ccache's 'depend_only' strategy. This avoids extra spawning of the pre-processor during cache misses, with the downside that is it less likely ccache will find a cache hit after semantically benign compiler flag changes. This is the lesser of two evils, as otherwise we can't use ccache at all under msys and remain inside the job time limit. If people are finding ccache to hurt their pipelines, it can be disabled by setting the 'CCACHE_DISABLE=1' env variable against their gitlab fork CI settings. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Message-Id: <20230804111054.281802-2-berrange@redhat.com> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20230829161528.2707696-2-alex.bennee@linaro.org>
2023-08-29 19:15:17 +03:00
- $env:CCACHE_BASEDIR = "$env:CI_PROJECT_DIR"
- $env:CCACHE_DIR = "$env:CCACHE_BASEDIR/ccache"
- $env:CCACHE_MAXSIZE = "500M"
- $env:CCACHE_DEPEND = 1 # cache misses are too expensive with preprocessor mode
- $env:CC = "ccache gcc"
- mkdir build
- cd build
gitlab: enable ccache for many build jobs The `ccache` tool can be very effective at reducing compilation times when re-running pipelines with only minor changes each time. For example a fresh 'build-system-fedora' job will typically take 20 minutes on the gitlab.com shared runners. With ccache this is reduced to as little as 6 minutes. Normally meson would auto-detect existance of ccache in $PATH and use it automatically, but the way we wrap meson from configure breaks this, as we're passing in an config file with explicitly set compiler paths. Thus we need to add $CCACHE_WRAPPERSPATH to the front of $PATH. For unknown reasons if doing this in msys though, gcc becomes unable to invoke 'cc1' when run from meson. For msys we thus set CC='ccache gcc' before invoking 'configure' instead. A second problem with msys is that cache misses are incredibly expensive, so enabling ccache massively slows down the build when the cache isn't well populated. This is suspected to be a result of the cost of spawning processes under the msys architecture. To deal with this we set CCACHE_DEPEND=1 which enables ccache's 'depend_only' strategy. This avoids extra spawning of the pre-processor during cache misses, with the downside that is it less likely ccache will find a cache hit after semantically benign compiler flag changes. This is the lesser of two evils, as otherwise we can't use ccache at all under msys and remain inside the job time limit. If people are finding ccache to hurt their pipelines, it can be disabled by setting the 'CCACHE_DISABLE=1' env variable against their gitlab fork CI settings. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Message-Id: <20230804111054.281802-2-berrange@redhat.com> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20230829161528.2707696-2-alex.bennee@linaro.org>
2023-08-29 19:15:17 +03:00
- ..\msys64\usr\bin\bash -lc "ccache --zero-stats"
- ..\msys64\usr\bin\bash -lc "../configure --enable-fdt=system $CONFIGURE_ARGS"
- ..\msys64\usr\bin\bash -lc "make"
- ..\msys64\usr\bin\bash -lc "make check MTESTARGS='$TEST_ARGS' || { cat meson-logs/testlog.txt; exit 1; } ;"
gitlab: enable ccache for many build jobs The `ccache` tool can be very effective at reducing compilation times when re-running pipelines with only minor changes each time. For example a fresh 'build-system-fedora' job will typically take 20 minutes on the gitlab.com shared runners. With ccache this is reduced to as little as 6 minutes. Normally meson would auto-detect existance of ccache in $PATH and use it automatically, but the way we wrap meson from configure breaks this, as we're passing in an config file with explicitly set compiler paths. Thus we need to add $CCACHE_WRAPPERSPATH to the front of $PATH. For unknown reasons if doing this in msys though, gcc becomes unable to invoke 'cc1' when run from meson. For msys we thus set CC='ccache gcc' before invoking 'configure' instead. A second problem with msys is that cache misses are incredibly expensive, so enabling ccache massively slows down the build when the cache isn't well populated. This is suspected to be a result of the cost of spawning processes under the msys architecture. To deal with this we set CCACHE_DEPEND=1 which enables ccache's 'depend_only' strategy. This avoids extra spawning of the pre-processor during cache misses, with the downside that is it less likely ccache will find a cache hit after semantically benign compiler flag changes. This is the lesser of two evils, as otherwise we can't use ccache at all under msys and remain inside the job time limit. If people are finding ccache to hurt their pipelines, it can be disabled by setting the 'CCACHE_DISABLE=1' env variable against their gitlab fork CI settings. Signed-off-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Message-Id: <20230804111054.281802-2-berrange@redhat.com> Signed-off-by: Alex Bennée <alex.bennee@linaro.org> Message-Id: <20230829161528.2707696-2-alex.bennee@linaro.org>
2023-08-29 19:15:17 +03:00
- ..\msys64\usr\bin\bash -lc "ccache --show-stats"
- Write-Output "Finished build at $(Get-Date -Format u)"
msys2-64bit:
extends: .shared_msys2_builder
variables:
MINGW_TARGET: mingw-w64-x86_64
MSYSTEM: MINGW64
# do not remove "--without-default-devices"!
# commit 9f8e6cad65a6 ("gitlab-ci: Speed up the msys2-64bit job by using --without-default-devices"
# changed to compile QEMU with the --without-default-devices switch
# for the msys2 64-bit job, due to the build could not complete within
CONFIGURE_ARGS: --target-list=x86_64-softmmu --without-default-devices -Ddebug=false -Doptimization=0
# qTests don't run successfully with "--without-default-devices",
# so let's exclude the qtests from CI for now.
TEST_ARGS: --no-suite qtest
msys2-32bit:
extends: .shared_msys2_builder
variables:
MINGW_TARGET: mingw-w64-i686
MSYSTEM: MINGW32
CONFIGURE_ARGS: --target-list=ppc64-softmmu -Ddebug=false -Doptimization=0
TEST_ARGS: --no-suite qtest