- tests/vm support for aarch64 VMs
- tests/tcg better cross-compiler detection
- update docker tooling to support registries
- update docker support for xtensa
- gitlab build docker images and store in registry
- gitlab use docker images for builds
- a number of skipIf updates to support move
- linux-user MAP_FIXED_NOREPLACE fix
- qht-bench compiler tweaks
- configure fix for secret keyring
- tsan fiber annotation clean-up
- doc updates for mttcg/icount/gdbstub
- fix cirrus to use brew bash for iotests
- revert virtio-gpu breakage
- fix LC_ALL to avoid sorting changes in iotests
-----BEGIN PGP SIGNATURE-----
iQEzBAABCgAdFiEEZoWumedRZ7yvyN81+9DbCVqeKkQFAl8J0yoACgkQ+9DbCVqe
KkSzTAf/Vn+9TU8Qt7nZvl7W4tz7Sy5K8EJGwj2RXx6CWWWLiFbsXurIM8Krw5Vc
RmvUxwa359b+J0lQpfeNDHYm1nM8RZLFlkG0a5bl0I8sW0EcPjBRtwNaGKXh2p0u
u2RS2QAi6A9AvYT4ZREYlBM+o9WzbxCEQm4s8fr6WEJCQfxBnb5/bGiEjWR64e8C
j9Kvou+zAKfVizbQMtu+mwqjsoPtcS1b3vVcO7anhNuUsuaEKkS0dFWzWvw3lwJR
STIYnb8Y/eJ1yKr0hPH2qtWv3n6yhlYvYmpUCH6AwshGMUoeFEzR2VoWS6yZPGG6
na6XA3UW5R9AxIDfkCJ5ueeo8t9xMQ==
=HRWa
-----END PGP SIGNATURE-----
Merge remote-tracking branch 'remotes/stsquad/tags/pull-testing-and-misc-110720-2' into staging
Testing and misc build updates:
- tests/vm support for aarch64 VMs
- tests/tcg better cross-compiler detection
- update docker tooling to support registries
- update docker support for xtensa
- gitlab build docker images and store in registry
- gitlab use docker images for builds
- a number of skipIf updates to support move
- linux-user MAP_FIXED_NOREPLACE fix
- qht-bench compiler tweaks
- configure fix for secret keyring
- tsan fiber annotation clean-up
- doc updates for mttcg/icount/gdbstub
- fix cirrus to use brew bash for iotests
- revert virtio-gpu breakage
- fix LC_ALL to avoid sorting changes in iotests
# gpg: Signature made Sat 11 Jul 2020 15:56:42 BST
# gpg: using RSA key 6685AE99E75167BCAFC8DF35FBD0DB095A9E2A44
# gpg: Good signature from "Alex Bennée (Master Work Key) <alex.bennee@linaro.org>" [full]
# Primary key fingerprint: 6685 AE99 E751 67BC AFC8 DF35 FBD0 DB09 5A9E 2A44
* remotes/stsquad/tags/pull-testing-and-misc-110720-2: (50 commits)
iotests: Set LC_ALL=C for sort
Revert "vga: build virtio-gpu as module"
tests: fix "make check-qtest" for modular builds
.cirrus.yml: add bash to the brew packages
tests/docker: update toolchain set in debian-xtensa-cross
tests/docker: fall back more gracefully when pull fails
docs: Add to gdbstub documentation the PhyMemMode
docs/devel: add some notes on tcg-icount for developers
docs/devel: convert and update MTTCG design document
tests/qht-bench: Adjust threshold computation
tests/qht-bench: Adjust testing rate by -1
travis.yml: Test also the other targets on s390x
shippable: pull images from registry instead of building
testing: add check-build target
containers.yml: build with docker.py tooling
gitlab: limit re-builds of the containers
tests: improve performance of device-introspect-test
gitlab: add avocado asset caching
gitlab: enable check-tcg for linux-user tests
linux-user/elfload: use MAP_FIXED_NOREPLACE in pgb_reserved_va
...
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Michael started to work on the AVR port few years ago [*] and kept
improving the code over various series.
List of people who help him (in chronological order):
- Richard Henderson
- Sarah Harris and Edward Robbins
- Philippe Mathieu-Daudé and Aleksandar Markovic
- Pavel Dovgalyuk
- Thomas Huth
[*] The oldest contribution I could find on the list is from 2016:
https://lists.nongnu.org/archive/html/qemu-devel/2016-06/msg02985.html
Tests included:
$ avocado --show=app run -t arch:avr tests/acceptance/
Fetching asset from tests/acceptance/machine_avr6.py:AVR6Machine.test_freertos
(1/1) tests/acceptance/machine_avr6.py:AVR6Machine.test_freertos: PASS (2.13 s)
RESULTS : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB TIME : 2.35 s
$ make check-qtest-avr
TEST check-qtest-avr: tests/qtest/boot-serial-test
TEST check-qtest-avr: tests/qtest/cdrom-test
TEST check-qtest-avr: tests/qtest/device-introspect-test
TEST check-qtest-avr: tests/qtest/machine-none-test
TEST check-qtest-avr: tests/qtest/qmp-test
TEST check-qtest-avr: tests/qtest/qmp-cmd-test
TEST check-qtest-avr: tests/qtest/qom-test
TEST check-qtest-avr: tests/qtest/test-hmp
TEST check-qtest-avr: tests/qtest/qos-test
CI results:
. https://cirrus-ci.com/build/5697049146425344
. https://gitlab.com/philmd/qemu/-/pipelines/165328058
. https://travis-ci.org/github/philmd/qemu/builds/705817933
. https://app.shippable.com/github/philmd/qemu/runs/822/summary/console
-----BEGIN PGP SIGNATURE-----
iQIzBAABCAAdFiEE+qvnXhKRciHc/Wuy4+MsLN6twN4FAl8JgE8ACgkQ4+MsLN6t
wN5KMhAA1IivMK9uD9x8vOK3H1fSIju6ufJz7mlynDXG/LV0dhms1t8n/AlPE/Gk
TLhBUBwwdsejVDrTu6GGukoybsKLyULdt8MIK5z6bd++TwY3MlUkdOJeteviBUuP
0hcagNR/Dyg1WLAq/VH4KGHfzisprfDM7sXTvjE3raKBSpqIwO5tfUn4kVm/LWB9
sQNsVbtyKqnM3UW+QYGAN3eGAOM2SWx8pUZqV+UrDDEDoFJIiip7jxoN1t7PJaQp
O6t2/omLzbOMrpwqmNAIfrsMjovRylrd8nDGlX/OF5SrEbwXi3qvdJBtEOdBPasp
owXbu2Uwo4VUu5x7kzAiTlflBBSOmDpILbYVn5jGLKMZmOjLTPVbrzMKGZqx5GXD
gWmmX2aD8ejl2XGmKM+gC1smQJ6/aMTILoYXq97hIKi8pMH7AB2a8Tmzseiqx/E/
Lz1DrrnIW5vwFQAPnhdJCU1GF3B9VUcHG3w0sjvgGKDfpe8tLEgkmISi7CUbbA9/
rJs2P24haqfdFXWLQU2sO1ygTR1vLNy5/ZbU1nyrAPpjWnGeX6GXVaWGQo83BaDd
rIfnx0upNYFdaO5Vi4cbHUFGe1fVuR3C/l1xiDbmnzx3yMgys3036Equ2h3fSPqO
4k79bee9ByAUl1YE0X8T4gasCRS6RaSWXZYz7lfdc9h5hGny8LM=
=gFE5
-----END PGP SIGNATURE-----
Merge remote-tracking branch 'remotes/philmd-gitlab/tags/avr-port-20200711' into staging
8bit AVR port from Michael Rolnik.
Michael started to work on the AVR port few years ago [*] and kept
improving the code over various series.
List of people who help him (in chronological order):
- Richard Henderson
- Sarah Harris and Edward Robbins
- Philippe Mathieu-Daudé and Aleksandar Markovic
- Pavel Dovgalyuk
- Thomas Huth
[*] The oldest contribution I could find on the list is from 2016:
https://lists.nongnu.org/archive/html/qemu-devel/2016-06/msg02985.html
Tests included:
$ avocado --show=app run -t arch:avr tests/acceptance/
Fetching asset from tests/acceptance/machine_avr6.py:AVR6Machine.test_freertos
(1/1) tests/acceptance/machine_avr6.py:AVR6Machine.test_freertos: PASS (2.13 s)
RESULTS : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB TIME : 2.35 s
$ make check-qtest-avr
TEST check-qtest-avr: tests/qtest/boot-serial-test
TEST check-qtest-avr: tests/qtest/cdrom-test
TEST check-qtest-avr: tests/qtest/device-introspect-test
TEST check-qtest-avr: tests/qtest/machine-none-test
TEST check-qtest-avr: tests/qtest/qmp-test
TEST check-qtest-avr: tests/qtest/qmp-cmd-test
TEST check-qtest-avr: tests/qtest/qom-test
TEST check-qtest-avr: tests/qtest/test-hmp
TEST check-qtest-avr: tests/qtest/qos-test
CI results:
. https://cirrus-ci.com/build/5697049146425344
. https://gitlab.com/philmd/qemu/-/pipelines/165328058
. https://travis-ci.org/github/philmd/qemu/builds/705817933
. https://app.shippable.com/github/philmd/qemu/runs/822/summary/console
# gpg: Signature made Sat 11 Jul 2020 10:03:11 BST
# gpg: using RSA key FAABE75E12917221DCFD6BB2E3E32C2CDEADC0DE
# gpg: Good signature from "Philippe Mathieu-Daudé (F4BUG) <f4bug@amsat.org>" [full]
# Primary key fingerprint: FAAB E75E 1291 7221 DCFD 6BB2 E3E3 2C2C DEAD C0DE
* remotes/philmd-gitlab/tags/avr-port-20200711: (32 commits)
target/avr/disas: Fix store instructions display order
target/avr/cpu: Fix $PC displayed address
target/avr/cpu: Drop tlb_flush() in avr_cpu_reset()
target/avr: Add section into QEMU documentation
tests/acceptance: Test the Arduino MEGA2560 board
tests/boot-serial: Test some Arduino boards (AVR based)
hw/avr: Add limited support for some Arduino boards
hw/avr: Add some ATmega microcontrollers
hw/avr: Add support for loading ELF/raw binaries
hw/misc: avr: Add limited support for power reduction device
hw/timer: avr: Add limited support for 16-bit timer peripheral
hw/char: avr: Add limited support for USART peripheral
tests/machine-none: Add AVR support
target/avr: Register AVR support with the rest of QEMU
target/avr: Add support for disassembling via option '-d in_asm'
target/avr: Initialize TCG register variables
target/avr: Add instruction translation - CPU main translation function
target/avr: Add instruction translation - MCU Control Instructions
target/avr: Add instruction translation - Bit and Bit-test Instructions
target/avr: Add instruction translation - Data Transfer Instructions
...
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Otherwise the result is basically unpredictable.
(Note that the precise environment variable to control sorting order is
LC_COLLATE, but LC_ALL overrides LC_COLLATE, and we do not want the
sorting order to be messed up if LC_ALL is set in the environment.)
Reported-by: John Snow <jsnow@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200710163253.381630-3-mreitz@redhat.com>
This reverts commit 8d5a24c83d.
Compiling all virtio-gpu objects into a single module isn't a good plan
because the individual objects have different CONFIG_* dependencies.
Leads to module load failures on s390x due to vga support being
disabled, which in turn breaks '-device virtio-gpu-device' (flagged by
travis ci).
So back to the drawing board for modular virtio-gpu ...
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200710203652.9708-3-kraxel@redhat.com>
Like the sed we include earlier we want something more recent for
iotests to work.
Fixes: 57ee95ed
Cc: Max Reitz <mreitz@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200710182238.10675-1-alex.bennee@linaro.org>
Switch to the prebuilt xtensa toolchains release 2020.07.
Drop csp toolchain as the csp core is not a part of QEMU.
Add de233_fpu and dsp3400 toolchains to enable DFPU and FPU2000 tests.
Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
[AJB: fix path in configure.sh]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200708082347.27318-1-jcmvbkbc@gmail.com>
Message-Id: <20200709141327.14631-13-alex.bennee@linaro.org>
I only spotted this in the small window between my testing with my
registry while waiting for the gitlab PR to go in. As we pre-pull the
registry image we know if that fails there isn't any point attempting
to use the cache. Fall back to the way we used to do it at that point.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20200709141327.14631-12-alex.bennee@linaro.org>
The PhyMemMode gdb extension command was missing from the gdb.rst
document.
Signed-off-by: Jon Doron <arilou@gmail.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20200601171609.1665397-1-arilou@gmail.com>
Message-Id: <20200709141327.14631-4-alex.bennee@linaro.org>
This attempts to bring together my understanding of the requirements
for icount behaviour into one reference document for our developer
notes.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Pavel Dovgalyuk <dovgaluk@ispras.ru>
Cc: Peter Maydell <peter.maydell@linaro.org>
Message-Id: <20200709141327.14631-3-alex.bennee@linaro.org>
Do a light conversion to .rst and clean-up some of the language at the
start now MTTCG has been merged for a while.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20200709141327.14631-2-alex.bennee@linaro.org>
In 06c4cc3660, we split the multiplication in two parts to avoid
a clang warning. But because double still rounds to 53 bits, this
does not provide additional precision beyond multiplication by
nextafter(0x1p64, 0), the largest representable value smaller
than 2**64.
However, since we have eliminated 1.0, mutiplying by 2**64 produces
a better distribution of input values to the output values.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200626200950.1015121-3-richard.henderson@linaro.org>
Since the seed must be non-zero, subtracting 1 means puts the
rate in 0..UINT64_MAX-1, which allows the 0 and UINT64_MAX
thresholds to corrspond to 0% (never) and 100% (always).
Suggested-by: Emilio G. Cota <cota@braap.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200626200950.1015121-2-richard.henderson@linaro.org>
s390x is our only big endian host in our CI, so building and testing QEMU
there is quite valuable. Thus let's also test the other targets with
additional jobs (also using different sets of pre-installed libraries to
get a better coverage of the things that we test).
Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Laurent Vivier <lvivier@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200608114049.4693-1-thuth@redhat.com>
Now we have a source for registry images pull from there rather than
re-building every time on shippable.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200701135652.1366-41-alex.bennee@linaro.org>
If we want to continue to split build and check phase it seems like a
good idea to allow building of the tests during our multi-threaded
build phase.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Cc: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20200701135652.1366-40-alex.bennee@linaro.org>
Instead of building the docker files directly use the same docker.py
scripting as we do for building locally. This should help ensure we
use the exact same steps and allow us to cache properly when building
locally.
To get this working you have to have a fairly recent docker binary
otherwise you will see the error message:
=> ERROR importing cache manifest from registry.gitlab....
So far docker 19.03.12 works (from the docker apt repos) but 18.09.1,
build 4c52b90 which is packaged in Debian Buster fails.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200701135652.1366-39-alex.bennee@linaro.org>
Most of the time we are just rebuilding the same things. We can skip
this although currently there is no mechanism for picking up new
distro releases.
Rather than try to be too fine grained allow any change to trigger all
the images being rebuilt.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200701135652.1366-38-alex.bennee@linaro.org>
Total execution time with "-m slow" and x86_64 QEMU, drops from 3
minutes 15 seconds, down to 54 seconds.
Individual tests drop from 17-20 seconds, down to 3-4 seconds.
The cost of this change is that any QOM bugs resulting in the test
failure will not be directly associated with the device that caused
the failure. The test case is not frequently identifying such bugs
though, and the cause is likely easily visible in the patch series
that causes the failure. So overall the shorter running time is
considered the more important factor.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
[thuth: Add the tree check to test_device_intro_none() and
test_device_intro_abstract(), too, just to be sure...]
Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Laurent Vivier <lvivier@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200710060719.22386-1-thuth@redhat.com>
These can be quite big so lets cache them. I couldn't find any nots on
ccache in the gitlab docs so I've just ignored it for now.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200701135652.1366-36-alex.bennee@linaro.org>
Switch to building in the new debian-all-test-cross image which has
most of the cross compilers inline.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20200701135652.1366-35-alex.bennee@linaro.org>
Given we assert the requested address matches what we asked we should
also make that clear in the mmap flags. Otherwise we see failures in
the GitLab environment for some currently unknown but allowable
reason. We use MAP_FIXED_NOREPLACE if we can so we don't just clobber
an existing mapping. Also include the strerror string for a bit more
info on failure.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200701135652.1366-34-alex.bennee@linaro.org>
We happily use all the cross images for both cross-building QEMU as
well as building the linux-user tests. However calling docker from
within docker seems not to work. As we can build in Debian anyway why
not include an image that has all the compilers available for
non-docker invocation.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200701135652.1366-33-alex.bennee@linaro.org>
We were missing a bunch of compilers which we could use if they were
locally installed. The defaults are based on Debian as they seem to be
the best distro for well distributed cross-build compilers.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200701135652.1366-32-alex.bennee@linaro.org>
As part of migrating things from Travis to GitLab add the acceptance
tests. To do this:
- rename system1 to system-ubuntu-main
- rename system2 to system-fedora-misc
- split into build/check/acceptance
- remove -j from check stages
- use artifacts to save build stage
- add post acceptance template and use
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200701135652.1366-31-alex.bennee@linaro.org>
This fails on GitLab but not when run locally on the same container
image. It's very confusing.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20200701135652.1366-30-alex.bennee@linaro.org>
For some reason these tests fail all the time on GitLab. I can
re-create the hang around 3% of the time locally but it doesn't seem
to be MTTCG related. For now skipIf on GITLAB_CI.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Cc: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>
Message-Id: <20200701135652.1366-29-alex.bennee@linaro.org>
The old path doesn't exist but the rx-virt.dtb file has the same
checksum so lets use that.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20200701135652.1366-28-alex.bennee@linaro.org>
Currently the test takes more the 900 seconds on GitLab and then times
out. Running on Travis seems to be OK.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200701135652.1366-27-alex.bennee@linaro.org>
We need additional python packages to run check-acceptance. Add them
to the docker images we will be using later.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20200701135652.1366-26-alex.bennee@linaro.org>
This allows us to point the tools towards a registry from which they
can grab pre-built layers instead of doing everything from scratch
each time. To enable this we need to be using the DOCKER_BUILDKIT
engine.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200701135652.1366-25-alex.bennee@linaro.org>
According to the documentation to be able to use --cache-from for
remote registries you need to enable both buildkit and inline the
metadata. We want to do this to support pulling from gitlab when users
build their local docker images.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200701135652.1366-24-alex.bennee@linaro.org>
Now that we're building standard container images from
dockerfiles in tests/docker/dockerfiles, we can convert
the build jobs to use them. The key benefit of this is
that a contributor can now more easily replicate the CI
environment on their local machine. The container images
are cached too, so we are not spending time waiting for
the apt-get/dnf package installs to complete.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20200622153318.751107-4-berrange@redhat.com>
[AJB: tweak naming convention]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20200701135652.1366-23-alex.bennee@linaro.org>
We have a number of container images in tests/docker/dockerfiles
that are intended to provide well defined environments for doing
test builds. We want our CI system to use these containers too.
This introduces builds of all of them as the first stage in the
CI, so that the built containers are available for later build
jobs. The containers are setup to use the GitLab container
registry as the cache, so we only pay the penalty of the full
build when the dockerfiles change. The main qemu-project/qemu
repo is used as a second cache, so that users forking QEMU will
see a fast turnaround time on their CI jobs.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20200622153318.751107-3-berrange@redhat.com>
[AJB: tweak the tag format]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20200701135652.1366-22-alex.bennee@linaro.org>
If no stage is listed, jobs get put in an implicit "test" stage.
Some jobs which create container images to be used by later stages
are currently listed as in a "build" stages.
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Acked-by: Laszlo Ersek <lersek@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200622153318.751107-2-berrange@redhat.com>
Message-Id: <20200701135652.1366-21-alex.bennee@linaro.org>
The edk2.yml and opensbi.yml files have recently been moved/renamed,
but the change has not been reflected in the rules in the YML files
yet.
Fixes: 922febe2af ("Move edk2 and opensbi YAML files to .gitlab-ci.d folder")
Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Message-Id: <20200625151627.24986-1-thuth@redhat.com>
Message-Id: <20200701135652.1366-20-alex.bennee@linaro.org>
The sooner we deprecate in-tree builds the sooner this mess of regexes
can be thrown away.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200701135652.1366-19-alex.bennee@linaro.org>
We've been misusing the tag naming scheme for some time by overloading
the post : section with the image type. Really it should be saved for
the revision of that particular build. Move the details to the other
side so we have:
qemu/image-name
with the implied :latest version added by the tooling.
Suggested-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200701135652.1366-18-alex.bennee@linaro.org>
Reported-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Suggested-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200701135652.1366-17-alex.bennee@linaro.org>
We currently limit TCG guests to -smp 1 but now we have added some
aarch64 guests we can do better when running on x86_64 hardware.
Raise the limit for TCG guests when it is safe to do so.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Robert Foley <robert.foley@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200701135652.1366-16-alex.bennee@linaro.org>
optparse has been deprecated since version 3.2 and argparse is the
blessed replacement. Take the opportunity to enhance our help output
showing defaults when called.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Robert Foley <robert.foley@linaro.org>
Message-Id: <20200701135652.1366-15-alex.bennee@linaro.org>
This adds support to basevm.py so that we always
drain the console chars. This makes use of
support added in an earlier commit that allows
QEMUMachine to use the ConsoleSocket.
This is a workaround we found was needed since
there is a known issue where QEMU will hang waiting
for console characters to be consumed.
We also added the option of logging the console to a file.
LOG_CONSOLE=1 will now log the output to a file.
Signed-off-by: Robert Foley <robert.foley@linaro.org>
Reviewed-by: Peter Puhov <peter.puhov@linaro.org>
Acked-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200601211421.1277-10-robert.foley@linaro.org>
Message-Id: <20200701135652.1366-14-alex.bennee@linaro.org>
We add the ConsoleSocket object, which has a socket interface
and which will consume all arriving characters on the
socket, placing them into an in memory buffer.
This will also provide those chars via recv() as
would a regular socket.
ConsoleSocket also has the option of dumping
the console bytes to a log file.
We also give QEMUMachine the option of using ConsoleSocket
to drain and to use for logging console to a file.
By default QEMUMachine does not use ConsoleSocket.
This is added in preparation for use by basevm.py in a later commit.
This is a workaround we found was needed for basevm.py since
there is a known issue where QEMU will hang waiting
for console characters to be consumed.
Cc: Eduardo Habkost <ehabkost@redhat.com>
Cc: Cleber Rosa <crosa@redhat.com>
Signed-off-by: Robert Foley <robert.foley@linaro.org>
Reviewed-by: Peter Puhov <peter.puhov@linaro.org>
Acked-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200601211421.1277-9-robert.foley@linaro.org>
Message-Id: <20200701135652.1366-13-alex.bennee@linaro.org>
This change converts existing scripts to using for example self.ROOT_PASS,
to self._config['root_pass'].
We made similar changes for GUEST_USER, and GUEST_PASS.
This allows us also to remove the change in basevm.py,
which adds __getattr__ for backwards compatibility.
Signed-off-by: Robert Foley <robert.foley@linaro.org>
Reviewed-by: Peter Puhov <peter.puhov@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200601211421.1277-8-robert.foley@linaro.org>
Message-Id: <20200701135652.1366-12-alex.bennee@linaro.org>
centos.aarch64 creates a CentOS 8 image.
Also added a new kickstart script used to build the centos.aarch64 image.
Signed-off-by: Robert Foley <robert.foley@linaro.org>
Reviewed-by: Peter Puhov <peter.puhov@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200601211421.1277-7-robert.foley@linaro.org>
Message-Id: <20200701135652.1366-11-alex.bennee@linaro.org>
ubuntu.aarch64 provides a script to create an Ubuntu 18.04 VM.
Another new file is also added aarch64vm.py, which is a module with
common methods used by aarch64 VMs, such as how to create the
flash images.
Signed-off-by: Robert Foley <robert.foley@linaro.org>
Reviewed-by: Peter Puhov <peter.puhov@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200601211421.1277-6-robert.foley@linaro.org>
Message-Id: <20200701135652.1366-10-alex.bennee@linaro.org>
Add a common Ubuntu python module and make use of
it with the ubuntu.i386 script.
This is preparation for adding an Ubuntu script
ubuntu.aarch64. Splitting out the common
logic such as build_image() will reduce duplication.
Signed-off-by: Robert Foley <robert.foley@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200601211421.1277-5-robert.foley@linaro.org>
Message-Id: <20200701135652.1366-9-alex.bennee@linaro.org>
Changes to tests/vm/basevm.py to allow accepting a configuration file
as a parameter. Allows for specifying VM options such as
cpu, machine, memory, and arbitrary qemu arguments for specifying options
such as NUMA configuration.
Also added an example conf_example_aarch64.yml and conf_example_x86.yml.
Signed-off-by: Robert Foley <robert.foley@linaro.org>
Reviewed-by: Peter Puhov <peter.puhov@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200601211421.1277-4-robert.foley@linaro.org>
Message-Id: <20200701135652.1366-8-alex.bennee@linaro.org>
Added use of a configuration to tests/vm/basevm.py.
The configuration provides parameters used to configure a VM.
This allows for providing alternate configurations to the VM being
created/launched. cpu, machine, memory, and NUMA configuration are all
examples of configuration which we might want to vary on the VM being created
or launched.
This will for example allow for creating an aarch64 vm.
Signed-off-by: Robert Foley <robert.foley@linaro.org>
Reviewed-by: Peter Puhov <peter.puhov@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200601211421.1277-3-robert.foley@linaro.org>
Message-Id: <20200701135652.1366-7-alex.bennee@linaro.org>
