Commit Graph

529 Commits

Author SHA1 Message Date
Philippe Mathieu-Daudé
d2427956e1 tests/docker: Include 'ccache' in Debian base image
Include the 'ccache' package to speed up compilation.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20201213211601.253530-1-f4bug@amsat.org>
Fixes: d6db2a1cdf ("docker: add debian-buster-arm64-cross")
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2021-01-06 17:30:02 +00:00
Daniel P. Berrangé
90e0c9b309 tests: update for rename of CentOS8 PowerTools repo
This was intentionally renamed recently to be all lowercase:

https://bugs.centos.org/view.php?id=17920
https://wiki.centos.org/Manuals/ReleaseNotes/CentOS8.2011#Yum_repo_file_and_repoid_changes

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20201216141653.213980-1-berrange@redhat.com>
[AJB: bump up FROM to trigger re-build, add diffutils]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2021-01-02 21:03:09 +01:00
Thomas Huth
c9d78b06c0 tests/docker: Remove the remainders of debian9 containers from the Makefile
The Debian 9 containers have been removed a while ago, so we can
delete the corresponding entries in the Makefile, too.

Fixes: e3755276d1 ("tests/docker: Remove old Debian 9 containers")
Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20201215083318.92205-1-thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2021-01-02 21:03:09 +01:00
Thomas Huth
ee381b7fe1 gitlab-CI: Test 32-bit builds with the fedora-i386-cross container
After adding some missing packages, it's possible to check 32-bit
builds and tests with the fedora-i386-cross container in the gitlab-CI,
too. Unfortunately, the code in subprojects/ ignores the --extra-cflags
(on purpose), so the vhost-user part has to be disabled for this.

While we're at it, update the container to Fedora 31. Unfortunately the
gcc from the later versions emits some very dubious format-truncation
warnings, so Fedora 32 and 33 are currently unsuitable for this job.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20201215083451.92322-1-thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2021-01-02 21:03:09 +01:00
Peter Maydell
6cfdaa88cf Remove obsolete setuptools dependency and fix Stefan's
Win32 builds.
 -----BEGIN PGP SIGNATURE-----
 
 iQFIBAABCAAyFiEE8TM4V0tmI4mGbHaCv/vSX3jHroMFAl+/uegUHHBib256aW5p
 QHJlZGhhdC5jb20ACgkQv/vSX3jHroPMiwgAj6VB/lZP10CmKfNCIC6s9TjONP2I
 ruI3ZMYBrmeiLMf/WZTDI0kRY/mmYMtn4nN+RlEHlBE9a9hSCmHWvQyZDgEIJUOA
 1bTAdl4jFAbBv4knKVVuPSu/kMDHVIkwdktEEmPhkMHunIxrI7Yobe4r0GMXXZJm
 Vu4dNtskzERu/fc+lgbdMjRU+xlXT/zYHy98L5mu+VRgKgmtY/6aF1et4t+EfM9f
 tu3vd/WEOn/HiYvOsVj55SNSR6IiXTGq0LS5d4fI2Q9B477qoO+x3QJDzjbHBo+u
 u58QkbqJa5OfO0ZoUAi+SvOjE0EhJFPL5ihVODjkTSbORpfWz3ULyAK1nQ==
 =Mdqh
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/bonzini-gitlab/tags/for-upstream' into staging

Remove obsolete setuptools dependency and fix Stefan's
Win32 builds.

# gpg: Signature made Thu 26 Nov 2020 14:21:28 GMT
# gpg:                using RSA key F13338574B662389866C7682BFFBD25F78C7AE83
# gpg:                issuer "pbonzini@redhat.com"
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>" [full]
# gpg:                 aka "Paolo Bonzini <pbonzini@redhat.com>" [full]
# Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4  E2F7 7E15 100C CD36 69B1
#      Subkey fingerprint: F133 3857 4B66 2389 866C  7682 BFFB D25F 78C7 AE83

* remotes/bonzini-gitlab/tags/for-upstream:
  nsis: Fix build for 64 bit installer
  tests/docker, tests/vm: remove setuptools from images
  configure: remove python pkg_resources check
  meson: use dependency() to find libjpeg

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-11-26 14:25:47 +00:00
Paolo Bonzini
92ea027017 tests/docker, tests/vm: remove setuptools from images
Setuptools is not needed anymore by the bundled copy of meson,
remove it.

Suggested-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-11-25 12:50:48 -05:00
Philippe Mathieu-Daudé
8e9419b790 tests/docker: Install liblttng-ust-dev package in Ubuntu 20.04 image
Install the liblttng-ust-dev package to be able to
build QEMU using the User-Space Tracer trace backend
(configure --enable-trace-backends=ust).

Suggested-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20201111121234.3246812-2-philmd@redhat.com>
Message-Id: <20201117173635.29101-6-alex.bennee@linaro.org>
2020-11-23 09:55:20 +00:00
Thomas Huth
5ed7ca3185 tests/docker/dockerfiles/centos: Use SDL2 instead of SDL1
We do not support SDL1 in QEMU anymore. Use SDL2 instead.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20201021072308.9224-1-thuth@redhat.com>
Message-Id: <20201021163136.27324-4-alex.bennee@linaro.org>
2020-10-27 09:53:47 +00:00
Thomas Huth
c4cb1c9f2e test/docker/dockerfiles: Add missing packages for acceptance tests
Some of the "check-acceptance" tests are still skipped in the CI
since the docker images do not provide the necessary packages, e.g.
the netcat binary. Add them to get more test coverage.

Message-Id: <20201023073351.251332-5-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-10-26 09:54:16 +01:00
Paolo Bonzini
345d7053ca add ninja to dockerfiles, CI configurations and test VMs
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Acked-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-10-17 10:45:50 -04:00
Paolo Bonzini
f2f984a3b3 dockerfiles: enable Centos 8 PowerTools
ninja is included in the CentOS PowerTools repository.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-10-17 10:45:50 -04:00
Thomas Huth
7e86e5d5cc tests/docker: Add genisoimage to the docker file
genisoimage is needed for running the tests/qtest/cdrom-test qtest.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20201006174347.152040-1-thuth@redhat.com>
Message-Id: <20201007160038.26953-7-alex.bennee@linaro.org>
2020-10-09 17:27:55 +01:00
Paolo Bonzini
d72d6dcb0d dockerfiles: add diffutils to Fedora
For some reason diffutils is not included in the Fedora containers anymore,
causing the build to fail.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Neal Gompa <ngompa13@gmail.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20201003085054.332992-2-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-10-05 09:14:19 +02:00
Thomas Huth
e3755276d1 tests/docker: Remove old Debian 9 containers
We do not support Debian 9 in QEMU anymore, and the Debian 9 containers
are now no longer used in the gitlab-CI. Time to remove them.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20200921174320.46062-6-thuth@redhat.com>
Message-Id: <20200925154027.12672-14-alex.bennee@linaro.org>
2020-10-02 12:29:15 +01:00
Thomas Huth
cf63177e59 tests/docker: Update the tricore container to debian 10
We do not support Debian 9 anymore, thus update the Tricore container
to Debian 10 now.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20200921174320.46062-4-thuth@redhat.com>
Message-Id: <20200925154027.12672-12-alex.bennee@linaro.org>
2020-10-02 12:29:08 +01:00
Thomas Huth
93cc0506f6 tests/docker: Use Fedora containers for MinGW cross-builds in the gitlab-CI
According to our support policy, we do not support Debian 9 in QEMU
anymore, and we only support building the Windows binaries with a
very recent version of the MinGW toolchain. So we should not test
the MinGW cross-compilation with Debian 9 anymore, but switch to
something newer like Fedora. To do this, we need a separate Fedora
container for each build that provides the QEMU_CONFIGURE_OPTS
environment variable.
Unfortunately, the MinGW 64-bit compiler seems to be a little bit
slow, so we also have to disable some features like "capstone" in the
build here to make sure that the CI pipelines still finish within a
reasonable amount of time.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200921174320.46062-2-thuth@redhat.com>
Message-Id: <20200925154027.12672-10-alex.bennee@linaro.org>
2020-10-02 12:29:02 +01:00
Peter Maydell
7cd77fb02b Remove texinfo dependency from docker and CI configs
We don't need texinfo to build the docs any more, so we can
drop that dependency from our docker and other CI configs.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Message-Id: <20200925162316.21205-22-peter.maydell@linaro.org>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2020-09-29 17:55:39 +02:00
Paolo Bonzini
17cd6e2bbf docker.py: always use --rm
Avoid that containers pile up.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Acked-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-09-18 10:44:47 +01:00
Paolo Bonzini
660f793093 Makefile: inline the relevant parts of rules.mak
Most of rules.mak is not used anymore, just inline what's needed.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-09-08 11:43:16 +02:00
Daniel P. Berrangé
affcc09c2d gitlab: expand test coverage for crypto builds
Most jobs test the latest nettle library. This adds explicit coverage
for latest gcrypt using Fedora, and old gcrypt and nettle using
CentOS-7. The latter does a minimal tools-only build, as we only need to
validate that the crypto code builds and unit tests pass. Finally a job
disabling both nettle and gcrypt is provided to validate that gnutls
still works.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20200901133050.381844-3-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-09-07 12:34:17 +02:00
Thomas Huth
d3dd34a1e5 dockerfiles/debian-win64-cross: Download WHPX MinGW headers
To compile-test the WHPX accelerator, we need to download these system
headers first (they are unfortunately not part of any released and
packaged MinGW toolchain yet).

Idea taken from another patch by Stefan Weil.

Message-Id: <20200804170055.2851-12-thuth@redhat.com>
Message-Id: <20200823111757.72002-6-thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-09-07 12:34:17 +02:00
Alex Bennée
90843750e3 tests/docker: add python3-setuptools the docker images
We need these now for builds to work.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200903112107.27367-4-alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-09-06 20:26:43 +02:00
Paolo Bonzini
b861f59593 tests/docker: add test script for static linux-user builds
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-08-21 06:18:35 -04:00
Thomas Huth
1caac1c0e4 tests/docker: Add python3-venv and netcat to the debian-amd64 container
Without python3-venv, I get the following message when trying to
run the acceptance tests within the debian container:

 The virtual environment was not created successfully because ensurepip is not
 available.  On Debian/Ubuntu systems, you need to install the python3-venv
 package using the following command.
    apt-get install python3-venv
 You may need to use sudo with that command.  After installing the python3-venv
 package, recreate your virtual environment.

Let's do it as the message suggests.

And while we're at it, also add netcat here since it is required for
some of the acceptance tests.

Message-Id: <20200730141326.8260-2-thuth@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-08-05 10:37:42 +02:00
Alex Bennée
4c5aeb1238 tests/docker: add support for DEB_KEYRING
For installing stuff from sid or ports you may need to manually
specify the location of the keyring. You can even import keys into
your personal keyring and point it there, e.g.:

  gpg --keyserver keyring.debian.org --recv-keys 84C573CD4E1AFD6C
  make docker-binfmt-image-debian-sid-hppa DEB_TYPE=sid DEB_ARCH=hppa \
      DEB_URL=http://ftp.ports.debian.org/debian-ports/ \
      EXECUTABLE=./hppa-linux-user/qemu-hppa V=1 \
      DEB_KEYRING=${HOME}/.gnupg/pubring.kbx

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20200724064509.331-12-alex.bennee@linaro.org>
2020-07-27 09:41:35 +01:00
Alex Bennée
182ec06076 tests/docker: fix binfmt_misc image building
When we updated the arguments for docker.py we missed a bit.

Fixes: dfae628459 ("docker.py/build: support -t and -f arguments")
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20200724064509.331-11-alex.bennee@linaro.org>
2020-07-27 09:41:22 +01:00
Alex Bennée
e336cec3a5 tests/docker: fix update command due to python3 str/bytes distinction
Does this seem convoluted to you? It feels a little complicated to me.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200724064509.331-10-alex.bennee@linaro.org>
2020-07-27 09:41:18 +01:00
Alex Bennée
f73e485285 docker.py: fix fetching of FROM layers
This worked on a system that was already bootstrapped because the
stage 2 images already existed even if they wouldn't be used. What we
should have pulled down was the FROM line containers first because
building on gitlab doesn't have the advantage of using our build
system to build the pre-requisite bits.

We still pull the image we want to build just in case we can use the
cached data.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20200713200415.26214-4-alex.bennee@linaro.org>
2020-07-15 11:52:43 +01:00
Thomas Huth
a77312e77c tests/docker: Remove the libssh workaround from the ubuntu 20.04 image
The libssh problem only exists in Ubuntu 18.04 - we can enable it
in 20.04 again.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20200713185237.9419-1-thuth@redhat.com>
Message-Id: <20200713200415.26214-3-alex.bennee@linaro.org>
2020-07-15 11:52:43 +01:00
Max Filippov
bf7801df66 tests/docker: update toolchain set in debian-xtensa-cross
Switch to the prebuilt xtensa toolchains release 2020.07.
Drop csp toolchain as the csp core is not a part of QEMU.
Add de233_fpu and dsp3400 toolchains to enable DFPU and FPU2000 tests.

Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
[AJB: fix path in configure.sh]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200708082347.27318-1-jcmvbkbc@gmail.com>
Message-Id: <20200709141327.14631-13-alex.bennee@linaro.org>
2020-07-11 15:53:09 +01:00
Alex Bennée
85c5e69923 tests/docker: fall back more gracefully when pull fails
I only spotted this in the small window between my testing with my
registry while waiting for the gitlab PR to go in. As we pre-pull the
registry image we know if that fails there isn't any point attempting
to use the cache. Fall back to the way we used to do it at that point.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20200709141327.14631-12-alex.bennee@linaro.org>
2020-07-11 15:53:00 +01:00
Alex Bennée
27ebeda0c0 tests/docker: add a linux-user testing focused image
We happily use all the cross images for both cross-building QEMU as
well as building the linux-user tests. However calling docker from
within docker seems not to work. As we can build in Debian anyway why
not include an image that has all the compilers available for
non-docker invocation.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200701135652.1366-33-alex.bennee@linaro.org>
2020-07-11 15:53:00 +01:00
Alex Bennée
cdc61815c7 tests/docker: add packages needed for check-acceptance
We need additional python packages to run check-acceptance. Add them
to the docker images we will be using later.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20200701135652.1366-26-alex.bennee@linaro.org>
2020-07-11 15:53:00 +01:00
Alex Bennée
e6f1306b10 tests/docker: add --registry support to tooling
This allows us to point the tools towards a registry from which they
can grab pre-built layers instead of doing everything from scratch
each time. To enable this we need to be using the DOCKER_BUILDKIT
engine.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200701135652.1366-25-alex.bennee@linaro.org>
2020-07-11 15:53:00 +01:00
Alex Bennée
767b6bd22b tests/docker: change tag naming scheme of our images
We've been misusing the tag naming scheme for some time by overloading
the post : section with the image type. Really it should be saved for
the revision of that particular build. Move the details to the other
side so we have:

  qemu/image-name

with the implied :latest version added by the tooling.

Suggested-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>

Message-Id: <20200701135652.1366-18-alex.bennee@linaro.org>
2020-07-11 15:53:00 +01:00
Alex Bennée
10c927dcc5 tests/docker: check for an parameters not empty string
Reported-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Suggested-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>

Message-Id: <20200701135652.1366-17-alex.bennee@linaro.org>
2020-07-11 15:53:00 +01:00
Robert Foley
df79fd5667 tests/docker: Added docker build support for TSan.
Added a new docker for ubuntu 20.04.
This docker has support for Thread Sanitizer
including one patch we need in one of the header files.
https://github.com/llvm/llvm-project/commit/a72dc86cd

This command will build with tsan enabled:
make docker-test-tsan-ubuntu2004 V=1

Also added the TSAN suppresion file to disable certain
cases of TSAN warnings.

Cc: Fam Zheng <fam@euphon.net>
Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Robert Foley <robert.foley@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200609200738.445-10-robert.foley@linaro.org>
Message-Id: <20200612190237.30436-13-alex.bennee@linaro.org>
2020-06-16 14:49:05 +01:00
Alex Bennée
7ec1edde9e tests/docker: bump fedora to 32
We should be keeping this up to date as Fedora goes out of support
quite quickly.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20200612190237.30436-2-alex.bennee@linaro.org>
2020-06-16 14:49:02 +01:00
Peter Maydell
7d3660e798 * Miscellaneous fixes and feature enablement (many)
* SEV refactoring (David)
 * Hyper-V initial support (Jon)
 * i386 TCG fixes (x87 and SSE, Joseph)
 * vmport cleanup and improvements (Philippe, Liran)
 * Use-after-free with vCPU hot-unplug (Nengyuan)
 * run-coverity-scan improvements (myself)
 * Record/replay fixes (Pavel)
 * -machine kernel_irqchip=split improvements for INTx (Peter)
 * Code cleanups (Philippe)
 * Crash and security fixes (PJP)
 * HVF cleanups (Roman)
 -----BEGIN PGP SIGNATURE-----
 
 iQFIBAABCAAyFiEE8TM4V0tmI4mGbHaCv/vSX3jHroMFAl7jpdAUHHBib256aW5p
 QHJlZGhhdC5jb20ACgkQv/vSX3jHroMfjwf/X7+0euuE9dwKFKDDMmIi+4lRWnq7
 gSOyE1BYSfDIUXRIukf64konXe0VpiotNYlyEaYnnQjkMdGm5E9iXKF+LgEwXj/t
 NSGkfj5J3VeWRG4JJp642CSN/aZWO8uzkenld3myCnu6TicuN351tDJchiFwAk9f
 wsXtgLKd67zE8MLVt8AP0rNTbzMHttPXnPaOXDCuwjMHNvMEKnC93UeOeM0M4H5s
 3Dl2HvsNWZ2SzUG9mAbWp0bWWuoIb+Ep9//87HWANvb7Z8jratRws18i6tYt1sPx
 8zOnUS87sVnh1CQlXBDd9fEcqBUVgR9pAlqaaYavNhFp5eC31euvpDU8Iw==
 =F4sU
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into staging

* Miscellaneous fixes and feature enablement (many)
* SEV refactoring (David)
* Hyper-V initial support (Jon)
* i386 TCG fixes (x87 and SSE, Joseph)
* vmport cleanup and improvements (Philippe, Liran)
* Use-after-free with vCPU hot-unplug (Nengyuan)
* run-coverity-scan improvements (myself)
* Record/replay fixes (Pavel)
* -machine kernel_irqchip=split improvements for INTx (Peter)
* Code cleanups (Philippe)
* Crash and security fixes (PJP)
* HVF cleanups (Roman)

# gpg: Signature made Fri 12 Jun 2020 16:57:04 BST
# gpg:                using RSA key F13338574B662389866C7682BFFBD25F78C7AE83
# gpg:                issuer "pbonzini@redhat.com"
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>" [full]
# gpg:                 aka "Paolo Bonzini <pbonzini@redhat.com>" [full]
# Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4  E2F7 7E15 100C CD36 69B1
#      Subkey fingerprint: F133 3857 4B66 2389 866C  7682 BFFB D25F 78C7 AE83

* remotes/bonzini/tags/for-upstream: (116 commits)
  target/i386: Remove obsolete TODO file
  stubs: move Xen stubs to accel/
  replay: fix replay shutdown for console mode
  exec/cpu-common: Move MUSB specific typedefs to 'hw/usb/hcd-musb.h'
  hw/usb: Move device-specific declarations to new 'hcd-musb.h' header
  exec/memory: Remove unused MemoryRegionMmio type
  checkpatch: reversed logic with acpi test checks
  target/i386: sev: Unify SEVState and SevGuestState
  target/i386: sev: Remove redundant handle field
  target/i386: sev: Remove redundant policy field
  target/i386: sev: Remove redundant cbitpos and reduced_phys_bits fields
  target/i386: sev: Partial cleanup to sev_state global
  target/i386: sev: Embed SEVState in SevGuestState
  target/i386: sev: Rename QSevGuestInfo
  target/i386: sev: Move local structure definitions into .c file
  target/i386: sev: Remove unused QSevGuestInfoClass
  xen: fix build without pci passthrough
  i386: hvf: Drop HVFX86EmulatorState
  i386: hvf: Move mmio_buf into CPUX86State
  i386: hvf: Move lazy_flags into CPUX86State
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

# Conflicts:
#	hw/i386/acpi-build.c
2020-06-12 23:06:22 +01:00
Paolo Bonzini
af509738f8 docker.py/build: support binary files in --extra-files
Read the --extra-files in binary mode to avoid encoding errors.

Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-06-10 12:09:31 -04:00
Paolo Bonzini
dfae628459 docker.py/build: support -t and -f arguments
The docker.py command line is subtly different from docker and podman's,
in that the tag and Dockerfile are passed via positional arguments.
Remove this gratuitous difference and just parse -f and -t.

-f was previously used by --extra-files, only keep the long option.

Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-06-10 12:09:31 -04:00
Philippe Mathieu-Daudé
c6dd82e8a6 tests/docker: Remove flex/bison packages
QEMU does not use flex/bison packages.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Claudio Fontana <cfontana@suse.de>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200515163029.12917-2-philmd@redhat.com>
2020-06-08 17:04:19 +01:00
Paolo Bonzini
a97098844b docker: update Ubuntu to 20.04
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200604231716.11354-1-pbonzini@redhat.com>
Message-Id: <20200605154929.26910-10-alex.bennee@linaro.org>
2020-06-08 17:04:19 +01:00
Alex Bennée
e035e6ffed tests/docker: fix pre-requisite for debian-tricore-cross
Reported-by: Bastian Koppelmann <kbastian@mail.uni-paderborn.de>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200605154929.26910-9-alex.bennee@linaro.org>
2020-06-08 17:04:19 +01:00
John Snow
2d110c1149 python: remove more instances of sys.version_info
We guarantee 3.5+ everywhere; remove more dead checks. In general, try
to avoid using version checks and instead prefer to attempt behavior
when possible.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200514035230.25756-1-jsnow@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-05-31 18:25:07 +02:00
Alex Bennée
c729a99d27 tests/docker: use a gcc-10 based image for arm64 tests
As we enable newer features that we want to test on arm64 targets we
need newer compilers. Split off a new debian-arm64-test-cross image
which we can use to build these new tests.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20200520140541.30256-11-alex.bennee@linaro.org>
2020-05-27 14:26:49 +01:00
Alex Bennée
086f269cf4 tests/docker: add debian11 base image
We won't use this for building QEMU but we do need newer GCC's and
binutils for building some of our test cases.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20200520140541.30256-10-alex.bennee@linaro.org>
2020-05-27 14:26:49 +01:00
Alex Bennée
551affb61e tests/docker: add docs FEATURE flag and use for test-misc
The test-misc docker test fails on a number of images which don't have
the prerequisites to build the docs. Use the FEATURES flag so we can
skip those tests.

As the sphinx test fails to detect whatever feature we need to get
hxtool to work we drop them from debian9 so the windows build doesn't
attempt to build the docs.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20200414200631.12799-7-alex.bennee@linaro.org>
2020-04-15 11:38:23 +01:00
Philippe Mathieu-Daudé
3e3207337b tests/docker: Add libepoxy and libudev packages to the Fedora image
Install optional dependencies of QEMU to get better coverage.

Suggested-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20200322120104.21267-5-philmd@redhat.com>
Message-Id: <20200323161514.23952-11-alex.bennee@linaro.org>
2020-03-27 13:43:20 +00:00
Philippe Mathieu-Daudé
6d8e7738b0 tests/docker: Use Python3 PyYAML in the Fedora image
The Python2 PyYAML is now pointless, switch to the Python3 version.

Fixes: bcbf27947 (docker: move tests from python2 to python3)
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20200322120104.21267-4-philmd@redhat.com>
Message-Id: <20200323161514.23952-10-alex.bennee@linaro.org>
2020-03-27 13:43:20 +00:00
Philippe Mathieu-Daudé
9274ae32c3 tests/docker: Install gcrypt devel package in Debian image
In commit 6f8bbb374b we enabled building with the gcrypt library
on the the Debian 'x86 host', which was based on Debian Stretch.
Later in commit 698a71edbe we upgraded the Debian base image to
Buster.

Apparently Debian Stretch was listing gcrypt as a QEMU dependency,
but this is not the case anymore in Buster, so we need to install
it manually (it it not listed by 'apt-get -s build-dep qemu' in
the common debian10.docker anymore). This fixes:

 $ ../configure $QEMU_CONFIGURE_OPTS

  ERROR: User requested feature gcrypt
         configure was not able to find it.
         Install gcrypt devel >= 1.5.0

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20200322120104.21267-3-philmd@redhat.com>
Message-Id: <20200323161514.23952-9-alex.bennee@linaro.org>
2020-03-27 13:43:20 +00:00
Philippe Mathieu-Daudé
cdb1a84cfe tests/docker: Keep package list sorted
Keep package list sorted, this eases rebase/cherry-pick.

Fixes: 3a6784813
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20200322120104.21267-2-philmd@redhat.com>
Message-Id: <20200323161514.23952-8-alex.bennee@linaro.org>
2020-03-27 13:43:20 +00:00
Peter Maydell
a1ba62a0f3 Python and tests (mostly acceptance) patches 2020-03-17
-----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEEeruW64tGuU1eD+m7ZX6NM6XyCfMFAl5xaEMACgkQZX6NM6Xy
 CfPUDQ/+IACVcfF6LdPB/vwhBaKx8/fqpJEnyL8AFCkYrIVWw0c9zizjTobWbl8H
 qxDfqvM+O4ENy12ClG0LkyRN9sVs6IGlpE4f50kKqttd5GA8bBSvamv5iCxZ1fn8
 IsR/uH5sQt8aqENxDiIszZH+UEATsFL916KyMre8faqEN8xzFvnFeQ+pT0s5sF9v
 XpXdS1D9JtI00pFmNz/4jkyzB4uG6D20NjoHOSS8IWMx9jOAQn5++VC6183lxdpK
 ty7adf0y0brC9BCEua4FdCHliDYgYbO5hdsk6oWWUQRvnYEFajXgV/nEdqV+rCG8
 9KEKC7sdbOwkbXxwtfjMQ7QK51Xhz7P0d2j0+vn96V0MIve4GM0Z9vx87Psy+4T3
 KJqM7GISDiFtMs2Zi2D+o4nyL4m4fS1AzbAeEWH1dwl7RaAb1zKr7F2hFSHIkNHh
 Z1cnoFhm0kXQFhFqMwjY5ackV+pYm8qSXroG3SvghPByb78LwBx5rzE37clW5HDa
 VO/pRnY0FIyYldOlu+XHGCwCgiaVQ0mUb9DOwpPL+WukZUY1y6bxWlamMvIBM0ko
 l1rYlnz0mEwpLptctlob0Vo/Js/jbyxT0XRcTbJq3FexFeEykFtFwW8hfT63xJXo
 En9cVkyEWzgT9Ici3HNOwLz5iJCfkWjx/ecRtA6TBeod0V/evo0=
 =DHn4
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/cleber/tags/python-next-pull-request' into staging

Python and tests (mostly acceptance) patches 2020-03-17

# gpg: Signature made Wed 18 Mar 2020 00:16:03 GMT
# gpg:                using RSA key 7ABB96EB8B46B94D5E0FE9BB657E8D33A5F209F3
# gpg: Good signature from "Cleber Rosa <crosa@redhat.com>" [marginal]
# gpg: WARNING: This key is not certified with sufficiently trusted signatures!
# gpg:          It is not certain that the signature belongs to the owner.
# Primary key fingerprint: 7ABB 96EB 8B46 B94D 5E0F  E9BB 657E 8D33 A5F2 09F3

* remotes/cleber/tags/python-next-pull-request:
  tests/docker: make "buildah bud" output similar to "docker build"
  tests/docker: add CentOS 8 Dockerfile
  Acceptance tests: add make targets to download images
  Acceptance test: add "boot_linux" tests
  Acceptance tests: introduce BUILD_DIR and SOURCE_DIR
  python/qemu/qmp.py: QMP debug with VM label

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-03-19 12:33:50 +00:00
Peter Maydell
9214813489 Testing and gdbstub updates:
- docker updates for VirGL
   - re-factor gdbstub for static GDBState
   - re-factor gdbstub for dynamic arrays
   - add SVE support to arm gdbstub
   - add some guest debug tests to check-tcg
   - add aarch64 userspace register tests
   - remove packet size limit to gdbstub
   - simplify gdbstub monitor code
   - report vContSupported in gdbstub to use proper single-step
 -----BEGIN PGP SIGNATURE-----
 
 iQEzBAABCgAdFiEEZoWumedRZ7yvyN81+9DbCVqeKkQFAl5xDUIACgkQ+9DbCVqe
 KkQwCwf/YtmUsNxxO+CgNctq2u3jV4FoOdQP3bejvmT2+cigKJhQuBlWPg1/YsqF
 RDNkmBQx2JaVVMuVmpnwVK1UD+kmYZqrtlOkPNcVrjPmLCq3BVI1LHe6Rjoerx8F
 QoZyH0IMNHbBgDo1I46lSFOWcxmOvo+Ow7NX5bPKwlRzf0dyEqSJahRaZLAgUscR
 taTtGfk9uQsnxoRsvH/efiQ4bZtUvrEQuhEX3WW/yVE1jTpcb2llwX4xONJb2It3
 /0WREGEEIT8PpnWw2S3FH4THY/BjWgz/FPDwNNZYCKBMWDjuG/8KHryd738T9rzo
 lkGP9YcXmiyxMMyFFwS8RD3SHr8LvQ==
 =Wm+a
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/stsquad/tags/pull-testing-and-gdbstub-170320-1' into staging

Testing and gdbstub updates:

  - docker updates for VirGL
  - re-factor gdbstub for static GDBState
  - re-factor gdbstub for dynamic arrays
  - add SVE support to arm gdbstub
  - add some guest debug tests to check-tcg
  - add aarch64 userspace register tests
  - remove packet size limit to gdbstub
  - simplify gdbstub monitor code
  - report vContSupported in gdbstub to use proper single-step

# gpg: Signature made Tue 17 Mar 2020 17:47:46 GMT
# gpg:                using RSA key 6685AE99E75167BCAFC8DF35FBD0DB095A9E2A44
# gpg: Good signature from "Alex Bennée (Master Work Key) <alex.bennee@linaro.org>" [full]
# Primary key fingerprint: 6685 AE99 E751 67BC AFC8  DF35 FBD0 DB09 5A9E 2A44

* remotes/stsquad/tags/pull-testing-and-gdbstub-170320-1: (28 commits)
  gdbstub: Fix single-step issue by confirming 'vContSupported+' feature to gdb
  gdbstub: do not split gdb_monitor_write payload
  gdbstub: change GDBState.last_packet to GByteArray
  tests/tcg/aarch64: add test-sve-ioctl guest-debug test
  tests/tcg/aarch64: add SVE iotcl test
  tests/tcg/aarch64: add a gdbstub testcase for SVE registers
  tests/guest-debug: add a simple test runner
  configure: allow user to specify what gdb to use
  tests/tcg/aarch64: userspace system register test
  target/arm: don't bother with id_aa64pfr0_read for USER_ONLY
  target/arm: generate xml description of our SVE registers
  target/arm: default SVE length to 64 bytes for linux-user
  target/arm: explicitly encode regnum in our XML
  target/arm: prepare for multiple dynamic XMLs
  gdbstub: extend GByteArray to read register helpers
  target/i386: use gdb_get_reg helpers
  target/m68k: use gdb_get_reg helpers
  target/arm: use gdb_get_reg helpers
  gdbstub: add helper for 128 bit registers
  gdbstub: move mem_buf to GDBState and use GByteArray
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-03-18 20:25:23 +00:00
Cleber Rosa
a51d6a5493 tests/docker: make "buildah bud" output similar to "docker build"
Podman users will most often be using buildah to build containers.
Among the differences between "buildah bud|build-using-dockerfile" and
a traditional "docker build" is that buildah does not run a container
during build.

To the best of my knowledge and experiments, this means that runtime
variables, such as ENV from one base image will not propagate into
another.  The end result is that the location for the cross compiler
binaries, defined in the base "qemu/debian9-mxe" image, are not passed
through this image.  Consequently, the cross compilers are not on PATH
and the build fails.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Acked-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200312193616.438922-3-crosa@redhat.com>
Signed-off-by: Cleber Rosa <crosa@redhat.com>
2020-03-17 19:16:16 -04:00
Cleber Rosa
e631eb2e8b tests/docker: add CentOS 8 Dockerfile
Which is currenly missing, and will be referenced later in the
contributed CI playbooks.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200312193616.438922-2-crosa@redhat.com>
Signed-off-by: Cleber Rosa <crosa@redhat.com>
2020-03-17 19:14:49 -04:00
Philippe Mathieu-Daudé
d2763944e2 tests/docker: Update VirGL to v0.8.0
Building the qemu:debian-amd64 fails when building VirGL:

  make[2]: Entering directory '/usr/src/virglrenderer/src/gallium/auxiliary'
    CC       cso_cache/cso_cache.lo
    CC       cso_cache/cso_hash.lo
    CC       os/os_misc.lo
    CC       util/u_debug.lo
    CC       util/u_debug_describe.lo
    CC       util/u_format.lo
    GEN      util/u_format_table.c
  Traceback (most recent call last):
    File "./util/u_format_table.py", line 168, in <module>
      main()
    File "./util/u_format_table.py", line 164, in main
      write_format_table(formats)
    File "./util/u_format_table.py", line 132, in write_format_table
      print("   %s,\t/* is_array */" % (bool_map(format.is_array()),))
    File "/usr/src/virglrenderer/src/gallium/auxiliary/util/u_format_parse.py", line 164, in is_array
      return self.array_element() != None
    File "/usr/src/virglrenderer/src/gallium/auxiliary/util/u_format_parse.py", line 73, in __eq__
      return self.type == other.type and self.norm == other.norm and self.pure == other.pure and self.size == other.size
  AttributeError: 'NoneType' object has no attribute 'type'
  make[2]: Leaving directory '/usr/src/virglrenderer/src/gallium/auxiliary'
  make[2]: *** [Makefile:906: util/u_format_table.c] Error 1
  make[1]: *** [Makefile:631: install-recursive] Error 1

VirGL commits a8962eda1..a613dcc82 fix this problem.
Update to VirGL 0.8.0 which contains them.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200212202709.12665-4-philmd@redhat.com>
Message-Id: <20200316172155.971-5-alex.bennee@linaro.org>
2020-03-17 17:38:33 +00:00
Philippe Mathieu-Daudé
5b9d40fafe tests/docker: Remove obsolete VirGL --with-glx configure option
The GLX configure option has been removed in 71c75f201d [*].
We missed that when updating to v0.7.0 in commit fab3220f97.

This silents:

  configure: creating ./config.status
  config.status: creating virglrenderer.pc
  ...
  configure: WARNING: unrecognized options: --with-glx

[*] https://gitlab.freedesktop.org/virgl/virglrenderer/commit/71c75f201d

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200212202709.12665-3-philmd@redhat.com>
Message-Id: <20200316172155.971-4-alex.bennee@linaro.org>
2020-03-17 17:38:31 +00:00
Philippe Mathieu-Daudé
72e3c1dd57 tests/docker: Update VirGL git repository URL
freedesktop.org is moving to a GitLab instance,
use the new url.

- https://www.fooishbar.org/blog/gitlab-fdo-introduction/
- https://gitlab.freedesktop.org/freedesktop/freedesktop/-/wikis/home

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200212202709.12665-2-philmd@redhat.com>
Message-Id: <20200316172155.971-3-alex.bennee@linaro.org>
2020-03-17 17:38:30 +00:00
Philippe Mathieu-Daudé
1bbf2d010f tests/docker: Install tools to cross-debug and build Linux kernels
We often run Linux kernels to test QEMU. We sometimes need
to build them manually to use non-default features. We only
miss the tiny 'bc' tool.

The ncurses library is helpful to run 'make menuconfig'.

Finally, gdb-multiarch allow us to debug a TCG guest when its
architecture is different than the host.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200212202738.12986-1-philmd@redhat.com>
Message-Id: <20200316172155.971-2-alex.bennee@linaro.org>
2020-03-17 17:38:27 +00:00
Philippe Mathieu-Daudé
a4aad716cb tests/docker: Install SASL library to extend code coverage on amd64
Install the SASL library to build the VNC SASL auth protocol code.

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200309122454.22551-3-philmd@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-03-16 23:02:23 +01:00
Juan Quintela
3a67848134 configure: Enable test and libs for zstd
Add it to several build systems to make testing good.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2020-02-28 09:24:43 +01:00
Paolo Bonzini
423edd9a31 drop "from __future__ import print_function"
This is only needed for Python 2, which we do not support anymore.

Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Acked-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20200204160604.19883-1-pbonzini@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:15:16 +01:00
Philippe Mathieu-Daudé
c88ee46cdb tests: Explicit usage of Python 3
Use the program search path to find the Python 3 interpreter.

Patch created mechanically by running:

  $ sed -i "s,^#\!/usr/bin/\(env\ \)\?python$,#\!/usr/bin/env python3," \
       $(git grep -l 'if __name__.*__main__')

Reported-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Suggested-by: Daniel P. Berrangé <berrange@redhat.com>
Suggested-by: Stefan Hajnoczi <stefanha@redhat.com>
Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20200130163232.10446-5-philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:12:48 +01:00
Alex Bennée
5e33f7fead tests/docker: better handle symlinked libs
When we are copying we want to ensure we grab the first
resolution (the found in path section). However even that binary might
be a symlink so lets make sure we chase the symlinks to copy the right
binary to where it can be found.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Robert Foley <robert.foley@linaro.org>

Message-Id: <20200203090932.19147-3-alex.bennee@linaro.org>
2020-02-04 17:13:28 +00:00
Alex Bennée
698a71edbe tests/docker: move most cross compilers to buster base
This includes fixing up the dependencies (Which were already wrong for
one of the mips variants).

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200203090932.19147-2-alex.bennee@linaro.org>
2020-02-04 17:13:28 +00:00
Marc-André Lureau
1409c047c1 dockerfiles: add dbus-daemon to some of latest distributions
To get dbus-vmstate test covered.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-01-06 18:41:32 +04:00
Paolo Bonzini
7ff5c1fa27 docker: gtester is no longer used
We are using tap-driver.pl, do not require anymore gtester to be installed
to run the testsuite in docker-based tests.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <1576632611-55032-1-git-send-email-pbonzini@redhat.com>
2019-12-19 08:20:08 +00:00
Thomas Huth
2831a7e580 Remove libbluetooth / bluez from the CI tests
Since the bluetooth code has been removed, we don't need to test
with this library anymore.

Message-Id: <20191120091014.16883-5-thuth@redhat.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2019-12-17 09:05:23 +01:00
Paolo Bonzini
ad430dd9da docker: remove libcap development packages
Libcap was dropped from virtio-9p (commit 7e46261368), so remove it from
the dockerfiles as well.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Acked-by: Alex Bennée <alex.bennee@linaro.org>
[groug, mention SHA1 that dropped libcap]
Signed-off-by: Greg Kurz <groug@kaod.org>
2019-12-09 17:59:33 +01:00
Alex Bennée
8ce2f68fc9 tests/docker: update Travis image to a more current version
This isn't the latest one available on hub.docker.com but it does
match the ID reported by the Xenial builds running on Travis:

  instance: ... travis-ci-sardonyx-xenial-1553530528-f909ac5

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-10-25 19:24:21 +01:00
Alex Bennée
06746324ad tests/docker: set HOST_ARCH if we don't have ARCH
As the docker rules want to be able to be run on a virgin unconfigured
checkout add a fallback and use it if we need to.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-10-25 19:24:21 +01:00
Paolo Bonzini
9e5ef16491 tests/docker: only enable ubsan for test-clang
-fsanitize=undefined is not the same thing as --enable-sanitizers.  After
commit 47c823e ("tests/docker: add sanitizers back to clang build", 2019-09-11)
test-clang is almost duplicating the asan (test-debug) test, so
partly revert commit 47c823e5b while leaving ubsan enabled.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2019-10-04 18:49:19 +02:00
Paolo Bonzini
6f89ec7442 docker: test-debug: disable LeakSanitizer
There are just too many leaks in device-introspect-test (especially for
the plethora of arm and aarch64 boards) to make LeakSanitizer useful;
disable it for now.

Whoever is interested in debugging leaks can also use valgrind like this:

   QTEST_QEMU_BINARY=aarch64-softmmu/qemu-system-aarch64 \
   QTEST_QEMU_IMG=qemu-img \
   valgrind --trace-children=yes --leak-check=full \
   tests/device-introspect-test -p /aarch64/device/introspect/concrete/defaults/none

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2019-10-04 18:49:17 +02:00
Alex Bennée
80394ccf21 tests/docker: remove debian-powerpc-user-cross
Despite our attempts in 4d26c7fef4 to keep this going it still gets in
the way of "make docker-test-build" completing because of course we
can't build a modern QEMU with the image. Let's put the thing out of
its misery and remove it.

People who really care about building on powerpc can still use the
binfmt_misc support to manually build an image (or just run the build
from pre this commit).

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Cc: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
2019-09-26 19:00:53 +01:00
John Snow
bcbf279473 docker: move tests from python2 to python3
As part of the push to drop python2 support, replace any explicit python2
dependencies with python3 versions.

For centos, python2 still exists as an implicit dependency, but by adding
python3 we will be able to build even if the configure script begins to
require python 3.5+.

Tested with centos7, fedora, ubuntu, ubuntu1804, and debian 9 (amd64).
Tested under a custom configure script that requires Python 3.5+.

the travis dockerfile is also moved to using python3, which was tested
by running `make docker-test-build@travis`, which I hope is sufficient.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20190923181140.7235-7-jsnow@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Tested-by: Cleber Rosa <crosa@redhat.com>
2019-09-26 19:00:53 +01:00
John Snow
2e90aec3d9 docker: remove unused debian-sid
debian-sid is listed as a partial image, so we cannot run tests against it.
Since it isn't used by any other testable image, remove it for now as it
is prone to bitrot.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20190923181140.7235-6-jsnow@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-09-26 19:00:53 +01:00
John Snow
dad47814f0 docker: remove unused debian-ports
debian-ports is listed as a partial image, so we cannot run tests against it.
Since it isn't used by any other testable image, remove it for now as it
is prone to bitrot.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20190923181140.7235-5-jsnow@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-09-26 19:00:53 +01:00
John Snow
ef8f3283ac docker: remove 'deprecated' image definitions
There isn't a debian.dockerfile anymore,
so perform some ghost-busting.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20190923181140.7235-4-jsnow@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Acked-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-09-26 19:00:53 +01:00
John Snow
afa1cf4329 docker: remove unused debian8 partial image
debian8 partial base is also not consumed by any image, so remove it.
For QEMU's development cycle, we only support debian9 (stretch) and
debian10 (buster).

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20190923181140.7235-3-jsnow@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-09-26 19:00:53 +01:00
John Snow
ccf20b3b4f docker: remove debian8-mxe definitions
We don't have a debian8-mxe dockerfile anymore.

Fixes: 67bd36beda
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20190923181140.7235-2-jsnow@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-09-26 19:00:53 +01:00
John Snow
b3a790be98 podman: fix command invocation
Oops; there's no argv here.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20190913193821.17756-1-jsnow@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Tested-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Tested-by: Thomas Huth <thuth@redhat.com>
2019-09-26 19:00:53 +01:00
Alex Bennée
529994e204 tests/docker: reduce scary warnings by cleaning up clean up
There was in the clean-up code caused by attempting to inspect images
which finished before we got there. Clean up the clean up code by:

  - only track the one instance at a time
  - use --filter for docker ps instead of doing it by hand
  - just call docker rm -f to be done with it
  - use uuid.uuid4() for a random uid

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
2019-09-26 19:00:53 +01:00
John Snow
5fac0cfaaa tests/docker: remove python2.7 from debian9-mxe
When it was based on debian8 which uses python-minimal, it needed this.
It no longer does.

Goodbye, python2.7.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20190918222546.11696-1-jsnow@redhat.com>
[AJB: fixed up commit message]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-09-26 19:00:53 +01:00
Alex Bennée
b4d66d3525 tests/docker: fix DOCKER_PARTIAL_IMAGES
Finger trouble in a previous clean-up inadvertently set
DEBIAN_PARTIAL_IMAGES instead of DOCKER_PARTIAL_IMAGES. Also fix the
typo to debian-9-mxe.

Fixes: 44d5a8bf5d
Signed-off-by: John Snow <jsnow@redhat.com>
[AJB: merged fix from Message-Id: <20190917185537.25417-1-jsnow@redhat.com>]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
2019-09-26 19:00:53 +01:00
John Snow
47c823e5be tests/docker: add sanitizers back to clang build
Fedora23 is but a distant twinkle. The sanitizer works again, and even
if not, we have --enable-sanitizers now.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20190912014442.5757-1-jsnow@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-09-26 19:00:53 +01:00
Alex Bennée
50d2dcaddb tests/docker: --disable-libssh on ubuntu1804 builds
Currently this stops the mega:

  make docker-test-build

from working. Once the source is patched to deal with the case this
workaround can be removed.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-09-10 14:14:31 +01:00
Alex Bennée
e5389e4414 tests/docker: add more images to PARTIAL_IMAGES when not on x86_64
This prevents us trying to do builds which we can't complete.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-09-10 14:14:31 +01:00
Alex Bennée
d2467284e8 tests/docker: use --arch-only for installing deps
The Debian QEMU packages require a bunch of cross compilers for
building firmware which aren't available on all host architectures.
Using --arch-only skips this particular requirement and allows us to
install just the dependencies we need.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-09-10 14:14:31 +01:00
Alex Bennée
ec22b72699 tests/docker: add debian-amd64-cross for non-x86 hosts
When building on a non-x86 host we need to setup the x86 build like
any other cross compiler.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-09-10 14:14:31 +01:00
Alex Bennée
bf5b512cd4 tests/docker: avoid $SHELL invoke bash directly
On some images SHELL is pointing at a limited /bin/sh which doesn't
understand noprofile/norc. Given the run script is running bash just
invoke it directly.

This fixes:

  $ make docker-test-build@IMAGE DEBUG=1
  [...]
  + echo '  ./test-build'
  ./test-build
  + echo '* Hit Ctrl-D to continue, or type '\''exit 1'\'' to abort'
  * Hit Ctrl-D to continue, or type 'exit 1' to abort
  + echo
  + /bin/sh --noprofile --norc
  /bin/sh: 0: Illegal option --

Fixes: 2b0c4fa13f
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-09-10 14:14:31 +01:00
Alex Bennée
44d5a8bf5d tests/docker: add debian9-mxe to DEBIAN_PARTIAL_IMAGES
Another image that can't be used directly to build QEMU.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-09-10 14:14:31 +01:00
Alex Bennée
ac276a4b54 tests/docker: add debian-xtensa-cross to DEBIAN_PARTIAL_IMAGES
This should have been marked when the docker recipe was added to
prevent it being used for cross compiling QEMU. Sort the
DEBIAN_PARTIAL_IMAGE list while we are at it.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-09-10 14:14:31 +01:00
Alex Bennée
4d26c7fef4 tests/docker: pin powerpc-user-cross to a snapshot
Now Jessie has entered LTS the powerpc architecture has been dropped
so we can no longer build the image from scratch. However we can use
the snapshot archive to build the last working version.

This now only lives on an example of setting up a user-cross image as
at least on x86-64 we can use the Buster packaged cross compiler for
building test images.

Suggested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Cc: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-09-10 14:14:31 +01:00
Alex Bennée
a3c1f1283b tests/docker: update Debian Sid image
While we are not currently using it we might as well keep the image
for later usage. So:

  - update to a more recent snapshot
  - clean up verbiage in commentary
  - remove duplicate shell from a merge failure

Suggested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-09-10 14:14:31 +01:00
Alex Bennée
8a4daee5f9 tests/docker: move our ppc64 cross compile to Buster
Now Buster is released we can stop relying on the movable feast that
is Sid for our cross-compiler for building tests.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-09-10 14:14:31 +01:00
Alex Bennée
c6e8f512d5 tests/docker: move our riscv64 cross compile to Buster
Now Buster is released we can stop relying on the movable feast that
is Sid for our cross-compiler for building tests.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-09-10 14:14:31 +01:00
Alex Bennée
4575a701ea tests/docker: move our mips64 cross compile to Buster
Now Buster is released we can stop relying on the movable feast that
is Sid for our cross-compiler for building tests.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-09-10 14:14:31 +01:00
Alex Bennée
923984c5e8 tests/docker: move our sh4 cross compile to Buster
Now Buster is released we can stop relying on the movable feast that
is Sid for our cross-compiler for building tests.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-09-10 14:14:31 +01:00
Alex Bennée
78d1d69641 tests/docker: move our sparc64 cross compile to Buster
Now Buster is released we can stop relying on the movable feast that
is Sid for our cross-compiler for building tests.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-09-10 14:14:31 +01:00
Alex Bennée
20f8b1a27a tests/docker: move our m68k cross compile to Buster
Now Buster is released we can stop relying on the movable feast that
is Sid for our cross-compiler for building tests.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-09-10 14:14:31 +01:00
Alex Bennée
c268700b9f tests/docker: move our HPPA cross compile to Buster
Now Buster is released we can stop relying on the movable feast that
is Sid for our cross-compiler for building tests.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-09-10 14:14:31 +01:00
Alex Bennée
e3386c276c tests/docker: move our Alpha cross compile to Buster
Now Buster is released we can stop relying on the movable feast that
is Sid for our cross-compiler for building tests.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-09-10 14:14:31 +01:00
Alex Bennée
ce49420913 tests/docker: move our powerpc cross compile to Buster
Now Buster is released we can stop relying on the movable feast that
is Sid for our cross-compiler for building tests.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-09-10 14:14:31 +01:00
Alex Bennée
18b6be4326 tests/docker: move our arm64 cross compile to Buster
Now Buster is released we can unify our cross build images for both
QEMU and tests.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-09-10 14:14:31 +01:00
Alex Bennée
2f45f2d446 tests/docker: add Buster to DOCKER_PARTIAL_IMAGES
We need to add additional packages to the base images to be able to
build QEMU so lets avoid building with it.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-09-10 14:14:31 +01:00
Alex Bennée
09bb808f0b tests/docker: set DEF_TARGET_LIST for some containers
You can assume the failures most people are interested in are the
cross-compile failures that are specific to the cross compile target.
Set DEF_TARGET_LIST based on what we use for shippable, the user can
always override by calling with TARGET_LIST set.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-09-10 14:14:31 +01:00
Alex Bennée
6945018a68 tests/docker: move DEF_TARGET_LIST setting to common.rc
We might as well not repeat ourselves. At the same time allow it to be
overridden which we will use later from docker targets.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-09-10 14:14:31 +01:00
Alex Bennée
eea2153ea8 tests/docker: fix final missing .encode when parsing solibs
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-09-10 09:38:33 +01:00
Alex Bennée
884fcafc9c tests/docker: handle missing encoding keyword for subprocess.check_output
This was only added in Python 3.6 and not all the build hosts have
that recent a python3. However we still need to ensure everything is
returns as a unicode string so checks higher up the call chain don't
barf.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

fixup! tests/docker: handle missing encoding keyword for subprocess.check_output
2019-09-10 09:38:33 +01:00
Alex Bennée
71ebbe09e9 tests/docker: fix "cc" command to work with podman
Podman requires a little bit of additional magic to the uid mapping
which was already done for the normal RunCommand. We simplify the
logic by pushing it directly into the Docker::run method to avoid
instantiating an extra Docker() object and ensure the CC command
always runs as the current user.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-09-10 09:38:33 +01:00
John Snow
63772d5cfd tests/docker: Use --userns=keep-id for podman
The workaround that attempts to accomplish the same result as --userns=keep-id
does not appear to work well with UIDs much above 1000 (like mine, which is
above 20000.)

Since we have official support for this "trick" now, use the supported method.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20190904232451.26466-1-jsnow@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-09-10 09:38:33 +01:00
Alex Bennée
8480517d4c configure: clean-up container cross compile detect
The introduction of podman support inadvertently broke configure's
detect of the container support as the configure probe didn't specify
an engine type. To fix this in docker.py:

  - only (re)set USE_ENGINE if --engine is specified
  - enhance the output so docker is no longer just yes

In the configure script we can at least start cleaning up the
detecting and naming of variables. To avoid too much churn the
conversion of the various make DOCKER_foo variables has been left for
future clean-ups.

Fixes: 9459f75413
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: David Hildenbrand <david@redhat.com>
2019-09-10 09:38:33 +01:00
Alex Bennée
4112aff7cd tests/docker: upgrade docker.py to python3
The recent podman changes (9459f75413) imported enum which is part
of the python3 standard library but only available as an external
library for python2. This causes problems on the fairly restricted
environment such as shippable. Lets bite the bullet and make the
script a fully python3 one. To that end:

  - drop the from __future__ import (we are there now ;-)
  - avoid the StringIO import hack
  - be consistent with the mode we read/write dockerfiles
  - s/iteritems/items/
  - ensure check_output returns strings for processing

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Cc: Marc-André Lureau <marcandre.lureau@redhat.com>
2019-09-02 10:52:22 +01:00
Marc-André Lureau
05af039d1e tests/docker: add podman support
Allow to specify the container engine to run with ENGINE variable.

By default, ENGINE=auto and will select either podman or docker.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2019-08-22 17:30:25 +04:00
Marc-André Lureau
9459f75413 docker.py: add podman support
Add a --engine option to select either docker, podman or auto.

Among other advantages, podman allows to run rootless & daemonless
containers, fortunately sharing compatible CLI with docker.

With current podman, we have to use a uidmap trick in order to be able
to rw-share the ccache directory with the container user.

With a user 1000, the default mapping is:                                                                                                                                                                         1000 (host) -> 0 (container).
So write access to /var/tmp/ccache ends will end with permission
denied error.

With "--uidmap 1000:0:1 --uidmap 0:1:1000", the mapping is:
1000 (host) -> 0 (container, 1st namespace) -> 1000 (container, 2nd namespace).
(the rest is mumbo jumbo to avoid holes in the range of UIDs)

A future podman version may have an option such as --userns-keep-uid.
Thanks to Debarshi Ray <rishi@redhat.com> for the help!

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Acked-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2019-08-22 17:30:25 +04:00
Marc-André Lureau
2461d80e6c docker.py: add --run-as-current-user
(podman will need further tweaks)

Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2019-08-22 17:30:25 +04:00
Philippe Mathieu-Daudé
98808c3d0c tests/docker: Refresh APT cache before installing new packages on Debian
Since docker caches the different layers, updating the package
list does not invalidate the previous "apt-get update" layer,
and it is likely "apt-get install" hits an outdated repository.

See commit beac6a98f6 and
https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#apt-get

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20190723141528.18023-1-philmd@redhat.com>
[AJB: manually applies and fixed up]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-07-23 17:19:11 +01:00
Alex Bennée
2b0c4fa13f tests/docker: invoke the DEBUG shell with --noprofile/--norc
It's very confusing when things work in the debug shell because the
environment is different from what the test is running. Fix this by
ensuring we only have the inherited environment from the run shell.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-07-23 15:53:25 +01:00
Philippe Mathieu-Daudé
5deb2de1d6 tests/docker: Let the test-mingw test generate a NSIS installer
The NSIS installer generates an executable suitable to install
QEMU on Windows.

Suggested-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20190715174817.18981-9-philmd@redhat.com>
[AJB: also --enable-docs in configure step]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-07-23 15:53:25 +01:00
Philippe Mathieu-Daudé
10d7166e42 tests/docker: Install texinfo in the Fedora image
The 'makeinfo' is required to generate the documentation from
the 'html' Makefile rule (called by 'install-doc').
The NSIS installer uses these files.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20190715174817.18981-6-philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-07-23 15:53:25 +01:00
Philippe Mathieu-Daudé
a88819edde tests/docker: Set the correct cross-PKG_CONFIG_PATH in the MXE images
This silents a bunch of warnings while compiling the Slirp objects:

 $ make
 [...]
   CC      slirp/src/tftp.o
 Package glib-2.0 was not found in the pkg-config search path.
 Perhaps you should add the directory containing `glib-2.0.pc'
 to the PKG_CONFIG_PATH environment variable
 No package 'glib-2.0' found
   CC      slirp/src/udp6.o
 Package glib-2.0 was not found in the pkg-config search path.
 Perhaps you should add the directory containing `glib-2.0.pc'
 to the PKG_CONFIG_PATH environment variable
 No package 'glib-2.0' found
 [...]

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20190715174817.18981-5-philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-07-23 15:53:25 +01:00
Philippe Mathieu-Daudé
c054f3f43a tests/docker: Install the NSIS tools in the MinGW capable images
This fixes:

  $ make installer
  (cd /tmp/qemu-nsis; \
           for i in qemu-system-*.exe; do \
             arch=${i%.exe}; \
             arch=${arch#qemu-system-}; \
             echo Section \"$arch\" Section_$arch; \
             echo SetOutPath \"\$INSTDIR\"; \
             echo File \"\${BINDIR}\\$i\"; \
             echo SectionEnd; \
           done \
          ) >/tmp/qemu-nsis/system-emulations.nsh
  makensis -V2 -NOCD \
                  -DCONFIG_DOCUMENTATION="y" \
                   \
                  -DBINDIR="/tmp/qemu-nsis" \
                   \
                  -DSRCDIR="/home/phil/source/qemu" \
                  -DOUTFILE="qemu-setup-4.0.90.exe" \
                  -DDISPLAYVERSION="4.0.90" \
                  /home/phil/source/qemu/qemu.nsi
  /bin/sh: 1: makensis: not found
  Makefile:1077: recipe for target 'qemu-setup-4.0.90.exe' failed
  make: *** [qemu-setup-4.0.90.exe] Error 127

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20190715174817.18981-4-philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-07-23 15:53:25 +01:00
Philippe Mathieu-Daudé
8d16a31028 tests/docker: Install Sphinx in the Debian images
Since commit 5f71eac06e the Sphinx tool is required
to build the rST documentation.

This fixes:

 $ ./configure --enable-docs

 ERROR: User requested feature docs
        configure was not able to find it.
        Install texinfo, Perl/perl-podlators and python-sphinx

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20190715174817.18981-3-philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-07-23 15:53:25 +01:00
Alex Bennée
67bd36beda tests/dockerfiles: update the win cross builds to stretch
While fixing up pkg.mxe.cc they move the URLs around a bit and dropped
Jessie support in favour of Stretch. We also need to update the keys
used to verify the packages.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-07-23 15:53:25 +01:00
Philippe Mathieu-Daudé
56e89a310b tests/docker: Install Ubuntu images noninteractively
We correctly use the DEBIAN_FRONTEND environment variable on
the Debian images, but forgot the Ubuntu ones are based on it.

Since building docker images is not interactive, we need to
inform the APT tools about it using the DEBIAN_FRONTEND
environment variable (we already use it on our Debian images).

This fixes:

  $ make docker-image-ubuntu V=1
  [...]
  Setting up tzdata (2019b-0ubuntu0.19.04) ...
  debconf: unable to initialize frontend: Dialog
  debconf: (TERM is not set, so the dialog frontend is not usable.)
  debconf: falling back to frontend: Readline
  Configuring tzdata
  ------------------

  Please select the geographic area in which you live. Subsequent configuration
  questions will narrow this down by presenting a list of cities, representing
  the time zones in which they are located.

    1. Africa      4. Australia  7. Atlantic  10. Pacific  13. Etc
    2. America     5. Arctic     8. Europe    11. SystemV
    3. Antarctica  6. Asia       9. Indian    12. US
  Geographic area: 12
  [HANG]

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20190711124805.26476-1-philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-07-23 15:53:25 +01:00
Philippe Mathieu-Daudé
8109b7986a tests/docker: Install Sphinx in the Fedora image
Since commit 5f71eac06e the Sphinx tool is required
to build the rST documentation.

This fixes:

 $ ./configure --enable-docs

 ERROR: User requested feature docs
        configure was not able to find it.
        Install texinfo, Perl/perl-podlators and python-sphinx

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20190711102710.2263-1-philmd@redhat.com>
[AJB: also add /usr/libexec/python3-sphinx/ to PATH]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-07-23 15:53:25 +01:00
Philippe Mathieu-Daudé
6d7b9c8c06 tests/docker: Install Sphinx in the Ubuntu images
Since commit 5f71eac06e the Sphinx tool is required
to build the rST documentation.

This fixes:

 $ ./configure --enable-docs

 ERROR: User requested feature docs
        configure was not able to find it.
        Install texinfo, Perl/perl-podlators and python-sphinx

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20190711120609.12773-1-philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Stefano Garzarella <sgarzare@redhat.com>
2019-07-23 15:53:25 +01:00
Alex Bennée
3c237b5c65 tests/docker: add test-misc for building tools & docs
Add yet another test type so we cna quickly exercise the miscellaneous
build products of the build system under various docer configurations.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-07-23 15:53:25 +01:00
Pino Toscano
b10d49d761 ssh: switch from libssh2 to libssh
Rewrite the implementation of the ssh block driver to use libssh instead
of libssh2.  The libssh library has various advantages over libssh2:
- easier API for authentication (for example for using ssh-agent)
- easier API for known_hosts handling
- supports newer types of keys in known_hosts

Use APIs/features available in libssh 0.8 conditionally, to support
older versions (which are not recommended though).

Adjust the iotest 207 according to the different error message, and to
find the default key type for localhost (to properly compare the
fingerprint with).
Contributed-by: Max Reitz <mreitz@redhat.com>

Adjust the various Docker/Travis scripts to use libssh when available
instead of libssh2. The mingw/mxe testing is dropped for now, as there
are no packages for it.

Signed-off-by: Pino Toscano <ptoscano@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Acked-by: Alex Bennée <alex.bennee@linaro.org>
Message-id: 20190620200840.17655-1-ptoscano@redhat.com
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-id: 5873173.t2JhDm7DL7@lindworm.usersys.redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-06-24 16:01:04 +02:00
Alex Bennée
248cf06cf2 tests/docker: Update the Ubuntu image to 19.04
This has aged a little and we have a separate LTS image for testing on
the older distros. Update it to a more recent release like its Fedora
cousin.

Besides it is useful to have something with gcc-9 on it for squashing
those stringop truncation errors.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
2019-06-12 17:53:22 +01:00
Alex Bennée
acc5c5061d tests/docker: Update the Fedora cross compile images to 30
While at it remove the bogus :latest tag for cris cross compiler. It
tends to break caching and cause confusion.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-06-12 17:53:22 +01:00
Philippe Mathieu-Daudé
c87e38399c tests/docker: Update the Fedora image to Fedora 30
Fedora 30 got released:

  https://fedoramagazine.org/announcing-fedora-30/

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Tested-by: Stefano Garzarella <sgarzare@redhat.com>
Reviewed-by: Stefano Garzarella <sgarzare@redhat.com>
Message-Id: <20190528153304.27157-1-philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-06-12 17:53:22 +01:00
David Gibson
c10aaaab0f tests: Fix up docker cross builds for ppc64 (BE) targets
We currently have docker cross building targets for powerpc (32-bit, BE)
and ppc64el (64-bit, LE), but not for pcp64 (64-bit, BE).  This is an
irritating gap in make check-tcg coverage so correct it.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2019-05-29 11:14:19 +10:00
Philippe Mathieu-Daudé
8ea5962f28 tests/docker: Test more components on the Fedora default image
Install optional dependencies of QEMU to get better coverage.

The following components are now enabled:

  $ ./configure
  ...
  Multipath support yes
  VNC SASL support  yes
  RDMA support      yes
  PVRDMA support    yes
  libiscsi support  yes
  seccomp support   yes
  libpmem support   yes
  libudev           yes

Note: The udev-devel package is provided by systemd-devel.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20190504055440.20406-1-philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Acked-by: Marcel Apfelbaum<marcel.apfelbaum@gmail.com>
2019-05-28 10:28:51 +01:00
Gerd Hoffmann
e8e875b1a3 tests/docker: add ubuntu 18.04
Based on the ubuntu.docker file.
Used to reproduce the build failure Peter was seeing.
Others might find this useful too ;)

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Message-Id: <20190503070241.24786-1-kraxel@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-05-28 10:28:51 +01:00
Wainer dos Santos Moschetta
87db901820 docker: trivial changes to make docker help
Apply double quotes and period punctuation uniformly.

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20190321212528.6100-1-wainersm@redhat.com>
Reviewed-by: Fam Zheng <fam@euphon.net>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-03-25 10:39:19 +00:00
Wainer dos Santos Moschetta
a6de52ac7a docker: Fix travis script unable to find source dir
The script generated from QEMU_SRC/.travis.yml uses BUILD_DIR and
SRC_DIR path relative to the current dir, unless these variables
are exported in environment.

Since commit 05790dafef BUILD_DIR is exported in the runner script,
although SRC_DIR is not, so that make docker-travis fails becase
the reference to source dir is wrong. So let's unset both BUILD_DIR
and SRC_DIR before calling the script, given it is executed from
the source dir already (as in Travis).

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20190320221207.11366-3-wainersm@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-03-25 10:35:42 +00:00
Wainer dos Santos Moschetta
e8ced6813d docker: Fix travis.py parser and misc change
Fixed the travis.py script that has failed to parse the current
QEMU_SRC/.travis.yml file. It no longer makes combinations from
env/matrix, instead it uses explicit includes. Also the compiler
can be omitted from matrix/include, so that Travis chooses the
first entry of the global compiler list.

Replaced yaml.load() with yaml.safe_load() so that quieting the
following deprecation warning:
https://github.com/yaml/pyyaml/wiki/PyYAML-yaml.load(input)-Deprecation

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20190320221207.11366-2-wainersm@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-03-25 10:35:42 +00:00
Alex Bennée
d72132c02e tests/tcg: enable cris base user-mode tests
This converts the existing Makefile into a Makefile.target and updates
it so it can be called by the tcg build system. The original Makefile
didn't set -cpu except for the v17 tests however that has broken (I
assume because linux-user is a "max" cpu) so here I force it to be
crisv17.

I've also replicated the GNU simulator targets (run-FOO-on-sim).

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-03-12 17:05:21 +00:00
Alex Bennée
824ad15f1c tests/docker: add fedora-cris-cross compilers
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-03-12 17:05:21 +00:00
Philippe Mathieu-Daudé
c72d9df181 tests/docker: add debian-xtensa-cross image
Xtensa cpu supported:
- dc232b
- dc233c
- csp

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-03-12 17:05:21 +00:00
Alex Bennée
e92a43bb18 tests/docker: peg netmap code to a specific version
Tracking head is always going to be at the whims of the upstream.
Let's use a defined release so things don't magically change under us.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
2019-02-22 09:32:32 +00:00
Alex Bennée
a48a1d1827 tests/docker: squash initial update and install step for debian9
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-02-22 09:32:20 +00:00
Alex Bennée
432d8ad5f6 tests: PEP8 cleanup of docker.py, mostly white space
My editor keeps putting squiggly lines under a bunch of the python
lines to remind me how non-PEP8 compliant it is. Clean that up so it's
easier to spot new errors.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-02-08 17:32:35 +00:00
Alex Bennée
d10404b193 tests: docker.py be even smarter with persistent binfmt_misc
If we have a persistent mapping we don't need the QEMU binary copied
into the container as the kernel has already opened the file and will
pass the fd in. However the support libraries will still need to be
there.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-02-08 17:32:35 +00:00
Alex Bennée
43c898b75a tests: make docker.py check for persistent configs
binfmt_misc configured with the "F" flag opens the interpreter at
config time. This means it can use an already open file-descriptor to
run QEMU so there is no point trying to copy the binary into a
container.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-02-08 17:32:35 +00:00
Alex Bennée
7e81d19840 tests: make docker.py update use configured binfmt path
When copying a QEMU binary into a linux-user docker image we should
check what the current configured binfmt_misc path is rather than
just assuming "/usr/bin/qemu-bin". Obviously if the user changes the
configuration afterwards they will break their images again.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-02-08 17:32:35 +00:00
Alex Bennée
d6db2a1cdf docker: add debian-buster-arm64-cross
We can't build QEMU with this but we can use this image to build newer
arm64 testcases which need more up to date tools.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-02-08 17:32:35 +00:00
Paolo Bonzini
dce5874fc7 docker: adjust Xen repository for CentOS 7
The Xen repository is failing to install, pick the right name for the release package.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2019-02-05 16:50:18 +01:00
Thomas Huth
335dbb5de1 tests/docker/test-mingw and docs: Remove --with-sdlabi=2.0
Patchew currently reports failures with the mingw docker test - this
is due to --with-sdlabi=2.0 configure flag which does not exist anymore.
Remove this remainder from the docker test and the docs now.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Acked-by: Alex Bennée <alex.bennee@linaro.org>
Message-id: 1549268743-18502-1-git-send-email-thuth@redhat.com
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-02-04 15:25:21 +00:00
Alex Bennée
d406015b6a tests/docker: remove SID_AGE test hack
Now we are using "named" snapshots of debian-sid we can rely on the
existing checksum mechanism for detecting changes.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-01-14 14:52:30 +00:00
Alex Bennée
3cc34204ea tests/docker: update our Travis image
We are now using Xenial based images on Travis so we should make the
same one available as our qemu:travis docker image.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-01-14 14:52:30 +00:00
Philippe Mathieu-Daudé
b4048a7cd1 docker: Use a stable snapshot for Debian Sid
The Debian Sid repository is not garanteed to be stable, as his
'unstable' name suggest :)
To allow quick testing, packages are pushed various time a day,
which my be annoying when trying to use it for stable development
(which is not recommended, but Sid provides edge packages we use
for testing).

Debian provides repositories snapshots which are suitable for our
use. Pick a recent date that works. When required, update to newer
releases will be easy.

This fixes current issues with this image:

  $ make docker-image-debian-sid
  [...]
  The following packages have unmet dependencies:
   build-essential : Depends: dpkg-dev (>= 1.17.11) but it is not going to be installed
   git : Depends: perl but it is not going to be installed
         Depends: liberror-perl but it is not going to be installed
   pkg-config : Depends: libdpkg-perl but it is not going to be installed
   texinfo : Depends: perl (>= 5.26.2-6) but it is not going to be installed
             Depends: libtext-unidecode-perl but it is not going to be installed
             Depends: libxml-libxml-perl but it is not going to be installed
  E: Unable to correct problems, you have held broken packages.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
[AJB: also tweak FROM to a earlier snapshot]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-01-14 14:52:30 +00:00
Alex Bennée
53fe2431c7 tests: use in-place sed magic for enabling deb-src in travis image
This avoids potential problems with duplicates.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-01-14 14:52:30 +00:00
Daniel P. Berrangé
62559b9167 tests: update Fedora i386 cross image to Fedora 29
Using the "latest" tag is not a good idea because this changes what
release it points to every 6 months. Together with caching of docker
builds this can cause confusion where CI has cached & built with Fedora
N, while a developer tries to reproduce a CI problem with Fedora N + 1,
or vica-verca.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-01-14 14:52:30 +00:00
Daniel P. Berrangé
19a9978db1 tests: update Fedora dockerfile to use Fedora 29
Fedora 29 is the current newest release, so switch to using that
from the current Fedora 28.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-01-14 14:52:30 +00:00
Daniel P. Berrangé
ea08a74620 tests: remove obsolete 'debian' dockerfile
The 'debian' dockerfile was deprecated in favour of versioned
dockerfiles in July 2017. That is enough time for developers to
be warned about the rename.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-01-14 14:52:30 +00:00
Daniel P. Berrangé
55923c7283 tests: run ldconfig after installing extra software
The docker file builds and installs software into /usr/local but does
not run ldconfig. As a result QEMU links to libvirglrenderer.so, but
then crashes in "make check" unable to find the library.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-01-14 14:52:30 +00:00
Philippe Mathieu-Daudé
fab3220f97 docker: Use stable git tag for virglrenderer
Use a stable tag instead of some random commit from mainstream
development, to avoid unexpected build failures.

This fixes:

    CC       virglrenderer.lo
  virglrenderer.c: In function 'virgl_has_gl_colorspace':
  virglrenderer.c:208:11: error: implicit declaration of function 'virgl_has_egl_khr_gl_colorspace' [-Werror=implicit-function-declaration]
             virgl_has_egl_khr_gl_colorspace(egl_info));
             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  virglrenderer.c:208:43: error: 'egl_info' undeclared (first use in this function)
             virgl_has_egl_khr_gl_colorspace(egl_info));
                                             ^~~~~~~~
  virglrenderer.c:208:43: note: each undeclared identifier is reported only once for each function it appears in
  cc1: some warnings being treated as errors

As of this commit 'git virglrenderer-0.7.0' is the last stable tag.
(virglrenderer commit breaking: fb4f7577f7ef)

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-01-14 14:52:30 +00:00
Paolo Bonzini
9df43317b8 test: replace gtester with a TAP driver
gtester is deprecated by upstream glib (see for example the announcement
at https://blog.gtk.org/2018/07/11/news-from-glib-2-58/) and it does
not support tests that call g_test_skip in some glib stable releases.

glib suggests instead using Automake's TAP support, which gtest itself
supports since version 2.38 (QEMU's minimum requirement is 2.40).
We do not support Automake, but we can use Automake's code to beautify
the TAP output.  I chose to use the Perl copy rather than the shell/awk
one, with some changes so that it can accept TAP through stdin, in order
to reuse Perl's TAP parsing package.  This also avoids duplicating the
parser between tap-driver.pl and tap-merge.pl.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <1543513531-1151-3-git-send-email-pbonzini@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2019-01-11 15:46:52 +01:00
Philippe Mathieu-Daudé
4ce58d861b docker: Use a stable snapshot for Debian Sid
The Debian Sid repository is not garanteed to be stable, as his
'unstable' name suggest :)

To allow quick testing, Debian maintainers might push packages
various time a day. Sometime package dependencies might break,
which is annoying when using this repository for stable development
(which is not recommended, but Sid provides edge packages we use
for testing).

Debian provides repositories snapshots which are suitable for our
use. Pick a recent date that works. When required, update to newer
releases will be easy.

This fixes current issues with this image:

  $ make docker-image-debian-sid
  [...]
  The following packages have unmet dependencies:
   build-essential : Depends: dpkg-dev (>= 1.17.11) but it is not going to be installed
   git : Depends: perl but it is not going to be installed
         Depends: liberror-perl but it is not going to be installed
   pkg-config : Depends: libdpkg-perl but it is not going to be installed
   texinfo : Depends: perl (>= 5.26.2-6) but it is not going to be installed
             Depends: libtext-unidecode-perl but it is not going to be installed
             Depends: libxml-libxml-perl but it is not going to be installed
  E: Unable to correct problems, you have held broken packages.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20181101183705.5422-1-philmd@redhat.com>
Signed-off-by: Fam Zheng <famz@redhat.com>
2019-01-09 09:38:34 +08:00
Stefan Hajnoczi
b4c0595446 docker: use HTTPS git URL for virglrenderer
When you clone the repository without previous commit history, 'git://'
doesn't protect from man-in-the-middle attacks.  HTTPS is more secure
since the client verifies the server certificate.

Cc: Philippe Mathieu-Daudé <f4bug@amsat.org>
Suggested-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Cornelia Huck <cohuck@redhat.com>
Message-id: 20181108111531.30671-9-stefanha@redhat.com
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-11-12 11:26:02 +00:00
Peter Maydell
a2e002ff79 QEMU trivial patches collected between June and October 2018
(Thank you to Thomas Huth)
 
 v2: fix 32bit build with updated patch (v3) from Philippe Mathieu-Daudé
     built in a 32bit debian sid chroot
 -----BEGIN PGP SIGNATURE-----
 
 iQIcBAABAgAGBQJb2D8VAAoJEPMMOL0/L748dZEP/11pPehjPPYVxesxM++pFeuf
 2EOrLuOTkwlRX23itj2JHv8UTY3YZR9Z8kkF3SWe7qYfp4kB4dTEYjnJY5Im6fWQ
 TUbC9D9SivknOOPyQUtGXZQRN8D8m6V4hN2ZcoXC2M48GT23/uqUWBwCKYeHxdLf
 iJQFmhwDnXSZr+D0l9mpMK2vBsZ5ywcbne8GufTtrkz7Dq9A0nDWVc/XUEHzzahf
 C+6r2fRPjtImxIjhAGQeAEzOk5tYnqK/3kXjy6T4UygvnZw0pkAS1rIb3hvlzm1e
 kBlbA+pgL0kKumMmT9LBR4Os4hlL95URUF+BDNGa3EusImSL/wmhsawslQbfxVyv
 5at3VKIdvPXr7GQvmhaJ3dllXiQixX7A+axevkwyZkuIcYLnuhvh6bCR3ap+4mq/
 GRk4vwXStS6S8rDLAzo4GA4DsE4EDYJSnU13wMEaj1L9sYPVg1224AgCjnlIBbQa
 ntGD3lY7+nG5q1BeVfZXmpNZ4+N4TSpu2uEBxNvWY2/YkaouleQXJ8W4eFirB1Eo
 G8TN2fbroLcKgxhOlpvgFrfrgs8T5ZprpqQnvpE2h6M2Nu4JWJq4008q3uIPOwTy
 o9MrquqOjdG0+OBHr8Ji5HwDKex68NRQhl8BYhqtPhi/+XycDo47YSodNBfw2U/Q
 Ec9301/TQjBcvCBLEzrt
 =sHPv
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/vivier2/tags/qemu-trivial-for-3.1-pull-request' into staging

QEMU trivial patches collected between June and October 2018
(Thank you to Thomas Huth)

v2: fix 32bit build with updated patch (v3) from Philippe Mathieu-Daudé
    built in a 32bit debian sid chroot

# gpg: Signature made Tue 30 Oct 2018 11:23:01 GMT
# gpg:                using RSA key F30C38BD3F2FBE3C
# gpg: Good signature from "Laurent Vivier <lvivier@redhat.com>"
# gpg:                 aka "Laurent Vivier <laurent@vivier.eu>"
# gpg:                 aka "Laurent Vivier (Red Hat) <lvivier@redhat.com>"
# Primary key fingerprint: CD2F 75DD C8E3 A4DC 2E4F  5173 F30C 38BD 3F2F BE3C

* remotes/vivier2/tags/qemu-trivial-for-3.1-pull-request:
  milkymist-minimac2: Use qemu_log_mask(GUEST_ERROR) instead of error_report
  ppc: move at24c to its own CONFIG_ symbol
  hw/intc/gicv3: Remove useless parenthesis around DIV_ROUND_UP macro
  hw/pci-host: Remove useless parenthesis around DIV_ROUND_UP macro
  tests/bios-tables-test: Remove an useless cast
  xen: Use the PCI_DEVICE macro
  qobject: Catch another straggler for use of qdict_put_str()
  configure: Support pkg-config for zlib
  tests: Fix typos in comments and help message (found by codespell)
  cpu.h: fix a typo in comment
  linux-user: fix comment s/atomic_write/atomic_set/
  qemu-iotests: make 218 executable
  scripts/qemu.py: remove trailing quotes on docstring
  scripts/decodetree.py: remove unused imports
  docs/devel/testing.rst: add missing newlines after code block
  qemu-iotests: fix filename containing checks
  tests/tcg/README: fix location for lm32 tests
  memory.h: fix typos in comments
  vga_int: remove unused function protype
  configs/alpha: Remove unused CONFIG_PARALLEL_ISA switch

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-10-30 15:49:55 +00:00
Stefan Weil
e50a61219f tests: Fix typos in comments and help message (found by codespell)
Fix also a grammar issue.

Signed-off-by: Stefan Weil <sw@weilnetz.de>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20180713054755.23323-1-sw@weilnetz.de>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
2018-10-26 17:17:32 +02:00
Paolo Bonzini
1578466c9a tests: docker: update test-mingw for GTK+ 2.0 removal
--with-gtkabi does not exist anymore; remove it from the configure invocation.

Fixes: 89d85cde75
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <1539886203-33670-1-git-send-email-pbonzini@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Fam Zheng <famz@redhat.com>
2018-10-26 22:03:21 +08:00
Mao Zhongyi
b91068cd34 debian-bootstrap.pre: Modern shell scripting (use $() instead of ``)
Various shell files contain a mix between obsolete ``
and modern $(); It would be nice to convert to using $()
everywhere.

Signed-off-by: Mao Zhongyi <maozhongyi@cmss.chinamobile.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2018-10-24 07:39:03 +01:00
Eduardo Habkost
ae68fdaba9 Revert "docker.py: Python 2.6 argparse compatibility"
This reverts commit c2d3189667.

scripts/argparse.py was removed from the tree, so we don't need
this hack anymore.

Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Message-Id: <20180618225131.13113-3-ehabkost@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2018-10-03 23:08:51 -03:00
Philippe Mathieu-Daudé
b28ef6b9eb docker: Clean dangling tarball files
When a container fails, it leaves a dangling tarball which name is
based on a timestamp. Further uses of make won't clean those files,
neither calling the 'docker-clean' target.

Use the .DELETE_ON_ERROR built-in target to let make remove those
temporary tarballs in case of failure.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20180818030337.22271-1-f4bug@amsat.org>
Signed-off-by: Fam Zheng <famz@redhat.com>
2018-09-10 11:50:06 +08:00
Philippe Mathieu-Daudé
25b8f085ac docker: Sort CentOS packages
As recommended in https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#sort-multi-line-arguments

  "This helps to avoid duplication of packages and make the
  list much easier to update."

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20180818015344.797-5-f4bug@amsat.org>
Signed-off-by: Fam Zheng <famz@redhat.com>
2018-09-10 11:50:06 +08:00
Philippe Mathieu-Daudé
2808b467dd docker: Sort Ubuntu packages
As recommended in https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#sort-multi-line-arguments

  "This helps to avoid duplication of packages and make the
  list much easier to update. This also makes PRs a lot easier
  to read and review."

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20180818015344.797-4-f4bug@amsat.org>
Signed-off-by: Fam Zheng <famz@redhat.com>
2018-09-10 11:50:06 +08:00
Philippe Mathieu-Daudé
dc587d450a docker: Sort Fedora packages (i386-cross)
As recommended in https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#sort-multi-line-arguments

  "This helps to avoid duplication of packages and make the
  list much easier to update. This also makes PRs a lot easier
  to read and review."

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20180818015344.797-3-f4bug@amsat.org>
Signed-off-by: Fam Zheng <famz@redhat.com>
2018-09-10 11:50:06 +08:00
Philippe Mathieu-Daudé
5f8ab4fdf2 docker: Sort Fedora packages
As recommended in https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#sort-multi-line-arguments

  "This helps to avoid duplication of packages and make the
  list much easier to update. This also makes PRs a lot easier
  to read and review."

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20180818015344.797-2-f4bug@amsat.org>
Signed-off-by: Fam Zheng <famz@redhat.com>
2018-09-10 11:50:06 +08:00
Fam Zheng
73fb4f1de3 tests: Allow overriding archive path with SRC_ARCHIVE
In VM based tests, the source archive is created in host, we don't have
to run archive-source.sh again, as it complicates the Makefile and
scripts.

Signed-off-by: Fam Zheng <famz@redhat.com>
Message-Id: <20180712012829.20231-4-famz@redhat.com>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Fam Zheng <famz@redhat.com>
2018-08-15 10:12:35 +08:00
Fam Zheng
8158ed48bb docker: Install more packages in centos7
This makes test-block work.

Signed-off-by: Fam Zheng <famz@redhat.com>
Message-Id: <20180711065813.14894-1-famz@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Fam Zheng <famz@redhat.com>
2018-08-15 10:12:35 +08:00
Alex Bennée
15352decf8 docker: perform basic binfmt_misc validation in docker.py
Setting up binfmt_misc is outside of the scope of the docker.py script
but we can at least validate it with any given executable so we have a
more useful error message than the sed line of deboostrap failing
cryptically.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reported-by: Richard Henderson <richard.henderson@linaro.org>
2018-07-24 11:45:25 +01:00
Alex Bennée
f7b446b5de docker: ignore distro versioning of debootstrap
We do a minimum version check for the debootstrap but if the distro
has added their own minor version tick it would fail and fall-back to
the SCM version. This is sub-optimal as the latest/greatest version
may be broken at any one particular time. We fix that with a little
sed magic on the version string before passing to our ugly shell
versioning check.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-07-24 11:45:25 +01:00
Alex Bennée
11cc24a126 docker: add commentary to debian-bootstrap.docker
This is just a note that later versions of debootstrap don't
technically need this hack.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-07-24 11:45:25 +01:00
Philippe Mathieu-Daudé
571fef5e07 docker: Update debootstrap script after Debian migration from Alioth to Salsa
This silents the following warning:

  Cloning into './debootstrap.git'...
  warning: redirecting to https://salsa.debian.org/installer-team/debootstrap.git/

See https://lists.debian.org/debian-devel-announce/2018/01/msg00004.html

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2018-07-24 11:45:25 +01:00
Alex Bennée
d19f5fc041 docker: report hint when docker.py check fails
When a check fails we currently just report why we failed. This is not
totally helpful to people who want to boot-strap a new image. Report a
hint as to why it failed.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Suggested-by: Fam Zheng <famz@redhat.com>
2018-07-24 11:45:25 +01:00
Alex Bennée
2796346069 docker: drop QEMU_TARGET check, fallback in EXECUTABLE not set
The addition of QEMU_TARGET was intended to ensure we fall back to
checking for the existence of an image if the build system was not
currently configured to build it. However this breaks the direct use
of the rule for building custom binfmt_misc images. We already check
for EXECUTABLE so let us just use that as a proxy for deciding if we
are just going to check the image exits.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-07-24 11:45:25 +01:00
Alex Bennée
ff1a390296 docker: add expansion for docker-test-FOO to Makefile.include
This allows us to run a particular test on all docker images. For
example:

  make docker-test-unit

Will run the unit tests on every supported image. At the same time
rename docker-test to docker-all-tests to be clearer.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-07-24 11:45:25 +01:00
Alex Bennée
d984f24c77 docker: add test-unit runner
This test doesn't even build QEMU, it just builds and runs all the
unit tests. Intended to make checking unit tests on all docker images
easier.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-07-24 11:45:25 +01:00
Alex Bennée
77b08f73c8 docker: Makefile.include don't include partial images
Rename DOCKER_INTERMEDIATE_IMAGES to DOCKER_PARTIAL_IMAGES and add the
incomplete cross compiler images that can build tests but can't build
QEMU itself. We also add debian, debian-bootstrap and the tricode
images to the list.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-07-24 11:45:25 +01:00
Alex Bennée
56c115a953 docker: gracefully skip check_qemu
Not all our images are able to run the tests. Rather than use features
we can just check for the existence and run-ability of gtester. If the
image has been setup for binfmt_misc it will be able to run anyway.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-07-24 11:45:25 +01:00
Alex Bennée
3f9747a738 docker: move make check into check_qemu helper
Not all docker images can run the check step. Let's move everything
into a common helper so we don't need to replicate checks in the
future.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-07-24 11:45:25 +01:00
Alex Bennée
e4ce964d94 docker: split configure_qemu from build_qemu
This allows some tests that just want to configure QEMU's source tree
to do so.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-07-24 11:45:25 +01:00
Alex Bennée
43e1b2ffec docker: fail more gracefully on docker.py check
As this is called directly from the Makefile while determining
dependencies and it is possible the user was configured in one window
but not have credentials in the other. Let's catch the Exceptions and
deal with it quietly.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reported-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-07-24 11:45:25 +01:00
Alex Bennée
be5e1f527a docker: par down QEMU_CONFIGURE_OPTS in debian-tricore-cross
This image isn't going to build anything significant as it is just
intended for building test cases. In case it does end up getting
inadvertently included in a build lets aim for the minimal possible
product.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-07-24 11:45:25 +01:00
Alex Bennée
e043d73212 docker: base debian-tricore on qemu:debian9
We need both git and a working compiler to build the tools. Although
the qemu:debian9 image also has a bunch of extra dependencies it would
be fairly unusual for a user not to already have this layer available
for one of our many other docker images so lets not complicate things.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-07-24 11:45:25 +01:00
Alex Bennée
19c9a18f45 docker: add linux-user powered cross builder for QEMU
We can't use cross compilers in the current Debian stable and Debian
sid is sketchy as hell. So for powerpc fall back to dog-fooding our
own linux-user to do the build.

As we can only build the base image with a suitably configured
source tree we fall back to checking for its existence when we can't
build it from scratch. However this does mean you don't have to keep
a static powerpc-linux-user in your active configuration just to
update the cross build image.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-07-05 16:57:39 +01:00
Alex Bennée
dc338fdb07 docker: add special rule for deboostrapped images
We might as well have a custom rule for this. For one thing the
dependencies are different. As the primary dependency for
docker-image-% could never be docker-image-debian-bootstrap we can
drop that test in the main rule as well.

Missing EXECUTABLE, DEB_ARCH and DEB_TYPE are treated as hard faults
now. We also error out if the EXECUTABLE file isn't there. We should
really do this with a dependency on any source rules but currently
subdir-FOO-linux-user isn't enough on a clean build.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-07-05 15:59:41 +01:00
Alex Bennée
547cb45ea3 docker: add special handling for FROM:debian-%-user targets
These will have been build with debootstrap so we need to check
against the debian-bootstrap dockerfile. This does mean sticking to
debian-FOO-user as the naming conventions for boot-strapped images.
The actual cross image is built on top.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-07-05 15:59:41 +01:00
Alex Bennée
300cf467fd docker: debian-bootstrap.pre allow customising of variant/url
We default to the buildd variant as most of our images are for
building. However lets give the user the ability to specify "minbase"
if they want to create a simple base image for experimentation.

Allowing the tweaking of DEB_URL means we can also bootstrap other
Debian based OS's. For example:

  make docker-binfmt-image-debian-ubuntu-bionic-arm64 \
       DEB_ARCH=arm64 DEB_TYPE=bionic \
       DEB_VARIANT=minbase DEB_URL=http://ports.ubuntu.com/ \
       EXECUTABLE=./aarch64-linux-user/qemu-aarch64

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-07-05 15:59:41 +01:00
Alex Bennée
e6bfdeca8b docker: drop QEMU build-dep from bootstrap
This is best done with any child images that actually need it.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2018-07-05 15:59:41 +01:00
Philippe Mathieu-Daudé
272e551b43 docker: Do not run tests in 'intermediate' images
We can still build the DOCKER_INTERMEDIATE_IMAGES images,
but they won't appear in 'make test*@$IMAGE'.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2018-07-05 15:59:41 +01:00
Philippe Mathieu-Daudé
48feb682db docker: Clean the MXE base image
Using the duplicated same package is confusing.

Reported-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2018-07-05 15:59:41 +01:00
Philippe Mathieu-Daudé
daf999f77a docker: ubuntu: Use SDL2
Do not test the deprecated API versions (see cabd358407).
Debian MXE MinGW cross images are already using SDL2.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2018-07-05 15:59:41 +01:00
Philippe Mathieu-Daudé
beac6a98f6 docker: ubuntu: Update the package list before installing new ones
Since docker caches the different layers, updating the package
list does not invalidate the previous "apt-get update" layer,
and it is likely "apt-get install" hits an outdated repository.

See https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#apt-get

This fixes:

  $ make docker-image-ubuntu V=1
  ./tests/docker/docker.py build qemu:ubuntu tests/docker/dockerfiles/ubuntu.docker   --add-current-user
  Sending build context to Docker daemon  3.072kB
  [...]
  E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/m/mesa/libgles2-mesa_17.0.7-0ubuntu0.16.04.2_amd64.deb  404  Not Found
  E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/m/mesa/libgles2-mesa-dev_17.0.7-0ubuntu0.16.04.2_amd64.deb  404  Not Found
  E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
  The command '/bin/sh -c apt-get -y install $PACKAGES' returned a non-zero code: 100
  tests/docker/Makefile.include:40: recipe for target 'docker-image-ubuntu' failed
  make: *** [docker-image-ubuntu] Error 1

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2018-07-05 15:59:41 +01:00
Alex Bennée
95f7aabf49 docker: add gcovr to travis image
Useful for debugging if nothing else as the gcovr on the Travis images
are a little old.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2018-07-05 15:59:41 +01:00