Commit Graph

582 Commits

Author SHA1 Message Date
Richard Henderson
e3acc2c196 tests/docker/dockerfiles: Bump fedora-i386-cross to fedora 34
For unknown and unrepeatable reasons, the cross-i386-tci test has
started failing.  "Fix" this by updating the container to use fedora 34.

Add sysprof-capture-devel as a new dependency of glib2-devel that
was not correctly spelled out in the rpm rules.

Use dnf update Just In Case -- there are presently out-of-date
packages in the upstream docker registry.

Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20211005205846.153724-1-richard.henderson@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-10-05 16:40:39 -07:00
Gerd Hoffmann
8a2d766f05 ci: add libusb for windows builds
Add CI coverage for usb passthrough on windows.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210623085249.1151901-3-kraxel@redhat.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
2021-07-29 11:18:24 +02:00
Alex Bennée
39ce923732 gitlab: enable a very minimal build with the tricore container
Rather than base of the shared Debian 10 container which would require
us to bring in even more dependencies just bring in what is needed for
building tricore-softmmu in GitLab. We don't even remove the container
from the DOCKER_PARTIAL_IMAGES lest we cause more confusion.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210720232703.10650-28-alex.bennee@linaro.org>
2021-07-23 17:22:16 +01:00
Gerd Hoffmann
8619b5ddb5 ci: build & store windows installer
Build windows installer for qemu in gitlab CI,
store the result as artifact.

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20210623091137.1156959-2-kraxel@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-07-19 09:33:39 +02:00
Daniel P. Berrangé
5606ce9ee4 tests/docker: expand opensuse-leap package list
This is the fully expanded list of build pre-requisites QEMU can
conceivably use in any scenario.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210623142245.307776-16-berrange@redhat.com>
Message-Id: <20210709143005.1554-24-alex.bennee@linaro.org>
2021-07-14 14:33:53 +01:00
Daniel P. Berrangé
bc4a117b79 tests/docker: expand ubuntu2004 package list
This is the fully expanded list of build pre-requisites QEMU can
conceivably use in any scenario.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210623142245.307776-15-berrange@redhat.com>
Message-Id: <20210709143005.1554-23-alex.bennee@linaro.org>
2021-07-14 14:33:53 +01:00
Daniel P. Berrangé
0fef572e43 tests/docker: expand ubuntu1804 package list
This is the fully expanded list of build pre-requisites QEMU can
conceivably use in any scenario.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210623142245.307776-14-berrange@redhat.com>
Message-Id: <20210709143005.1554-22-alex.bennee@linaro.org>
2021-07-14 14:33:53 +01:00
Daniel P. Berrangé
5a9c1498e3 tests/docker: expand fedora package list
This is the fully expanded list of build pre-requisites QEMU can
conceivably use in any scenario.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210623142245.307776-13-berrange@redhat.com>
Message-Id: <20210709143005.1554-21-alex.bennee@linaro.org>
2021-07-14 14:33:53 +01:00
Daniel P. Berrangé
b5883710c4 tests/docker: expand centos8 package list
This is the fully expanded list of build pre-requisites QEMU can
conceivably use in any scenario.

[AJB: added centos-release-advanced-virtualization/epel-release]

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210623142245.307776-12-berrange@redhat.com>
Message-Id: <20210709143005.1554-20-alex.bennee@linaro.org>
2021-07-14 14:33:53 +01:00
Daniel P. Berrangé
52141ab37d tests/docker: remove mingw packages from Fedora
There are dedicated containers providing mingw packages for Fedora.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210623142245.307776-11-berrange@redhat.com>
Message-Id: <20210709143005.1554-19-alex.bennee@linaro.org>
2021-07-14 14:33:53 +01:00
Daniel P. Berrangé
dfaaacc67f tests/docker: fix mistakes in ubuntu package lists
librados-dev is not required by QEMU directly, only librbd-dev.

glusterfs-common is not directly needed by QEMU.

QEMU uses ncursesw only on non-Windows hosts.

The clang package is clang 10.

flex and bison are not required by QEMU.

Standardize on nmap ncat implementation to match Fedora/CentOS.

Remove vim since it is not a build pre-requisite and no other containers
include it.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210623142245.307776-10-berrange@redhat.com>
Message-Id: <20210709143005.1554-18-alex.bennee@linaro.org>
2021-07-14 14:33:53 +01:00
Daniel P. Berrangé
e053de7e63 tests/docker: fix mistakes in fedora package list
libblockdev-mpath-devel is not used by QEMU, rather it wants
device-mapper-multipath-devel.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210623142245.307776-9-berrange@redhat.com>
Message-Id: <20210709143005.1554-17-alex.bennee@linaro.org>
2021-07-14 14:33:53 +01:00
Daniel P. Berrangé
1d3a56d7f1 tests/docker: fix mistakes in centos package lists
mesa-libEGL-devel is not used in QEMU at all, but mesa-libgbm-devel is.

spice-glib-devel is not use in QEMU at all, but spice-protocol is.
We also need the -devel package for spice-server, not the runtime.

There is no need to specifically refer to python36, we can just
use python3 as in other distros.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210623142245.307776-8-berrange@redhat.com>
Message-Id: <20210709143005.1554-16-alex.bennee@linaro.org>
2021-07-14 14:33:53 +01:00
Daniel P. Berrangé
52dab55657 tests/docker: fix sorting in package lists
This will make diffs in later patches clearer.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210623142245.307776-7-berrange@redhat.com>
Message-Id: <20210709143005.1554-15-alex.bennee@linaro.org>
2021-07-14 14:33:53 +01:00
Daniel P. Berrangé
888673bbb9 tests/docker: remove FEATURES env var from templates
In preparation for switching to auto-generated dockerfiles, remove the
FEATURES env variable. The equivalent functionality can be achieved in
most cases by just looking for existance of a binary.

The cases which don't correspond to binaries are simply dropped because
configure/meson will probe for any requested feature anyway.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210623142245.307776-6-berrange@redhat.com>
Message-Id: <20210709143005.1554-14-alex.bennee@linaro.org>
2021-07-14 14:33:53 +01:00
Daniel P. Berrangé
1ae2786c7c tests/docker: use explicit docker.io registry
It is good practice to use an explicit registry for referencing the base
image. This is because some distros will inject their own registries
into the search path. For example registry.fedoraproject.org comes ahead
of docker.io. Using an explicit registry avoids wasting time querying
multiple registries for images that they won't have.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210623142245.307776-5-berrange@redhat.com>
Message-Id: <20210709143005.1554-13-alex.bennee@linaro.org>
2021-07-14 14:33:53 +01:00
Daniel P. Berrangé
102cd5c294 tests/docker: use project specific container registries
Since Docker Hub has started to enforce pull rate limits on clients, it
is preferrable to use project specific container registries where they
are available. Fedora and OpenSUSE projects provide registries.

The images in these registries are also refreshed on a more regular
basis than the ones in docker hub, so the package update should
generally be faster.

While CentOS also has a registry it is considerably outdated compared
to docker.io, and also only provides x86 images, while docker.io images
are multi-arch.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210623142245.307776-4-berrange@redhat.com>
Message-Id: <20210709143005.1554-12-alex.bennee@linaro.org>
2021-07-14 14:33:53 +01:00
Daniel P. Berrangé
6ddc3dc7a8 tests/docker: don't use BUILDKIT in GitLab either
Using BUILDKIT breaks with certain container registries such as CentOS,
with docker build reporting an error such as

  failed to solve with frontend dockerfile.v0:
  failed to build LLB: failed to load cache key:
  unexpected status code
  https://registry.centos.org/v2/centos/manifests/7:
  403 Forbidden

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210623142245.307776-3-berrange@redhat.com>
Message-Id: <20210709143005.1554-11-alex.bennee@linaro.org>
2021-07-14 14:33:53 +01:00
Daniel P. Berrangé
51f5c849c1 hw/usb/ccid: remove references to NSS
The NSS package was previously pre-requisite for building CCID related
features, however, this became obsolete when the libcacard library was
spun off to a separate project:

    commit 7b02f5447c
    Author: Marc-André Lureau <marcandre.lureau@redhat.com>
    Date:   Sun Aug 30 11:48:40 2015 +0200

        libcacard: use the standalone project

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210623142245.307776-2-berrange@redhat.com>
Message-Id: <20210709143005.1554-10-alex.bennee@linaro.org>
2021-07-14 14:33:53 +01:00
Richard Henderson
22f15579fa tcg: Build ffi data structures for helpers
Add libffi as a build requirement for TCI.
Add libffi to the dockerfiles to satisfy that requirement.

Construct an ffi_cif structure for each unique typemask.
Record the result in a separate hash table for later lookup;
this allows helper_table to stay const.

Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2021-06-19 08:51:11 -07:00
Peter Maydell
8c345b3e6a * Update the references to some doc files (use *.rst instead of *.txt)
* Bump minimum versions of some requirements after removing CentOS 7 support
 -----BEGIN PGP SIGNATURE-----
 
 iQJFBAABCAAvFiEEJ7iIR+7gJQEY8+q5LtnXdP5wLbUFAmC3L1IRHHRodXRoQHJl
 ZGhhdC5jb20ACgkQLtnXdP5wLbU8wA//aRAIn0Qa3xoZdsT434P99GHauCZ5ePq3
 meY4co69c+TfkQZ/b0xlyvT3+7bd9Ni92CQL7n/LtX5bs3pIhRjHNIsm50e4x6Da
 0jb422cKWfRIltXlCfdL/dnFRtJluH83M0seRGMvmhveuWPZ19oSIasEiQjeg//e
 KMNK5z4HEnJ1czb3Bf8p38bmmY/O/QEAA5wAqk7iJkJHz6T/GlqImLYYwpFPlHj6
 JQttm0aWsHrsqEXxnuV0/DT1yHyXDB6S4iuAvABZWhv/M/nCaXo0ib0gW5NPtRPo
 Yf3HO163F9/fewJCc1AUsBe1C/2UwmSWRhEtxpr9uuW2Mv9qEl3hkJwd4k6sEOvh
 U4i+GONC4eElPcmECKUfHA9EP+7faDs6xnM6Ev/PIEp+cPJ2QRfklZv4qpMUWYtb
 3KkADchOyVZAsdB8cGmnznDEVmno1Dt0adVUq8CF6uW6MwD3pb4838arrfwIfwOp
 g4yTI1AQQykkxxOaR7GSNoxWRti7TH4fzLhx/xXDd9TKIOWuOepiyuhB7+Q48WAJ
 6EW/JUIzOe7k3GsI0iBsk+y67ED2tpATiRKWWw4QS1BwIhNEMUqgZkucFfKE6Ze1
 B1Xw+Di+1CzrLHV6lCjtdLVmqWqcGcO43HqhSK4sodE87BDGGwy9UxE6oBVt2C1x
 HPQ6/omDlFk=
 =RAkI
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/thuth-gitlab/tags/pull-request-2021-06-02' into staging

* Update the references to some doc files (use *.rst instead of *.txt)
* Bump minimum versions of some requirements after removing CentOS 7 support

# gpg: Signature made Wed 02 Jun 2021 08:12:18 BST
# gpg:                using RSA key 27B88847EEE0250118F3EAB92ED9D774FE702DB5
# gpg:                issuer "thuth@redhat.com"
# gpg: Good signature from "Thomas Huth <th.huth@gmx.de>" [full]
# gpg:                 aka "Thomas Huth <thuth@redhat.com>" [full]
# gpg:                 aka "Thomas Huth <huth@tuxfamily.org>" [full]
# gpg:                 aka "Thomas Huth <th.huth@posteo.de>" [unknown]
# Primary key fingerprint: 27B8 8847 EEE0 2501 18F3  EAB9 2ED9 D774 FE70 2DB5

* remotes/thuth-gitlab/tags/pull-request-2021-06-02:
  configure: bump min required CLang to 6.0 / XCode 10.0
  configure: bump min required GCC to 7.5.0
  configure: bump min required glib version to 2.56
  tests/docker: drop CentOS 7 container
  tests/vm: convert centos VM recipe to CentOS 8
  crypto: drop used conditional check
  crypto: bump min gnutls to 3.5.18, dropping RHEL-7 support
  crypto: bump min gcrypt to 1.8.0, dropping RHEL-7 support
  crypto: drop back compatibility typedefs for nettle
  crypto: bump min nettle to 3.4, dropping RHEL-7 support
  patchew: move quick build job from CentOS 7 to CentOS 8 container
  block/ssh: Bump minimum libssh version to 0.8.7
  docs: fix references to docs/devel/s390-dasd-ipl.rst
  docs: fix references to docs/specs/tpm.rst
  docs: fix references to docs/devel/build-system.rst
  docs: fix references to docs/devel/atomics.rst
  docs: fix references to docs/devel/tracing.rst

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-06-02 17:08:11 +01:00
Daniel P. Berrangé
6f0748a048 tests/docker: drop CentOS 7 container
It has been over two years since RHEL-8 was released, and thus per the
platform build policy, we no longer need to support RHEL-7 as a build
target.

Reviewed-by: Thomas Huth <thuth@redhat.com>
Acked-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20210514120415.1368922-10-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-06-02 09:11:32 +02:00
John Snow
6b9c277797 gitlab: add python linters to CI
Add a Python container that has just enough juice for us to run the
Python code quality analysis tools. Base this container on Fedora,
because Fedora has very convenient packaging for testing multiple Python
versions.

We need python3, pip (for pulling packages), pipenv and virtualenv for
creating virtual environments, and tox for running tests. make is needed
for running 'make check-tox' and 'make venv-check' targets. Python3.10
is needed explicitly because the tox package only pulls in 3.6-3.9, but
we wish to test the forthcoming release of Python as well to help
predict any problems. Lastly, we need gcc to compile PyPI packages that
may not have a binary distribution available.

Add two tests:

check-python-pipenv uses pipenv to test a frozen, very explicit set of
packages against our minimum supported python version, Python 3.6. This
test is not allowed to fail. The dependencies this test uses do not
change unless python/Pipfile.lock is changed.

check-python-tox uses tox to install the latest versions of required
python dependencies against a wide array of Python versions from 3.6 to
3.9, even including the yet-to-be-released Python 3.10. This test is
allowed to fail with a warning.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Message-id: 20210527211715.394144-32-jsnow@redhat.com
[Fix rebase conflict over .gitlab-ci.yml --js]
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01 16:21:21 -04:00
Matheus Ferst
cf9b6d215e tests/docker: gcc-10 based images for ppc64{,le} tests
A newer compiler is needed to build tests for Power10 instructions. As
done for arm64 on c729a99d27, a new '-test-cross' image is created for
ppc64 and ppc64le. As done on 936fda4d77, a test for compiler support
is added to verify that the toolchain in use has '-mpower10'. Finally,
Unused images (docker-power-cross and docker-ppc64-cross) are removed.

Signed-off-by: Matheus Ferst <matheus.ferst@eldorado.org.br>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210423205757.1752480-2-matheus.ferst@eldorado.org.br>
Message-Id: <20210512102051.12134-27-alex.bennee@linaro.org>
2021-05-18 09:36:21 +01:00
Andrew Melnychenko
6a7e70006a tests/docker: Added libbpf library to the docker files.
The series of patches for eBPF RSS adds libbpf dependency for qemu.
https://lists.gnu.org/archive/html/qemu-devel/2021-03/msg08887.html

With this patch, libbpf added:
  Alpine - added libbpf-dev
  Centos 8 - added libbpf-devel
  Fedora - added libbpf-devel

Signed-off-by: Andrew Melnychenko <andrew@daynix.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20210406082947.672708-2-andrew@daynix.com>
Message-Id: <20210512102051.12134-11-alex.bennee@linaro.org>
2021-05-18 09:36:21 +01:00
Alessandro Di Federico
afbdf0a44e docker: Add Hexagon image
This image is a little special because it takes a long time to build.
As such most users don't want to be doing that and just pull random
binaries from the ether as intended by the container gods. This
involves someone with credentials and a beefy machine running:

  make docker-image-debian-hexagon-cross V=1 NOCACHE=1 J=30
  docker tag qemu/debian-hexagon-cross registry.gitlab.com/qemu-project/qemu/qemu/debian-hexagon-cross
  docker push registry.gitlab.com/qemu-project/qemu/qemu/debian-hexagon-cross

With a suitable binary in the "cloud" a normal user will run:

  make docker-image-debian-hexagon-cross

or have it run for them through the dependency mechanism of our
over-engineered makefiles and get the binary they wanted. There are a
few wrinkles of course including needing to tweak the final image to
have the credentials of the user so we can actually do our cross
compiles.

Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Cc: Alessandro Di Federico <ale@rev.ng>
Cc: Philippe Mathieu-Daudé <f4bug@amsat.org>
Cc: Brian Cain <bcain@quicinc.com>

Message-Id: <20210512102051.12134-7-alex.bennee@linaro.org>
2021-05-18 09:35:39 +01:00
Alex Bennée
c3ad904393 tests/docker: add "fetch" sub-command
This simply wraps up fetching a build from the registry and tagging it
as the local build.

Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210512102051.12134-6-alex.bennee@linaro.org>
2021-05-18 09:35:39 +01:00
Alex Bennée
bf46c0eed3 tests/docker: allow "update" to add the current user
The current user functionality is used for cross compiling to avoid
complications with permissions when building test programs. However
for images that come from the registry we still need the ability to
add the user after the fact.

Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210512102051.12134-5-alex.bennee@linaro.org>
2021-05-18 09:35:39 +01:00
Alex Bennée
8d628d075d tests/docker: make executable an optional argument to "update"
We're going to extend the abilities of the command shortly.

Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

Message-Id: <20210512102051.12134-4-alex.bennee@linaro.org>
2021-05-18 09:35:39 +01:00
Alex Bennée
3218d829e3 tests/docker: fix copying of executable in "update"
We have the same symlink chasing problem when doing an "update"
operation. Fix that.

Based-on: 5e33f7fead ("tests/docker: better handle symlinked libs")
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210512102051.12134-3-alex.bennee@linaro.org>
2021-05-18 09:35:35 +01:00
Peter Maydell
9b1e81d1c2 * Replace YAML anchors by extends in the gitlab-CI yaml files
* Many small qtest fixes (e.g. to fix issues discovered by Coverity)
 * Poison more config switches in common code
 * Fix the failing Travis-CI and Cirrus-CI tasks
 -----BEGIN PGP SIGNATURE-----
 
 iQJFBAABCAAvFiEEJ7iIR+7gJQEY8+q5LtnXdP5wLbUFAmCeXFMRHHRodXRoQHJl
 ZGhhdC5jb20ACgkQLtnXdP5wLbUV7g//VRxN74v2pO6zSsSNavYTkMa3jZE1Io3l
 w9lsOr3DM0HIMhyEMwSJsiygj0s8TNanUtFBHfDfo1k1gmZYd5FiM1sB7ZB/sB/S
 9Q9wjmeV2NMG7pcr9zLmiJaEM2LGfGbso44/m0c+gpSyxzg2TeH7sF+38AUZI9/R
 J6/gOzIs4WWdSV4f8kp+YPPQCyOtxZsfxDuk1z1fKsBgXE5iEBUqYyyZUm4gYJkN
 4ALmqc/NVeLszt08qkPHxwXQc488nCJP31psxx6MQ7toKfCAamT07Pp3oi70cakC
 y+HVfsIHc7SxaZyj8sbJCU3LJHwDd8N82ydJUrv216qDffb38fNb+m6y9mcYy2MH
 C25uGe9mucTuTP1WIttC6nCKg/MCgi4PWIzqEhkeKj3TxpTUJJP+BUIfSubV38Gc
 T+XUCNkWWW8sTeRiyE3m9pEZ+gz1MFubaIr/Owephch1SjRYn4zUwJFHHi3I4PY4
 7XjEo8y2M9tKckW3pCfYi+aIDzC1DcRtzvXUGtdemX5xVjTAGlKkFLWl/brdCn2U
 y+JrwrL8cQ3Fy7bXzmK6m9mmhcIW6PcSOBE7RnSESyKzqM5VBSBgnjMqnHcP3FrV
 FYzihTCDcFKy3ELUe3Kbc1TgG/07stQs9xInGXN7ZFsIwadfHgoNVQ6JpOqq1oQa
 fvmLPBhq+a4=
 =QZuG
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/thuth-gitlab/tags/pull-request-2021-05-14' into staging

* Replace YAML anchors by extends in the gitlab-CI yaml files
* Many small qtest fixes (e.g. to fix issues discovered by Coverity)
* Poison more config switches in common code
* Fix the failing Travis-CI and Cirrus-CI tasks

# gpg: Signature made Fri 14 May 2021 12:17:39 BST
# gpg:                using RSA key 27B88847EEE0250118F3EAB92ED9D774FE702DB5
# gpg:                issuer "thuth@redhat.com"
# gpg: Good signature from "Thomas Huth <th.huth@gmx.de>" [full]
# gpg:                 aka "Thomas Huth <thuth@redhat.com>" [full]
# gpg:                 aka "Thomas Huth <huth@tuxfamily.org>" [full]
# gpg:                 aka "Thomas Huth <th.huth@posteo.de>" [unknown]
# Primary key fingerprint: 27B8 8847 EEE0 2501 18F3  EAB9 2ED9 D774 FE70 2DB5

* remotes/thuth-gitlab/tags/pull-request-2021-05-14:
  cirrus.yml: Fix the MSYS2 task
  pc-bios/s390-ccw: Fix inline assembly for older versions of Clang
  tests/qtest/migration-test: Use g_autofree to avoid leaks on error paths
  configure: Poison all current target-specific #defines
  migration: Move populate_vfio_info() into a separate file
  include/sysemu: Poison all accelerator CONFIG switches in common code
  tests: Avoid side effects inside g_assert() arguments
  tests/qtest/rtc-test: Remove pointless NULL check
  tests/qtest/tpm-util.c: Free memory with correct free function
  tests/migration-test: Fix "true" vs true
  tests/qtest/npcm7xx_pwm-test.c: Avoid g_assert_true() for non-test assertions
  tests/qtest/ahci-test.c: Calculate iso_size with 64-bit arithmetic
  util/compatfd.c: Replaced a malloc call with g_malloc.
  libqtest: refuse QTEST_QEMU_BINARY=qemu-kvm
  docs/devel/qgraph: add troubleshooting information
  libqos/qgraph: fix "UNAVAILBLE" typo
  gitlab-ci: Replace YAML anchors by extends (native_test_job)
  gitlab-ci: Replace YAML anchors by extends (native_build_job)
  gitlab-ci: Replace YAML anchors by extends (container_job)
  tests/docker/dockerfiles: Add ccache to containers where it was missing

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-05-14 19:33:23 +01:00
Marc-André Lureau
73e6aec652 sphinx: adopt kernel readthedoc theme
The default "alabaster" sphinx theme has a couple shortcomings:
- the navbar moves along the page
- the search bar is not always at the same place
- it lacks some contrast and colours

The "rtd" theme from readthedocs.org is a popular third party theme used
notably by the kernel, with a custom style sheet. I like it better,
perhaps others do too. It also simplifies the "Edit on Gitlab" links.

Tweak a bit the custom theme to match qemu.org style, use the
QEMU logo, and favicon etc.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Tested-by: Bin Meng <bmeng.cn@gmail.com>
Message-Id: <20210323115328.4146052-1-marcandre.lureau@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
2021-05-14 15:05:03 +04:00
Thomas Huth
5a487950f9 tests/docker/dockerfiles: Add ccache to containers where it was missing
Make sure that ccache is available in all containers.

Message-Id: <20210414081907.871437-2-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-05-14 12:24:20 +02:00
Thomas Huth
a2e5bbf0c4 gitlab-ci.yml: Test the dtrace backend in one of the jobs
We are using the dtrace backend in downstream RHEL, so testing this
in the CentOS 8 task seems to be a good fit.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210331160351.3071279-1-thuth@redhat.com>
Message-Id: <20210401102530.12030-12-alex.bennee@linaro.org>
2021-04-06 15:04:50 +01:00
Alex Bennée
34019198a1 tests/docker: don't set DOCKER_REGISTRY on non-x86_64
Currently our gitlab registry is x86_64 only so attempting to pull an
image from it on something else will end in tears.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20210401102530.12030-3-alex.bennee@linaro.org>
2021-04-06 15:04:42 +01:00
Philippe Mathieu-Daudé
327910dea1 tests/docker: Use --arch-only when building Debian cross image
When building a Docker image based on debian10.docker on
a non-x86 host, we get:

 [2/4] RUN apt update &&     DEBIAN_FRONTEND=noninteractive eatmydata     apt build-dep -yy qemu
 Reading package lists... Done
 Building dependency tree
 Reading state information... Done
 Some packages could not be installed. This may mean that you have
 requested an impossible situation or if you are using the unstable
 distribution that some required packages have not yet been created
 or been moved out of Incoming.
 The following information may help to resolve the situation:

 The following packages have unmet dependencies:
  builddeps:qemu : Depends: gcc-s390x-linux-gnu but it is not installable
                   Depends: gcc-alpha-linux-gnu but it is not installable
 E: Unable to correct problems, you have held broken packages.

Fix by using the --arch-only option suggested here:
https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1866032/comments/1

Suggested-by: Christian Ehrhardt <christian.ehrhardt@canonical.com>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210223211115.2971565-1-f4bug@amsat.org>
Message-Id: <20210305092328.31792-10-alex.bennee@linaro.org>
2021-03-10 15:34:11 +00:00
Alex Bennée
dd5af6ece9 tests/docker: add a test-tcg for building then running check-tcg
This is mostly useful for verifying containers will work on the CI
setup.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20210305092328.31792-4-alex.bennee@linaro.org>
2021-03-10 12:13:59 +00:00
Philippe Mathieu-Daudé
9d66a0eada docker: Bump Fedora images to release 33
Fedora 33 was released on October 27, 2020.

Update all the Fedora 32 images to this new release.

Suggested-by: Daniel Berrangé <berrange@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20210118181255.314672-1-philmd@redhat.com>
Message-Id: <20210222101455.12640-3-alex.bennee@linaro.org>
2021-02-24 11:05:10 +00:00
Daniel P. Berrangé
d79f5c8b23 tests/docker: remove travis container
The travis container that we have no longer matches what travis
currently uses. As all x86 jobs are being moved to GitLab CI too,
there is no compelling reason to update the travis container. It
is simpler to just remove it.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20210209135011.1224992-2-berrange@redhat.com>
Message-Id: <20210211122750.22645-9-alex.bennee@linaro.org>
2021-02-15 09:38:33 +00:00
Philippe Mathieu-Daudé
04250c69bc travis.yml: Move gprof/gcov test across to gitlab
Similarly to commit 8cdb2cef3f, move the gprof/gcov test to GitLab.

The coverage-summary.sh script is not Travis-CI specific, make it
generic.

[thuth: Add gcovr and bsdmainutils which are required for the
        coverage-summary.sh script to the ubuntu docker file,
        and use 'check' as test target]

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20201108204535.2319870-10-philmd@redhat.com>
Message-Id: <20210211045455.456371-2-thuth@redhat.com>
Message-Id: <20210211122750.22645-2-alex.bennee@linaro.org>
2021-02-15 09:38:19 +00:00
Alex Bennée
ddd5ed8331 tests/docker: add a docker-exec-copy-test
This provides test machinery for checking the QEMU copying logic works
properly. It takes considerably less time to run than starting a
debootstrap only for it to fail later. I considered adding a remove
command to docker.py but figured that might be gold plating given the
relative size of the containers compared to the ones with actual stuff
in them.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210202134001.25738-7-alex.bennee@linaro.org>
2021-02-08 09:41:00 +00:00
Alex Bennée
6147c2495d tests/docker: alias docker-help target for consistency
We have a bunch of -help targets so this will save some cognitive
dissonance. Keep the original for those with muscle memory.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20210202134001.25738-6-alex.bennee@linaro.org>
2021-02-08 09:41:00 +00:00
Alex Bennée
3971c70f15 tests/docker: preserve original name when copying libs
While it is important we chase down the symlinks to copy the correct
data we can confuse the kernel by renaming the interpreter to what is
in the binary. Extend _copy_with_mkdir to preserve the original name
of the file when asked.

Fixes: 5e33f7fead ("tests/docker: better handle symlinked libs")
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210202134001.25738-5-alex.bennee@linaro.org>
2021-02-08 09:41:00 +00:00
Alex Bennée
dffccf3d34 tests/docker: make _copy_with_mkdir accept missing files
Depending on the linker/ldd setup we might get a file with no path.
Typically this is the psuedo library linux-vdso.so which doesn't
actually exist on the disk. Rather than try and catch these distro
specific edge cases just shout about it and try and continue.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20210202134001.25738-4-alex.bennee@linaro.org>
2021-02-08 09:41:00 +00:00
Philippe Mathieu-Daudé
dc23bbc3df tests/docker: Fix typo in help message
To have the variable properly passed, we need to set it,
ie. NOUSER=1. Fix the message displayed by 'make docker'.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210119052120.522069-1-f4bug@amsat.org>
Message-Id: <20210202134001.25738-3-alex.bennee@linaro.org>
2021-02-08 09:41:00 +00:00
Philippe Mathieu-Daudé
4d8f630915 tests/docker: Fix _get_so_libs() for docker-binfmt-image
Fix a variable rename mistake from commit 5e33f7fead:

  Traceback (most recent call last):
    File "./tests/docker/docker.py", line 710, in <module>
      sys.exit(main())
    File "./tests/docker/docker.py", line 706, in main
      return args.cmdobj.run(args, argv)
    File "./tests/docker/docker.py", line 489, in run
      _copy_binary_with_libs(args.include_executable,
    File "./tests/docker/docker.py", line 149, in _copy_binary_with_libs
      libs = _get_so_libs(src)
    File "./tests/docker/docker.py", line 123, in _get_so_libs
      libs.append(s.group(1))
  NameError: name 's' is not defined

Fixes: 5e33f7fead ("tests/docker: better handle symlinked libs")
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210119050149.516910-1-f4bug@amsat.org>
Message-Id: <20210202134001.25738-2-alex.bennee@linaro.org>
2021-02-08 09:41:00 +00:00
Thomas Huth
db0108d5d8 gitlab-ci.yml: Avoid some submodules to speed up the CI a little bit
Since the meson build system rework, the configure script prefers the
git submodules over the system libraries. So we are testing compilation
with capstone, fdt and libslirp as a submodule all over the place,
burning CPU cycles by recompiling these third party modules and wasting
some network bandwidth in the CI by cloning the submodules each time.
Let's stop doing that in at least a couple of jobs and use the system
libraries instead.

While we're at it, also install meson in the Fedora container, since
it is new enough already, so we do not need to check out the meson
submodule here.

Message-Id: <20210121174451.658924-1-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-01-26 18:38:37 +01:00
Philippe Mathieu-Daudé
4762c82cbd tests/docker: Install static libc package in CentOS 7
We need to install the static libc package to be able to run
the TCG tests:

  $ make check-tcg
  ...
    BUILD   TCG tests for x86_64-softmmu
    BUILD   x86_64-softmmu guest-tests with cc
  /usr/bin/ld: hello: warning: allocated section `.notes' not in segment
  /usr/bin/ld: memory: warning: allocated section `.notes' not in segment
    BUILD   TCG tests for x86_64-linux-user
    BUILD   x86_64-linux-user guest-tests with cc
  /usr/bin/ld: cannot find -lpthread
  /usr/bin/ld: cannot find -lc
  collect2: error: ld returned 1 exit status
  make[2]: *** [threadcount] Error 1
  make[1]: *** [cross-build-guest-tests] Error 2
  make: *** [build-tcg-tests-x86_64-linux-user] Error 2

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20210121172829.1643620-2-f4bug@amsat.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-01-26 15:39:34 +01:00
Jiaxun Yang
8df04b04ed tests/docker: Add dockerfile for Alpine Linux
Alpine Linux[1] is a security-oriented, lightweight Linux distribution
based on musl libc and busybox.

It it popular among Docker guests and embedded applications.

Adding it to test against different libc.

[1]: https://alpinelinux.org/

Signed-off-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
Message-Id: <20210118063808.12471-9-jiaxun.yang@flygoat.com>
[thuth: Dropped some unnecessary packages, replaced build-base with its deps]
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-01-20 10:46:54 +01:00
Alex Bennée
ec9d7c38ec docker: expand debian-amd64 image to include tag tools
This is going to be helpful when we want to both test the tool
integration and in the case of global generate a xref doc build.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210114165730.31607-9-alex.bennee@linaro.org>
2021-01-18 10:04:31 +00:00
Alessandro Di Federico
e405a3ebf5 Add newline when generating Dockerfile
Signed-off-by: Alessandro Di Federico <ale@rev.ng>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <1610080146-14968-36-git-send-email-tsimpson@quicinc.com>
Message-Id: <20210114165730.31607-5-alex.bennee@linaro.org>
2021-01-18 10:04:31 +00:00
Philippe Mathieu-Daudé
8714e811a5 tests/docker: Remove Debian 9 remnant lines
Debian 9 base container has been removed in commits
e3755276d1 and c9d78b06c0. Remove the last remnants.

Fixes: e3755276d1 ("tests/docker: Remove old Debian 9 containers")
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210107072933.3828450-1-f4bug@amsat.org>
Message-Id: <20210114165730.31607-2-alex.bennee@linaro.org>
2021-01-18 10:04:31 +00:00
Cho, Yu-Chen
b115ea3a0d gitlab-ci.yml: Add openSUSE Leap 15.2 for gitlab CI/CD
Add build-system-opensuse jobs and opensuse-leap.docker dockerfile.
Use openSUSE Leap 15.2 container image in the gitlab-CI.

Signed-off-by: Cho, Yu-Chen <acho@suse.com>
Tested-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20201229085046.8536-1-acho@suse.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-01-11 14:59:08 +01:00
Philippe Mathieu-Daudé
d2427956e1 tests/docker: Include 'ccache' in Debian base image
Include the 'ccache' package to speed up compilation.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20201213211601.253530-1-f4bug@amsat.org>
Fixes: d6db2a1cdf ("docker: add debian-buster-arm64-cross")
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2021-01-06 17:30:02 +00:00
Daniel P. Berrangé
90e0c9b309 tests: update for rename of CentOS8 PowerTools repo
This was intentionally renamed recently to be all lowercase:

https://bugs.centos.org/view.php?id=17920
https://wiki.centos.org/Manuals/ReleaseNotes/CentOS8.2011#Yum_repo_file_and_repoid_changes

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20201216141653.213980-1-berrange@redhat.com>
[AJB: bump up FROM to trigger re-build, add diffutils]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2021-01-02 21:03:09 +01:00
Thomas Huth
c9d78b06c0 tests/docker: Remove the remainders of debian9 containers from the Makefile
The Debian 9 containers have been removed a while ago, so we can
delete the corresponding entries in the Makefile, too.

Fixes: e3755276d1 ("tests/docker: Remove old Debian 9 containers")
Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20201215083318.92205-1-thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2021-01-02 21:03:09 +01:00
Thomas Huth
ee381b7fe1 gitlab-CI: Test 32-bit builds with the fedora-i386-cross container
After adding some missing packages, it's possible to check 32-bit
builds and tests with the fedora-i386-cross container in the gitlab-CI,
too. Unfortunately, the code in subprojects/ ignores the --extra-cflags
(on purpose), so the vhost-user part has to be disabled for this.

While we're at it, update the container to Fedora 31. Unfortunately the
gcc from the later versions emits some very dubious format-truncation
warnings, so Fedora 32 and 33 are currently unsuitable for this job.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20201215083451.92322-1-thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2021-01-02 21:03:09 +01:00
Peter Maydell
6cfdaa88cf Remove obsolete setuptools dependency and fix Stefan's
Win32 builds.
 -----BEGIN PGP SIGNATURE-----
 
 iQFIBAABCAAyFiEE8TM4V0tmI4mGbHaCv/vSX3jHroMFAl+/uegUHHBib256aW5p
 QHJlZGhhdC5jb20ACgkQv/vSX3jHroPMiwgAj6VB/lZP10CmKfNCIC6s9TjONP2I
 ruI3ZMYBrmeiLMf/WZTDI0kRY/mmYMtn4nN+RlEHlBE9a9hSCmHWvQyZDgEIJUOA
 1bTAdl4jFAbBv4knKVVuPSu/kMDHVIkwdktEEmPhkMHunIxrI7Yobe4r0GMXXZJm
 Vu4dNtskzERu/fc+lgbdMjRU+xlXT/zYHy98L5mu+VRgKgmtY/6aF1et4t+EfM9f
 tu3vd/WEOn/HiYvOsVj55SNSR6IiXTGq0LS5d4fI2Q9B477qoO+x3QJDzjbHBo+u
 u58QkbqJa5OfO0ZoUAi+SvOjE0EhJFPL5ihVODjkTSbORpfWz3ULyAK1nQ==
 =Mdqh
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/bonzini-gitlab/tags/for-upstream' into staging

Remove obsolete setuptools dependency and fix Stefan's
Win32 builds.

# gpg: Signature made Thu 26 Nov 2020 14:21:28 GMT
# gpg:                using RSA key F13338574B662389866C7682BFFBD25F78C7AE83
# gpg:                issuer "pbonzini@redhat.com"
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>" [full]
# gpg:                 aka "Paolo Bonzini <pbonzini@redhat.com>" [full]
# Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4  E2F7 7E15 100C CD36 69B1
#      Subkey fingerprint: F133 3857 4B66 2389 866C  7682 BFFB D25F 78C7 AE83

* remotes/bonzini-gitlab/tags/for-upstream:
  nsis: Fix build for 64 bit installer
  tests/docker, tests/vm: remove setuptools from images
  configure: remove python pkg_resources check
  meson: use dependency() to find libjpeg

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-11-26 14:25:47 +00:00
Paolo Bonzini
92ea027017 tests/docker, tests/vm: remove setuptools from images
Setuptools is not needed anymore by the bundled copy of meson,
remove it.

Suggested-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-11-25 12:50:48 -05:00
Philippe Mathieu-Daudé
8e9419b790 tests/docker: Install liblttng-ust-dev package in Ubuntu 20.04 image
Install the liblttng-ust-dev package to be able to
build QEMU using the User-Space Tracer trace backend
(configure --enable-trace-backends=ust).

Suggested-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20201111121234.3246812-2-philmd@redhat.com>
Message-Id: <20201117173635.29101-6-alex.bennee@linaro.org>
2020-11-23 09:55:20 +00:00
Thomas Huth
5ed7ca3185 tests/docker/dockerfiles/centos: Use SDL2 instead of SDL1
We do not support SDL1 in QEMU anymore. Use SDL2 instead.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20201021072308.9224-1-thuth@redhat.com>
Message-Id: <20201021163136.27324-4-alex.bennee@linaro.org>
2020-10-27 09:53:47 +00:00
Thomas Huth
c4cb1c9f2e test/docker/dockerfiles: Add missing packages for acceptance tests
Some of the "check-acceptance" tests are still skipped in the CI
since the docker images do not provide the necessary packages, e.g.
the netcat binary. Add them to get more test coverage.

Message-Id: <20201023073351.251332-5-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-10-26 09:54:16 +01:00
Paolo Bonzini
345d7053ca add ninja to dockerfiles, CI configurations and test VMs
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Acked-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-10-17 10:45:50 -04:00
Paolo Bonzini
f2f984a3b3 dockerfiles: enable Centos 8 PowerTools
ninja is included in the CentOS PowerTools repository.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-10-17 10:45:50 -04:00
Thomas Huth
7e86e5d5cc tests/docker: Add genisoimage to the docker file
genisoimage is needed for running the tests/qtest/cdrom-test qtest.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20201006174347.152040-1-thuth@redhat.com>
Message-Id: <20201007160038.26953-7-alex.bennee@linaro.org>
2020-10-09 17:27:55 +01:00
Paolo Bonzini
d72d6dcb0d dockerfiles: add diffutils to Fedora
For some reason diffutils is not included in the Fedora containers anymore,
causing the build to fail.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Neal Gompa <ngompa13@gmail.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20201003085054.332992-2-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-10-05 09:14:19 +02:00
Thomas Huth
e3755276d1 tests/docker: Remove old Debian 9 containers
We do not support Debian 9 in QEMU anymore, and the Debian 9 containers
are now no longer used in the gitlab-CI. Time to remove them.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20200921174320.46062-6-thuth@redhat.com>
Message-Id: <20200925154027.12672-14-alex.bennee@linaro.org>
2020-10-02 12:29:15 +01:00
Thomas Huth
cf63177e59 tests/docker: Update the tricore container to debian 10
We do not support Debian 9 anymore, thus update the Tricore container
to Debian 10 now.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20200921174320.46062-4-thuth@redhat.com>
Message-Id: <20200925154027.12672-12-alex.bennee@linaro.org>
2020-10-02 12:29:08 +01:00
Thomas Huth
93cc0506f6 tests/docker: Use Fedora containers for MinGW cross-builds in the gitlab-CI
According to our support policy, we do not support Debian 9 in QEMU
anymore, and we only support building the Windows binaries with a
very recent version of the MinGW toolchain. So we should not test
the MinGW cross-compilation with Debian 9 anymore, but switch to
something newer like Fedora. To do this, we need a separate Fedora
container for each build that provides the QEMU_CONFIGURE_OPTS
environment variable.
Unfortunately, the MinGW 64-bit compiler seems to be a little bit
slow, so we also have to disable some features like "capstone" in the
build here to make sure that the CI pipelines still finish within a
reasonable amount of time.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200921174320.46062-2-thuth@redhat.com>
Message-Id: <20200925154027.12672-10-alex.bennee@linaro.org>
2020-10-02 12:29:02 +01:00
Peter Maydell
7cd77fb02b Remove texinfo dependency from docker and CI configs
We don't need texinfo to build the docs any more, so we can
drop that dependency from our docker and other CI configs.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Message-Id: <20200925162316.21205-22-peter.maydell@linaro.org>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2020-09-29 17:55:39 +02:00
Paolo Bonzini
17cd6e2bbf docker.py: always use --rm
Avoid that containers pile up.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Acked-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-09-18 10:44:47 +01:00
Paolo Bonzini
660f793093 Makefile: inline the relevant parts of rules.mak
Most of rules.mak is not used anymore, just inline what's needed.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-09-08 11:43:16 +02:00
Daniel P. Berrangé
affcc09c2d gitlab: expand test coverage for crypto builds
Most jobs test the latest nettle library. This adds explicit coverage
for latest gcrypt using Fedora, and old gcrypt and nettle using
CentOS-7. The latter does a minimal tools-only build, as we only need to
validate that the crypto code builds and unit tests pass. Finally a job
disabling both nettle and gcrypt is provided to validate that gnutls
still works.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20200901133050.381844-3-berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-09-07 12:34:17 +02:00
Thomas Huth
d3dd34a1e5 dockerfiles/debian-win64-cross: Download WHPX MinGW headers
To compile-test the WHPX accelerator, we need to download these system
headers first (they are unfortunately not part of any released and
packaged MinGW toolchain yet).

Idea taken from another patch by Stefan Weil.

Message-Id: <20200804170055.2851-12-thuth@redhat.com>
Message-Id: <20200823111757.72002-6-thuth@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-09-07 12:34:17 +02:00
Alex Bennée
90843750e3 tests/docker: add python3-setuptools the docker images
We need these now for builds to work.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200903112107.27367-4-alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-09-06 20:26:43 +02:00
Paolo Bonzini
b861f59593 tests/docker: add test script for static linux-user builds
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-08-21 06:18:35 -04:00
Thomas Huth
1caac1c0e4 tests/docker: Add python3-venv and netcat to the debian-amd64 container
Without python3-venv, I get the following message when trying to
run the acceptance tests within the debian container:

 The virtual environment was not created successfully because ensurepip is not
 available.  On Debian/Ubuntu systems, you need to install the python3-venv
 package using the following command.
    apt-get install python3-venv
 You may need to use sudo with that command.  After installing the python3-venv
 package, recreate your virtual environment.

Let's do it as the message suggests.

And while we're at it, also add netcat here since it is required for
some of the acceptance tests.

Message-Id: <20200730141326.8260-2-thuth@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-08-05 10:37:42 +02:00
Alex Bennée
4c5aeb1238 tests/docker: add support for DEB_KEYRING
For installing stuff from sid or ports you may need to manually
specify the location of the keyring. You can even import keys into
your personal keyring and point it there, e.g.:

  gpg --keyserver keyring.debian.org --recv-keys 84C573CD4E1AFD6C
  make docker-binfmt-image-debian-sid-hppa DEB_TYPE=sid DEB_ARCH=hppa \
      DEB_URL=http://ftp.ports.debian.org/debian-ports/ \
      EXECUTABLE=./hppa-linux-user/qemu-hppa V=1 \
      DEB_KEYRING=${HOME}/.gnupg/pubring.kbx

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20200724064509.331-12-alex.bennee@linaro.org>
2020-07-27 09:41:35 +01:00
Alex Bennée
182ec06076 tests/docker: fix binfmt_misc image building
When we updated the arguments for docker.py we missed a bit.

Fixes: dfae628459 ("docker.py/build: support -t and -f arguments")
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20200724064509.331-11-alex.bennee@linaro.org>
2020-07-27 09:41:22 +01:00
Alex Bennée
e336cec3a5 tests/docker: fix update command due to python3 str/bytes distinction
Does this seem convoluted to you? It feels a little complicated to me.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200724064509.331-10-alex.bennee@linaro.org>
2020-07-27 09:41:18 +01:00
Alex Bennée
f73e485285 docker.py: fix fetching of FROM layers
This worked on a system that was already bootstrapped because the
stage 2 images already existed even if they wouldn't be used. What we
should have pulled down was the FROM line containers first because
building on gitlab doesn't have the advantage of using our build
system to build the pre-requisite bits.

We still pull the image we want to build just in case we can use the
cached data.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20200713200415.26214-4-alex.bennee@linaro.org>
2020-07-15 11:52:43 +01:00
Thomas Huth
a77312e77c tests/docker: Remove the libssh workaround from the ubuntu 20.04 image
The libssh problem only exists in Ubuntu 18.04 - we can enable it
in 20.04 again.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20200713185237.9419-1-thuth@redhat.com>
Message-Id: <20200713200415.26214-3-alex.bennee@linaro.org>
2020-07-15 11:52:43 +01:00
Max Filippov
bf7801df66 tests/docker: update toolchain set in debian-xtensa-cross
Switch to the prebuilt xtensa toolchains release 2020.07.
Drop csp toolchain as the csp core is not a part of QEMU.
Add de233_fpu and dsp3400 toolchains to enable DFPU and FPU2000 tests.

Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
[AJB: fix path in configure.sh]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200708082347.27318-1-jcmvbkbc@gmail.com>
Message-Id: <20200709141327.14631-13-alex.bennee@linaro.org>
2020-07-11 15:53:09 +01:00
Alex Bennée
85c5e69923 tests/docker: fall back more gracefully when pull fails
I only spotted this in the small window between my testing with my
registry while waiting for the gitlab PR to go in. As we pre-pull the
registry image we know if that fails there isn't any point attempting
to use the cache. Fall back to the way we used to do it at that point.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20200709141327.14631-12-alex.bennee@linaro.org>
2020-07-11 15:53:00 +01:00
Alex Bennée
27ebeda0c0 tests/docker: add a linux-user testing focused image
We happily use all the cross images for both cross-building QEMU as
well as building the linux-user tests. However calling docker from
within docker seems not to work. As we can build in Debian anyway why
not include an image that has all the compilers available for
non-docker invocation.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200701135652.1366-33-alex.bennee@linaro.org>
2020-07-11 15:53:00 +01:00
Alex Bennée
cdc61815c7 tests/docker: add packages needed for check-acceptance
We need additional python packages to run check-acceptance. Add them
to the docker images we will be using later.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20200701135652.1366-26-alex.bennee@linaro.org>
2020-07-11 15:53:00 +01:00
Alex Bennée
e6f1306b10 tests/docker: add --registry support to tooling
This allows us to point the tools towards a registry from which they
can grab pre-built layers instead of doing everything from scratch
each time. To enable this we need to be using the DOCKER_BUILDKIT
engine.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200701135652.1366-25-alex.bennee@linaro.org>
2020-07-11 15:53:00 +01:00
Alex Bennée
767b6bd22b tests/docker: change tag naming scheme of our images
We've been misusing the tag naming scheme for some time by overloading
the post : section with the image type. Really it should be saved for
the revision of that particular build. Move the details to the other
side so we have:

  qemu/image-name

with the implied :latest version added by the tooling.

Suggested-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>

Message-Id: <20200701135652.1366-18-alex.bennee@linaro.org>
2020-07-11 15:53:00 +01:00
Alex Bennée
10c927dcc5 tests/docker: check for an parameters not empty string
Reported-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Suggested-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>

Message-Id: <20200701135652.1366-17-alex.bennee@linaro.org>
2020-07-11 15:53:00 +01:00
Robert Foley
df79fd5667 tests/docker: Added docker build support for TSan.
Added a new docker for ubuntu 20.04.
This docker has support for Thread Sanitizer
including one patch we need in one of the header files.
https://github.com/llvm/llvm-project/commit/a72dc86cd

This command will build with tsan enabled:
make docker-test-tsan-ubuntu2004 V=1

Also added the TSAN suppresion file to disable certain
cases of TSAN warnings.

Cc: Fam Zheng <fam@euphon.net>
Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Robert Foley <robert.foley@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200609200738.445-10-robert.foley@linaro.org>
Message-Id: <20200612190237.30436-13-alex.bennee@linaro.org>
2020-06-16 14:49:05 +01:00
Alex Bennée
7ec1edde9e tests/docker: bump fedora to 32
We should be keeping this up to date as Fedora goes out of support
quite quickly.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20200612190237.30436-2-alex.bennee@linaro.org>
2020-06-16 14:49:02 +01:00
Peter Maydell
7d3660e798 * Miscellaneous fixes and feature enablement (many)
* SEV refactoring (David)
 * Hyper-V initial support (Jon)
 * i386 TCG fixes (x87 and SSE, Joseph)
 * vmport cleanup and improvements (Philippe, Liran)
 * Use-after-free with vCPU hot-unplug (Nengyuan)
 * run-coverity-scan improvements (myself)
 * Record/replay fixes (Pavel)
 * -machine kernel_irqchip=split improvements for INTx (Peter)
 * Code cleanups (Philippe)
 * Crash and security fixes (PJP)
 * HVF cleanups (Roman)
 -----BEGIN PGP SIGNATURE-----
 
 iQFIBAABCAAyFiEE8TM4V0tmI4mGbHaCv/vSX3jHroMFAl7jpdAUHHBib256aW5p
 QHJlZGhhdC5jb20ACgkQv/vSX3jHroMfjwf/X7+0euuE9dwKFKDDMmIi+4lRWnq7
 gSOyE1BYSfDIUXRIukf64konXe0VpiotNYlyEaYnnQjkMdGm5E9iXKF+LgEwXj/t
 NSGkfj5J3VeWRG4JJp642CSN/aZWO8uzkenld3myCnu6TicuN351tDJchiFwAk9f
 wsXtgLKd67zE8MLVt8AP0rNTbzMHttPXnPaOXDCuwjMHNvMEKnC93UeOeM0M4H5s
 3Dl2HvsNWZ2SzUG9mAbWp0bWWuoIb+Ep9//87HWANvb7Z8jratRws18i6tYt1sPx
 8zOnUS87sVnh1CQlXBDd9fEcqBUVgR9pAlqaaYavNhFp5eC31euvpDU8Iw==
 =F4sU
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into staging

* Miscellaneous fixes and feature enablement (many)
* SEV refactoring (David)
* Hyper-V initial support (Jon)
* i386 TCG fixes (x87 and SSE, Joseph)
* vmport cleanup and improvements (Philippe, Liran)
* Use-after-free with vCPU hot-unplug (Nengyuan)
* run-coverity-scan improvements (myself)
* Record/replay fixes (Pavel)
* -machine kernel_irqchip=split improvements for INTx (Peter)
* Code cleanups (Philippe)
* Crash and security fixes (PJP)
* HVF cleanups (Roman)

# gpg: Signature made Fri 12 Jun 2020 16:57:04 BST
# gpg:                using RSA key F13338574B662389866C7682BFFBD25F78C7AE83
# gpg:                issuer "pbonzini@redhat.com"
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>" [full]
# gpg:                 aka "Paolo Bonzini <pbonzini@redhat.com>" [full]
# Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4  E2F7 7E15 100C CD36 69B1
#      Subkey fingerprint: F133 3857 4B66 2389 866C  7682 BFFB D25F 78C7 AE83

* remotes/bonzini/tags/for-upstream: (116 commits)
  target/i386: Remove obsolete TODO file
  stubs: move Xen stubs to accel/
  replay: fix replay shutdown for console mode
  exec/cpu-common: Move MUSB specific typedefs to 'hw/usb/hcd-musb.h'
  hw/usb: Move device-specific declarations to new 'hcd-musb.h' header
  exec/memory: Remove unused MemoryRegionMmio type
  checkpatch: reversed logic with acpi test checks
  target/i386: sev: Unify SEVState and SevGuestState
  target/i386: sev: Remove redundant handle field
  target/i386: sev: Remove redundant policy field
  target/i386: sev: Remove redundant cbitpos and reduced_phys_bits fields
  target/i386: sev: Partial cleanup to sev_state global
  target/i386: sev: Embed SEVState in SevGuestState
  target/i386: sev: Rename QSevGuestInfo
  target/i386: sev: Move local structure definitions into .c file
  target/i386: sev: Remove unused QSevGuestInfoClass
  xen: fix build without pci passthrough
  i386: hvf: Drop HVFX86EmulatorState
  i386: hvf: Move mmio_buf into CPUX86State
  i386: hvf: Move lazy_flags into CPUX86State
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

# Conflicts:
#	hw/i386/acpi-build.c
2020-06-12 23:06:22 +01:00
Paolo Bonzini
af509738f8 docker.py/build: support binary files in --extra-files
Read the --extra-files in binary mode to avoid encoding errors.

Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-06-10 12:09:31 -04:00
Paolo Bonzini
dfae628459 docker.py/build: support -t and -f arguments
The docker.py command line is subtly different from docker and podman's,
in that the tag and Dockerfile are passed via positional arguments.
Remove this gratuitous difference and just parse -f and -t.

-f was previously used by --extra-files, only keep the long option.

Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-06-10 12:09:31 -04:00
Philippe Mathieu-Daudé
c6dd82e8a6 tests/docker: Remove flex/bison packages
QEMU does not use flex/bison packages.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Claudio Fontana <cfontana@suse.de>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200515163029.12917-2-philmd@redhat.com>
2020-06-08 17:04:19 +01:00
Paolo Bonzini
a97098844b docker: update Ubuntu to 20.04
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200604231716.11354-1-pbonzini@redhat.com>
Message-Id: <20200605154929.26910-10-alex.bennee@linaro.org>
2020-06-08 17:04:19 +01:00
Alex Bennée
e035e6ffed tests/docker: fix pre-requisite for debian-tricore-cross
Reported-by: Bastian Koppelmann <kbastian@mail.uni-paderborn.de>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200605154929.26910-9-alex.bennee@linaro.org>
2020-06-08 17:04:19 +01:00
John Snow
2d110c1149 python: remove more instances of sys.version_info
We guarantee 3.5+ everywhere; remove more dead checks. In general, try
to avoid using version checks and instead prefer to attempt behavior
when possible.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200514035230.25756-1-jsnow@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-05-31 18:25:07 +02:00
Alex Bennée
c729a99d27 tests/docker: use a gcc-10 based image for arm64 tests
As we enable newer features that we want to test on arm64 targets we
need newer compilers. Split off a new debian-arm64-test-cross image
which we can use to build these new tests.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20200520140541.30256-11-alex.bennee@linaro.org>
2020-05-27 14:26:49 +01:00
Alex Bennée
086f269cf4 tests/docker: add debian11 base image
We won't use this for building QEMU but we do need newer GCC's and
binutils for building some of our test cases.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20200520140541.30256-10-alex.bennee@linaro.org>
2020-05-27 14:26:49 +01:00
Alex Bennée
551affb61e tests/docker: add docs FEATURE flag and use for test-misc
The test-misc docker test fails on a number of images which don't have
the prerequisites to build the docs. Use the FEATURES flag so we can
skip those tests.

As the sphinx test fails to detect whatever feature we need to get
hxtool to work we drop them from debian9 so the windows build doesn't
attempt to build the docs.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20200414200631.12799-7-alex.bennee@linaro.org>
2020-04-15 11:38:23 +01:00
Philippe Mathieu-Daudé
3e3207337b tests/docker: Add libepoxy and libudev packages to the Fedora image
Install optional dependencies of QEMU to get better coverage.

Suggested-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20200322120104.21267-5-philmd@redhat.com>
Message-Id: <20200323161514.23952-11-alex.bennee@linaro.org>
2020-03-27 13:43:20 +00:00
Philippe Mathieu-Daudé
6d8e7738b0 tests/docker: Use Python3 PyYAML in the Fedora image
The Python2 PyYAML is now pointless, switch to the Python3 version.

Fixes: bcbf27947 (docker: move tests from python2 to python3)
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20200322120104.21267-4-philmd@redhat.com>
Message-Id: <20200323161514.23952-10-alex.bennee@linaro.org>
2020-03-27 13:43:20 +00:00
Philippe Mathieu-Daudé
9274ae32c3 tests/docker: Install gcrypt devel package in Debian image
In commit 6f8bbb374b we enabled building with the gcrypt library
on the the Debian 'x86 host', which was based on Debian Stretch.
Later in commit 698a71edbe we upgraded the Debian base image to
Buster.

Apparently Debian Stretch was listing gcrypt as a QEMU dependency,
but this is not the case anymore in Buster, so we need to install
it manually (it it not listed by 'apt-get -s build-dep qemu' in
the common debian10.docker anymore). This fixes:

 $ ../configure $QEMU_CONFIGURE_OPTS

  ERROR: User requested feature gcrypt
         configure was not able to find it.
         Install gcrypt devel >= 1.5.0

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20200322120104.21267-3-philmd@redhat.com>
Message-Id: <20200323161514.23952-9-alex.bennee@linaro.org>
2020-03-27 13:43:20 +00:00
Philippe Mathieu-Daudé
cdb1a84cfe tests/docker: Keep package list sorted
Keep package list sorted, this eases rebase/cherry-pick.

Fixes: 3a6784813
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20200322120104.21267-2-philmd@redhat.com>
Message-Id: <20200323161514.23952-8-alex.bennee@linaro.org>
2020-03-27 13:43:20 +00:00
Peter Maydell
a1ba62a0f3 Python and tests (mostly acceptance) patches 2020-03-17
-----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEEeruW64tGuU1eD+m7ZX6NM6XyCfMFAl5xaEMACgkQZX6NM6Xy
 CfPUDQ/+IACVcfF6LdPB/vwhBaKx8/fqpJEnyL8AFCkYrIVWw0c9zizjTobWbl8H
 qxDfqvM+O4ENy12ClG0LkyRN9sVs6IGlpE4f50kKqttd5GA8bBSvamv5iCxZ1fn8
 IsR/uH5sQt8aqENxDiIszZH+UEATsFL916KyMre8faqEN8xzFvnFeQ+pT0s5sF9v
 XpXdS1D9JtI00pFmNz/4jkyzB4uG6D20NjoHOSS8IWMx9jOAQn5++VC6183lxdpK
 ty7adf0y0brC9BCEua4FdCHliDYgYbO5hdsk6oWWUQRvnYEFajXgV/nEdqV+rCG8
 9KEKC7sdbOwkbXxwtfjMQ7QK51Xhz7P0d2j0+vn96V0MIve4GM0Z9vx87Psy+4T3
 KJqM7GISDiFtMs2Zi2D+o4nyL4m4fS1AzbAeEWH1dwl7RaAb1zKr7F2hFSHIkNHh
 Z1cnoFhm0kXQFhFqMwjY5ackV+pYm8qSXroG3SvghPByb78LwBx5rzE37clW5HDa
 VO/pRnY0FIyYldOlu+XHGCwCgiaVQ0mUb9DOwpPL+WukZUY1y6bxWlamMvIBM0ko
 l1rYlnz0mEwpLptctlob0Vo/Js/jbyxT0XRcTbJq3FexFeEykFtFwW8hfT63xJXo
 En9cVkyEWzgT9Ici3HNOwLz5iJCfkWjx/ecRtA6TBeod0V/evo0=
 =DHn4
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/cleber/tags/python-next-pull-request' into staging

Python and tests (mostly acceptance) patches 2020-03-17

# gpg: Signature made Wed 18 Mar 2020 00:16:03 GMT
# gpg:                using RSA key 7ABB96EB8B46B94D5E0FE9BB657E8D33A5F209F3
# gpg: Good signature from "Cleber Rosa <crosa@redhat.com>" [marginal]
# gpg: WARNING: This key is not certified with sufficiently trusted signatures!
# gpg:          It is not certain that the signature belongs to the owner.
# Primary key fingerprint: 7ABB 96EB 8B46 B94D 5E0F  E9BB 657E 8D33 A5F2 09F3

* remotes/cleber/tags/python-next-pull-request:
  tests/docker: make "buildah bud" output similar to "docker build"
  tests/docker: add CentOS 8 Dockerfile
  Acceptance tests: add make targets to download images
  Acceptance test: add "boot_linux" tests
  Acceptance tests: introduce BUILD_DIR and SOURCE_DIR
  python/qemu/qmp.py: QMP debug with VM label

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-03-19 12:33:50 +00:00
Peter Maydell
9214813489 Testing and gdbstub updates:
- docker updates for VirGL
   - re-factor gdbstub for static GDBState
   - re-factor gdbstub for dynamic arrays
   - add SVE support to arm gdbstub
   - add some guest debug tests to check-tcg
   - add aarch64 userspace register tests
   - remove packet size limit to gdbstub
   - simplify gdbstub monitor code
   - report vContSupported in gdbstub to use proper single-step
 -----BEGIN PGP SIGNATURE-----
 
 iQEzBAABCgAdFiEEZoWumedRZ7yvyN81+9DbCVqeKkQFAl5xDUIACgkQ+9DbCVqe
 KkQwCwf/YtmUsNxxO+CgNctq2u3jV4FoOdQP3bejvmT2+cigKJhQuBlWPg1/YsqF
 RDNkmBQx2JaVVMuVmpnwVK1UD+kmYZqrtlOkPNcVrjPmLCq3BVI1LHe6Rjoerx8F
 QoZyH0IMNHbBgDo1I46lSFOWcxmOvo+Ow7NX5bPKwlRzf0dyEqSJahRaZLAgUscR
 taTtGfk9uQsnxoRsvH/efiQ4bZtUvrEQuhEX3WW/yVE1jTpcb2llwX4xONJb2It3
 /0WREGEEIT8PpnWw2S3FH4THY/BjWgz/FPDwNNZYCKBMWDjuG/8KHryd738T9rzo
 lkGP9YcXmiyxMMyFFwS8RD3SHr8LvQ==
 =Wm+a
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/stsquad/tags/pull-testing-and-gdbstub-170320-1' into staging

Testing and gdbstub updates:

  - docker updates for VirGL
  - re-factor gdbstub for static GDBState
  - re-factor gdbstub for dynamic arrays
  - add SVE support to arm gdbstub
  - add some guest debug tests to check-tcg
  - add aarch64 userspace register tests
  - remove packet size limit to gdbstub
  - simplify gdbstub monitor code
  - report vContSupported in gdbstub to use proper single-step

# gpg: Signature made Tue 17 Mar 2020 17:47:46 GMT
# gpg:                using RSA key 6685AE99E75167BCAFC8DF35FBD0DB095A9E2A44
# gpg: Good signature from "Alex Bennée (Master Work Key) <alex.bennee@linaro.org>" [full]
# Primary key fingerprint: 6685 AE99 E751 67BC AFC8  DF35 FBD0 DB09 5A9E 2A44

* remotes/stsquad/tags/pull-testing-and-gdbstub-170320-1: (28 commits)
  gdbstub: Fix single-step issue by confirming 'vContSupported+' feature to gdb
  gdbstub: do not split gdb_monitor_write payload
  gdbstub: change GDBState.last_packet to GByteArray
  tests/tcg/aarch64: add test-sve-ioctl guest-debug test
  tests/tcg/aarch64: add SVE iotcl test
  tests/tcg/aarch64: add a gdbstub testcase for SVE registers
  tests/guest-debug: add a simple test runner
  configure: allow user to specify what gdb to use
  tests/tcg/aarch64: userspace system register test
  target/arm: don't bother with id_aa64pfr0_read for USER_ONLY
  target/arm: generate xml description of our SVE registers
  target/arm: default SVE length to 64 bytes for linux-user
  target/arm: explicitly encode regnum in our XML
  target/arm: prepare for multiple dynamic XMLs
  gdbstub: extend GByteArray to read register helpers
  target/i386: use gdb_get_reg helpers
  target/m68k: use gdb_get_reg helpers
  target/arm: use gdb_get_reg helpers
  gdbstub: add helper for 128 bit registers
  gdbstub: move mem_buf to GDBState and use GByteArray
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-03-18 20:25:23 +00:00
Cleber Rosa
a51d6a5493 tests/docker: make "buildah bud" output similar to "docker build"
Podman users will most often be using buildah to build containers.
Among the differences between "buildah bud|build-using-dockerfile" and
a traditional "docker build" is that buildah does not run a container
during build.

To the best of my knowledge and experiments, this means that runtime
variables, such as ENV from one base image will not propagate into
another.  The end result is that the location for the cross compiler
binaries, defined in the base "qemu/debian9-mxe" image, are not passed
through this image.  Consequently, the cross compilers are not on PATH
and the build fails.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Acked-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200312193616.438922-3-crosa@redhat.com>
Signed-off-by: Cleber Rosa <crosa@redhat.com>
2020-03-17 19:16:16 -04:00
Cleber Rosa
e631eb2e8b tests/docker: add CentOS 8 Dockerfile
Which is currenly missing, and will be referenced later in the
contributed CI playbooks.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200312193616.438922-2-crosa@redhat.com>
Signed-off-by: Cleber Rosa <crosa@redhat.com>
2020-03-17 19:14:49 -04:00
Philippe Mathieu-Daudé
d2763944e2 tests/docker: Update VirGL to v0.8.0
Building the qemu:debian-amd64 fails when building VirGL:

  make[2]: Entering directory '/usr/src/virglrenderer/src/gallium/auxiliary'
    CC       cso_cache/cso_cache.lo
    CC       cso_cache/cso_hash.lo
    CC       os/os_misc.lo
    CC       util/u_debug.lo
    CC       util/u_debug_describe.lo
    CC       util/u_format.lo
    GEN      util/u_format_table.c
  Traceback (most recent call last):
    File "./util/u_format_table.py", line 168, in <module>
      main()
    File "./util/u_format_table.py", line 164, in main
      write_format_table(formats)
    File "./util/u_format_table.py", line 132, in write_format_table
      print("   %s,\t/* is_array */" % (bool_map(format.is_array()),))
    File "/usr/src/virglrenderer/src/gallium/auxiliary/util/u_format_parse.py", line 164, in is_array
      return self.array_element() != None
    File "/usr/src/virglrenderer/src/gallium/auxiliary/util/u_format_parse.py", line 73, in __eq__
      return self.type == other.type and self.norm == other.norm and self.pure == other.pure and self.size == other.size
  AttributeError: 'NoneType' object has no attribute 'type'
  make[2]: Leaving directory '/usr/src/virglrenderer/src/gallium/auxiliary'
  make[2]: *** [Makefile:906: util/u_format_table.c] Error 1
  make[1]: *** [Makefile:631: install-recursive] Error 1

VirGL commits a8962eda1..a613dcc82 fix this problem.
Update to VirGL 0.8.0 which contains them.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200212202709.12665-4-philmd@redhat.com>
Message-Id: <20200316172155.971-5-alex.bennee@linaro.org>
2020-03-17 17:38:33 +00:00
Philippe Mathieu-Daudé
5b9d40fafe tests/docker: Remove obsolete VirGL --with-glx configure option
The GLX configure option has been removed in 71c75f201d [*].
We missed that when updating to v0.7.0 in commit fab3220f97.

This silents:

  configure: creating ./config.status
  config.status: creating virglrenderer.pc
  ...
  configure: WARNING: unrecognized options: --with-glx

[*] https://gitlab.freedesktop.org/virgl/virglrenderer/commit/71c75f201d

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200212202709.12665-3-philmd@redhat.com>
Message-Id: <20200316172155.971-4-alex.bennee@linaro.org>
2020-03-17 17:38:31 +00:00
Philippe Mathieu-Daudé
72e3c1dd57 tests/docker: Update VirGL git repository URL
freedesktop.org is moving to a GitLab instance,
use the new url.

- https://www.fooishbar.org/blog/gitlab-fdo-introduction/
- https://gitlab.freedesktop.org/freedesktop/freedesktop/-/wikis/home

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200212202709.12665-2-philmd@redhat.com>
Message-Id: <20200316172155.971-3-alex.bennee@linaro.org>
2020-03-17 17:38:30 +00:00
Philippe Mathieu-Daudé
1bbf2d010f tests/docker: Install tools to cross-debug and build Linux kernels
We often run Linux kernels to test QEMU. We sometimes need
to build them manually to use non-default features. We only
miss the tiny 'bc' tool.

The ncurses library is helpful to run 'make menuconfig'.

Finally, gdb-multiarch allow us to debug a TCG guest when its
architecture is different than the host.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200212202738.12986-1-philmd@redhat.com>
Message-Id: <20200316172155.971-2-alex.bennee@linaro.org>
2020-03-17 17:38:27 +00:00
Philippe Mathieu-Daudé
a4aad716cb tests/docker: Install SASL library to extend code coverage on amd64
Install the SASL library to build the VNC SASL auth protocol code.

Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200309122454.22551-3-philmd@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-03-16 23:02:23 +01:00
Juan Quintela
3a67848134 configure: Enable test and libs for zstd
Add it to several build systems to make testing good.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2020-02-28 09:24:43 +01:00
Paolo Bonzini
423edd9a31 drop "from __future__ import print_function"
This is only needed for Python 2, which we do not support anymore.

Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Acked-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20200204160604.19883-1-pbonzini@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:15:16 +01:00
Philippe Mathieu-Daudé
c88ee46cdb tests: Explicit usage of Python 3
Use the program search path to find the Python 3 interpreter.

Patch created mechanically by running:

  $ sed -i "s,^#\!/usr/bin/\(env\ \)\?python$,#\!/usr/bin/env python3," \
       $(git grep -l 'if __name__.*__main__')

Reported-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Suggested-by: Daniel P. Berrangé <berrange@redhat.com>
Suggested-by: Stefan Hajnoczi <stefanha@redhat.com>
Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20200130163232.10446-5-philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:12:48 +01:00
Alex Bennée
5e33f7fead tests/docker: better handle symlinked libs
When we are copying we want to ensure we grab the first
resolution (the found in path section). However even that binary might
be a symlink so lets make sure we chase the symlinks to copy the right
binary to where it can be found.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Robert Foley <robert.foley@linaro.org>

Message-Id: <20200203090932.19147-3-alex.bennee@linaro.org>
2020-02-04 17:13:28 +00:00
Alex Bennée
698a71edbe tests/docker: move most cross compilers to buster base
This includes fixing up the dependencies (Which were already wrong for
one of the mips variants).

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200203090932.19147-2-alex.bennee@linaro.org>
2020-02-04 17:13:28 +00:00
Marc-André Lureau
1409c047c1 dockerfiles: add dbus-daemon to some of latest distributions
To get dbus-vmstate test covered.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2020-01-06 18:41:32 +04:00
Paolo Bonzini
7ff5c1fa27 docker: gtester is no longer used
We are using tap-driver.pl, do not require anymore gtester to be installed
to run the testsuite in docker-based tests.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <1576632611-55032-1-git-send-email-pbonzini@redhat.com>
2019-12-19 08:20:08 +00:00
Thomas Huth
2831a7e580 Remove libbluetooth / bluez from the CI tests
Since the bluetooth code has been removed, we don't need to test
with this library anymore.

Message-Id: <20191120091014.16883-5-thuth@redhat.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2019-12-17 09:05:23 +01:00
Paolo Bonzini
ad430dd9da docker: remove libcap development packages
Libcap was dropped from virtio-9p (commit 7e46261368), so remove it from
the dockerfiles as well.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Acked-by: Alex Bennée <alex.bennee@linaro.org>
[groug, mention SHA1 that dropped libcap]
Signed-off-by: Greg Kurz <groug@kaod.org>
2019-12-09 17:59:33 +01:00
Alex Bennée
8ce2f68fc9 tests/docker: update Travis image to a more current version
This isn't the latest one available on hub.docker.com but it does
match the ID reported by the Xenial builds running on Travis:

  instance: ... travis-ci-sardonyx-xenial-1553530528-f909ac5

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-10-25 19:24:21 +01:00
Alex Bennée
06746324ad tests/docker: set HOST_ARCH if we don't have ARCH
As the docker rules want to be able to be run on a virgin unconfigured
checkout add a fallback and use it if we need to.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-10-25 19:24:21 +01:00
Paolo Bonzini
9e5ef16491 tests/docker: only enable ubsan for test-clang
-fsanitize=undefined is not the same thing as --enable-sanitizers.  After
commit 47c823e ("tests/docker: add sanitizers back to clang build", 2019-09-11)
test-clang is almost duplicating the asan (test-debug) test, so
partly revert commit 47c823e5b while leaving ubsan enabled.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2019-10-04 18:49:19 +02:00
Paolo Bonzini
6f89ec7442 docker: test-debug: disable LeakSanitizer
There are just too many leaks in device-introspect-test (especially for
the plethora of arm and aarch64 boards) to make LeakSanitizer useful;
disable it for now.

Whoever is interested in debugging leaks can also use valgrind like this:

   QTEST_QEMU_BINARY=aarch64-softmmu/qemu-system-aarch64 \
   QTEST_QEMU_IMG=qemu-img \
   valgrind --trace-children=yes --leak-check=full \
   tests/device-introspect-test -p /aarch64/device/introspect/concrete/defaults/none

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2019-10-04 18:49:17 +02:00
Alex Bennée
80394ccf21 tests/docker: remove debian-powerpc-user-cross
Despite our attempts in 4d26c7fef4 to keep this going it still gets in
the way of "make docker-test-build" completing because of course we
can't build a modern QEMU with the image. Let's put the thing out of
its misery and remove it.

People who really care about building on powerpc can still use the
binfmt_misc support to manually build an image (or just run the build
from pre this commit).

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Cc: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
2019-09-26 19:00:53 +01:00
John Snow
bcbf279473 docker: move tests from python2 to python3
As part of the push to drop python2 support, replace any explicit python2
dependencies with python3 versions.

For centos, python2 still exists as an implicit dependency, but by adding
python3 we will be able to build even if the configure script begins to
require python 3.5+.

Tested with centos7, fedora, ubuntu, ubuntu1804, and debian 9 (amd64).
Tested under a custom configure script that requires Python 3.5+.

the travis dockerfile is also moved to using python3, which was tested
by running `make docker-test-build@travis`, which I hope is sufficient.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20190923181140.7235-7-jsnow@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Tested-by: Cleber Rosa <crosa@redhat.com>
2019-09-26 19:00:53 +01:00
John Snow
2e90aec3d9 docker: remove unused debian-sid
debian-sid is listed as a partial image, so we cannot run tests against it.
Since it isn't used by any other testable image, remove it for now as it
is prone to bitrot.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20190923181140.7235-6-jsnow@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-09-26 19:00:53 +01:00
John Snow
dad47814f0 docker: remove unused debian-ports
debian-ports is listed as a partial image, so we cannot run tests against it.
Since it isn't used by any other testable image, remove it for now as it
is prone to bitrot.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20190923181140.7235-5-jsnow@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-09-26 19:00:53 +01:00
John Snow
ef8f3283ac docker: remove 'deprecated' image definitions
There isn't a debian.dockerfile anymore,
so perform some ghost-busting.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20190923181140.7235-4-jsnow@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Acked-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-09-26 19:00:53 +01:00
John Snow
afa1cf4329 docker: remove unused debian8 partial image
debian8 partial base is also not consumed by any image, so remove it.
For QEMU's development cycle, we only support debian9 (stretch) and
debian10 (buster).

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20190923181140.7235-3-jsnow@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-09-26 19:00:53 +01:00
John Snow
ccf20b3b4f docker: remove debian8-mxe definitions
We don't have a debian8-mxe dockerfile anymore.

Fixes: 67bd36beda
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20190923181140.7235-2-jsnow@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-09-26 19:00:53 +01:00
John Snow
b3a790be98 podman: fix command invocation
Oops; there's no argv here.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20190913193821.17756-1-jsnow@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Tested-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Tested-by: Thomas Huth <thuth@redhat.com>
2019-09-26 19:00:53 +01:00
Alex Bennée
529994e204 tests/docker: reduce scary warnings by cleaning up clean up
There was in the clean-up code caused by attempting to inspect images
which finished before we got there. Clean up the clean up code by:

  - only track the one instance at a time
  - use --filter for docker ps instead of doing it by hand
  - just call docker rm -f to be done with it
  - use uuid.uuid4() for a random uid

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
2019-09-26 19:00:53 +01:00
John Snow
5fac0cfaaa tests/docker: remove python2.7 from debian9-mxe
When it was based on debian8 which uses python-minimal, it needed this.
It no longer does.

Goodbye, python2.7.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20190918222546.11696-1-jsnow@redhat.com>
[AJB: fixed up commit message]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-09-26 19:00:53 +01:00
Alex Bennée
b4d66d3525 tests/docker: fix DOCKER_PARTIAL_IMAGES
Finger trouble in a previous clean-up inadvertently set
DEBIAN_PARTIAL_IMAGES instead of DOCKER_PARTIAL_IMAGES. Also fix the
typo to debian-9-mxe.

Fixes: 44d5a8bf5d
Signed-off-by: John Snow <jsnow@redhat.com>
[AJB: merged fix from Message-Id: <20190917185537.25417-1-jsnow@redhat.com>]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
2019-09-26 19:00:53 +01:00
John Snow
47c823e5be tests/docker: add sanitizers back to clang build
Fedora23 is but a distant twinkle. The sanitizer works again, and even
if not, we have --enable-sanitizers now.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20190912014442.5757-1-jsnow@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-09-26 19:00:53 +01:00
Alex Bennée
50d2dcaddb tests/docker: --disable-libssh on ubuntu1804 builds
Currently this stops the mega:

  make docker-test-build

from working. Once the source is patched to deal with the case this
workaround can be removed.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-09-10 14:14:31 +01:00
Alex Bennée
e5389e4414 tests/docker: add more images to PARTIAL_IMAGES when not on x86_64
This prevents us trying to do builds which we can't complete.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-09-10 14:14:31 +01:00
Alex Bennée
d2467284e8 tests/docker: use --arch-only for installing deps
The Debian QEMU packages require a bunch of cross compilers for
building firmware which aren't available on all host architectures.
Using --arch-only skips this particular requirement and allows us to
install just the dependencies we need.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-09-10 14:14:31 +01:00
Alex Bennée
ec22b72699 tests/docker: add debian-amd64-cross for non-x86 hosts
When building on a non-x86 host we need to setup the x86 build like
any other cross compiler.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-09-10 14:14:31 +01:00
Alex Bennée
bf5b512cd4 tests/docker: avoid $SHELL invoke bash directly
On some images SHELL is pointing at a limited /bin/sh which doesn't
understand noprofile/norc. Given the run script is running bash just
invoke it directly.

This fixes:

  $ make docker-test-build@IMAGE DEBUG=1
  [...]
  + echo '  ./test-build'
  ./test-build
  + echo '* Hit Ctrl-D to continue, or type '\''exit 1'\'' to abort'
  * Hit Ctrl-D to continue, or type 'exit 1' to abort
  + echo
  + /bin/sh --noprofile --norc
  /bin/sh: 0: Illegal option --

Fixes: 2b0c4fa13f
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-09-10 14:14:31 +01:00
Alex Bennée
44d5a8bf5d tests/docker: add debian9-mxe to DEBIAN_PARTIAL_IMAGES
Another image that can't be used directly to build QEMU.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-09-10 14:14:31 +01:00
Alex Bennée
ac276a4b54 tests/docker: add debian-xtensa-cross to DEBIAN_PARTIAL_IMAGES
This should have been marked when the docker recipe was added to
prevent it being used for cross compiling QEMU. Sort the
DEBIAN_PARTIAL_IMAGE list while we are at it.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-09-10 14:14:31 +01:00
Alex Bennée
4d26c7fef4 tests/docker: pin powerpc-user-cross to a snapshot
Now Jessie has entered LTS the powerpc architecture has been dropped
so we can no longer build the image from scratch. However we can use
the snapshot archive to build the last working version.

This now only lives on an example of setting up a user-cross image as
at least on x86-64 we can use the Buster packaged cross compiler for
building test images.

Suggested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Cc: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-09-10 14:14:31 +01:00
Alex Bennée
a3c1f1283b tests/docker: update Debian Sid image
While we are not currently using it we might as well keep the image
for later usage. So:

  - update to a more recent snapshot
  - clean up verbiage in commentary
  - remove duplicate shell from a merge failure

Suggested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-09-10 14:14:31 +01:00
Alex Bennée
8a4daee5f9 tests/docker: move our ppc64 cross compile to Buster
Now Buster is released we can stop relying on the movable feast that
is Sid for our cross-compiler for building tests.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-09-10 14:14:31 +01:00
Alex Bennée
c6e8f512d5 tests/docker: move our riscv64 cross compile to Buster
Now Buster is released we can stop relying on the movable feast that
is Sid for our cross-compiler for building tests.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-09-10 14:14:31 +01:00
Alex Bennée
4575a701ea tests/docker: move our mips64 cross compile to Buster
Now Buster is released we can stop relying on the movable feast that
is Sid for our cross-compiler for building tests.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-09-10 14:14:31 +01:00
Alex Bennée
923984c5e8 tests/docker: move our sh4 cross compile to Buster
Now Buster is released we can stop relying on the movable feast that
is Sid for our cross-compiler for building tests.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-09-10 14:14:31 +01:00
Alex Bennée
78d1d69641 tests/docker: move our sparc64 cross compile to Buster
Now Buster is released we can stop relying on the movable feast that
is Sid for our cross-compiler for building tests.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-09-10 14:14:31 +01:00
Alex Bennée
20f8b1a27a tests/docker: move our m68k cross compile to Buster
Now Buster is released we can stop relying on the movable feast that
is Sid for our cross-compiler for building tests.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-09-10 14:14:31 +01:00
Alex Bennée
c268700b9f tests/docker: move our HPPA cross compile to Buster
Now Buster is released we can stop relying on the movable feast that
is Sid for our cross-compiler for building tests.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-09-10 14:14:31 +01:00
Alex Bennée
e3386c276c tests/docker: move our Alpha cross compile to Buster
Now Buster is released we can stop relying on the movable feast that
is Sid for our cross-compiler for building tests.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-09-10 14:14:31 +01:00
Alex Bennée
ce49420913 tests/docker: move our powerpc cross compile to Buster
Now Buster is released we can stop relying on the movable feast that
is Sid for our cross-compiler for building tests.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-09-10 14:14:31 +01:00
Alex Bennée
18b6be4326 tests/docker: move our arm64 cross compile to Buster
Now Buster is released we can unify our cross build images for both
QEMU and tests.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-09-10 14:14:31 +01:00
Alex Bennée
2f45f2d446 tests/docker: add Buster to DOCKER_PARTIAL_IMAGES
We need to add additional packages to the base images to be able to
build QEMU so lets avoid building with it.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-09-10 14:14:31 +01:00
Alex Bennée
09bb808f0b tests/docker: set DEF_TARGET_LIST for some containers
You can assume the failures most people are interested in are the
cross-compile failures that are specific to the cross compile target.
Set DEF_TARGET_LIST based on what we use for shippable, the user can
always override by calling with TARGET_LIST set.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-09-10 14:14:31 +01:00
Alex Bennée
6945018a68 tests/docker: move DEF_TARGET_LIST setting to common.rc
We might as well not repeat ourselves. At the same time allow it to be
overridden which we will use later from docker targets.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-09-10 14:14:31 +01:00
Alex Bennée
eea2153ea8 tests/docker: fix final missing .encode when parsing solibs
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-09-10 09:38:33 +01:00
Alex Bennée
884fcafc9c tests/docker: handle missing encoding keyword for subprocess.check_output
This was only added in Python 3.6 and not all the build hosts have
that recent a python3. However we still need to ensure everything is
returns as a unicode string so checks higher up the call chain don't
barf.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>

fixup! tests/docker: handle missing encoding keyword for subprocess.check_output
2019-09-10 09:38:33 +01:00
Alex Bennée
71ebbe09e9 tests/docker: fix "cc" command to work with podman
Podman requires a little bit of additional magic to the uid mapping
which was already done for the normal RunCommand. We simplify the
logic by pushing it directly into the Docker::run method to avoid
instantiating an extra Docker() object and ensure the CC command
always runs as the current user.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-09-10 09:38:33 +01:00
John Snow
63772d5cfd tests/docker: Use --userns=keep-id for podman
The workaround that attempts to accomplish the same result as --userns=keep-id
does not appear to work well with UIDs much above 1000 (like mine, which is
above 20000.)

Since we have official support for this "trick" now, use the supported method.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20190904232451.26466-1-jsnow@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-09-10 09:38:33 +01:00
Alex Bennée
8480517d4c configure: clean-up container cross compile detect
The introduction of podman support inadvertently broke configure's
detect of the container support as the configure probe didn't specify
an engine type. To fix this in docker.py:

  - only (re)set USE_ENGINE if --engine is specified
  - enhance the output so docker is no longer just yes

In the configure script we can at least start cleaning up the
detecting and naming of variables. To avoid too much churn the
conversion of the various make DOCKER_foo variables has been left for
future clean-ups.

Fixes: 9459f75413
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: David Hildenbrand <david@redhat.com>
2019-09-10 09:38:33 +01:00
Alex Bennée
4112aff7cd tests/docker: upgrade docker.py to python3
The recent podman changes (9459f75413) imported enum which is part
of the python3 standard library but only available as an external
library for python2. This causes problems on the fairly restricted
environment such as shippable. Lets bite the bullet and make the
script a fully python3 one. To that end:

  - drop the from __future__ import (we are there now ;-)
  - avoid the StringIO import hack
  - be consistent with the mode we read/write dockerfiles
  - s/iteritems/items/
  - ensure check_output returns strings for processing

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Cc: Marc-André Lureau <marcandre.lureau@redhat.com>
2019-09-02 10:52:22 +01:00
Marc-André Lureau
05af039d1e tests/docker: add podman support
Allow to specify the container engine to run with ENGINE variable.

By default, ENGINE=auto and will select either podman or docker.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2019-08-22 17:30:25 +04:00
Marc-André Lureau
9459f75413 docker.py: add podman support
Add a --engine option to select either docker, podman or auto.

Among other advantages, podman allows to run rootless & daemonless
containers, fortunately sharing compatible CLI with docker.

With current podman, we have to use a uidmap trick in order to be able
to rw-share the ccache directory with the container user.

With a user 1000, the default mapping is:                                                                                                                                                                         1000 (host) -> 0 (container).
So write access to /var/tmp/ccache ends will end with permission
denied error.

With "--uidmap 1000:0:1 --uidmap 0:1:1000", the mapping is:
1000 (host) -> 0 (container, 1st namespace) -> 1000 (container, 2nd namespace).
(the rest is mumbo jumbo to avoid holes in the range of UIDs)

A future podman version may have an option such as --userns-keep-uid.
Thanks to Debarshi Ray <rishi@redhat.com> for the help!

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Acked-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2019-08-22 17:30:25 +04:00
Marc-André Lureau
2461d80e6c docker.py: add --run-as-current-user
(podman will need further tweaks)

Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2019-08-22 17:30:25 +04:00
Philippe Mathieu-Daudé
98808c3d0c tests/docker: Refresh APT cache before installing new packages on Debian
Since docker caches the different layers, updating the package
list does not invalidate the previous "apt-get update" layer,
and it is likely "apt-get install" hits an outdated repository.

See commit beac6a98f6 and
https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#apt-get

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20190723141528.18023-1-philmd@redhat.com>
[AJB: manually applies and fixed up]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-07-23 17:19:11 +01:00
Alex Bennée
2b0c4fa13f tests/docker: invoke the DEBUG shell with --noprofile/--norc
It's very confusing when things work in the debug shell because the
environment is different from what the test is running. Fix this by
ensuring we only have the inherited environment from the run shell.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-07-23 15:53:25 +01:00
Philippe Mathieu-Daudé
5deb2de1d6 tests/docker: Let the test-mingw test generate a NSIS installer
The NSIS installer generates an executable suitable to install
QEMU on Windows.

Suggested-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20190715174817.18981-9-philmd@redhat.com>
[AJB: also --enable-docs in configure step]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-07-23 15:53:25 +01:00
Philippe Mathieu-Daudé
10d7166e42 tests/docker: Install texinfo in the Fedora image
The 'makeinfo' is required to generate the documentation from
the 'html' Makefile rule (called by 'install-doc').
The NSIS installer uses these files.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20190715174817.18981-6-philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-07-23 15:53:25 +01:00
Philippe Mathieu-Daudé
a88819edde tests/docker: Set the correct cross-PKG_CONFIG_PATH in the MXE images
This silents a bunch of warnings while compiling the Slirp objects:

 $ make
 [...]
   CC      slirp/src/tftp.o
 Package glib-2.0 was not found in the pkg-config search path.
 Perhaps you should add the directory containing `glib-2.0.pc'
 to the PKG_CONFIG_PATH environment variable
 No package 'glib-2.0' found
   CC      slirp/src/udp6.o
 Package glib-2.0 was not found in the pkg-config search path.
 Perhaps you should add the directory containing `glib-2.0.pc'
 to the PKG_CONFIG_PATH environment variable
 No package 'glib-2.0' found
 [...]

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20190715174817.18981-5-philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-07-23 15:53:25 +01:00
Philippe Mathieu-Daudé
c054f3f43a tests/docker: Install the NSIS tools in the MinGW capable images
This fixes:

  $ make installer
  (cd /tmp/qemu-nsis; \
           for i in qemu-system-*.exe; do \
             arch=${i%.exe}; \
             arch=${arch#qemu-system-}; \
             echo Section \"$arch\" Section_$arch; \
             echo SetOutPath \"\$INSTDIR\"; \
             echo File \"\${BINDIR}\\$i\"; \
             echo SectionEnd; \
           done \
          ) >/tmp/qemu-nsis/system-emulations.nsh
  makensis -V2 -NOCD \
                  -DCONFIG_DOCUMENTATION="y" \
                   \
                  -DBINDIR="/tmp/qemu-nsis" \
                   \
                  -DSRCDIR="/home/phil/source/qemu" \
                  -DOUTFILE="qemu-setup-4.0.90.exe" \
                  -DDISPLAYVERSION="4.0.90" \
                  /home/phil/source/qemu/qemu.nsi
  /bin/sh: 1: makensis: not found
  Makefile:1077: recipe for target 'qemu-setup-4.0.90.exe' failed
  make: *** [qemu-setup-4.0.90.exe] Error 127

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20190715174817.18981-4-philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-07-23 15:53:25 +01:00
Philippe Mathieu-Daudé
8d16a31028 tests/docker: Install Sphinx in the Debian images
Since commit 5f71eac06e the Sphinx tool is required
to build the rST documentation.

This fixes:

 $ ./configure --enable-docs

 ERROR: User requested feature docs
        configure was not able to find it.
        Install texinfo, Perl/perl-podlators and python-sphinx

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20190715174817.18981-3-philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-07-23 15:53:25 +01:00
Alex Bennée
67bd36beda tests/dockerfiles: update the win cross builds to stretch
While fixing up pkg.mxe.cc they move the URLs around a bit and dropped
Jessie support in favour of Stretch. We also need to update the keys
used to verify the packages.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-07-23 15:53:25 +01:00
Philippe Mathieu-Daudé
56e89a310b tests/docker: Install Ubuntu images noninteractively
We correctly use the DEBIAN_FRONTEND environment variable on
the Debian images, but forgot the Ubuntu ones are based on it.

Since building docker images is not interactive, we need to
inform the APT tools about it using the DEBIAN_FRONTEND
environment variable (we already use it on our Debian images).

This fixes:

  $ make docker-image-ubuntu V=1
  [...]
  Setting up tzdata (2019b-0ubuntu0.19.04) ...
  debconf: unable to initialize frontend: Dialog
  debconf: (TERM is not set, so the dialog frontend is not usable.)
  debconf: falling back to frontend: Readline
  Configuring tzdata
  ------------------

  Please select the geographic area in which you live. Subsequent configuration
  questions will narrow this down by presenting a list of cities, representing
  the time zones in which they are located.

    1. Africa      4. Australia  7. Atlantic  10. Pacific  13. Etc
    2. America     5. Arctic     8. Europe    11. SystemV
    3. Antarctica  6. Asia       9. Indian    12. US
  Geographic area: 12
  [HANG]

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20190711124805.26476-1-philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-07-23 15:53:25 +01:00
Philippe Mathieu-Daudé
8109b7986a tests/docker: Install Sphinx in the Fedora image
Since commit 5f71eac06e the Sphinx tool is required
to build the rST documentation.

This fixes:

 $ ./configure --enable-docs

 ERROR: User requested feature docs
        configure was not able to find it.
        Install texinfo, Perl/perl-podlators and python-sphinx

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20190711102710.2263-1-philmd@redhat.com>
[AJB: also add /usr/libexec/python3-sphinx/ to PATH]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-07-23 15:53:25 +01:00
Philippe Mathieu-Daudé
6d7b9c8c06 tests/docker: Install Sphinx in the Ubuntu images
Since commit 5f71eac06e the Sphinx tool is required
to build the rST documentation.

This fixes:

 $ ./configure --enable-docs

 ERROR: User requested feature docs
        configure was not able to find it.
        Install texinfo, Perl/perl-podlators and python-sphinx

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20190711120609.12773-1-philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Stefano Garzarella <sgarzare@redhat.com>
2019-07-23 15:53:25 +01:00
Alex Bennée
3c237b5c65 tests/docker: add test-misc for building tools & docs
Add yet another test type so we cna quickly exercise the miscellaneous
build products of the build system under various docer configurations.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-07-23 15:53:25 +01:00
Pino Toscano
b10d49d761 ssh: switch from libssh2 to libssh
Rewrite the implementation of the ssh block driver to use libssh instead
of libssh2.  The libssh library has various advantages over libssh2:
- easier API for authentication (for example for using ssh-agent)
- easier API for known_hosts handling
- supports newer types of keys in known_hosts

Use APIs/features available in libssh 0.8 conditionally, to support
older versions (which are not recommended though).

Adjust the iotest 207 according to the different error message, and to
find the default key type for localhost (to properly compare the
fingerprint with).
Contributed-by: Max Reitz <mreitz@redhat.com>

Adjust the various Docker/Travis scripts to use libssh when available
instead of libssh2. The mingw/mxe testing is dropped for now, as there
are no packages for it.

Signed-off-by: Pino Toscano <ptoscano@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Acked-by: Alex Bennée <alex.bennee@linaro.org>
Message-id: 20190620200840.17655-1-ptoscano@redhat.com
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-id: 5873173.t2JhDm7DL7@lindworm.usersys.redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-06-24 16:01:04 +02:00
Alex Bennée
248cf06cf2 tests/docker: Update the Ubuntu image to 19.04
This has aged a little and we have a separate LTS image for testing on
the older distros. Update it to a more recent release like its Fedora
cousin.

Besides it is useful to have something with gcc-9 on it for squashing
those stringop truncation errors.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
2019-06-12 17:53:22 +01:00
Alex Bennée
acc5c5061d tests/docker: Update the Fedora cross compile images to 30
While at it remove the bogus :latest tag for cris cross compiler. It
tends to break caching and cause confusion.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-06-12 17:53:22 +01:00
Philippe Mathieu-Daudé
c87e38399c tests/docker: Update the Fedora image to Fedora 30
Fedora 30 got released:

  https://fedoramagazine.org/announcing-fedora-30/

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Tested-by: Stefano Garzarella <sgarzare@redhat.com>
Reviewed-by: Stefano Garzarella <sgarzare@redhat.com>
Message-Id: <20190528153304.27157-1-philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-06-12 17:53:22 +01:00
David Gibson
c10aaaab0f tests: Fix up docker cross builds for ppc64 (BE) targets
We currently have docker cross building targets for powerpc (32-bit, BE)
and ppc64el (64-bit, LE), but not for pcp64 (64-bit, BE).  This is an
irritating gap in make check-tcg coverage so correct it.

Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
2019-05-29 11:14:19 +10:00
Philippe Mathieu-Daudé
8ea5962f28 tests/docker: Test more components on the Fedora default image
Install optional dependencies of QEMU to get better coverage.

The following components are now enabled:

  $ ./configure
  ...
  Multipath support yes
  VNC SASL support  yes
  RDMA support      yes
  PVRDMA support    yes
  libiscsi support  yes
  seccomp support   yes
  libpmem support   yes
  libudev           yes

Note: The udev-devel package is provided by systemd-devel.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20190504055440.20406-1-philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Acked-by: Marcel Apfelbaum<marcel.apfelbaum@gmail.com>
2019-05-28 10:28:51 +01:00
Gerd Hoffmann
e8e875b1a3 tests/docker: add ubuntu 18.04
Based on the ubuntu.docker file.
Used to reproduce the build failure Peter was seeing.
Others might find this useful too ;)

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Message-Id: <20190503070241.24786-1-kraxel@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-05-28 10:28:51 +01:00
Wainer dos Santos Moschetta
87db901820 docker: trivial changes to make docker help
Apply double quotes and period punctuation uniformly.

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20190321212528.6100-1-wainersm@redhat.com>
Reviewed-by: Fam Zheng <fam@euphon.net>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-03-25 10:39:19 +00:00
Wainer dos Santos Moschetta
a6de52ac7a docker: Fix travis script unable to find source dir
The script generated from QEMU_SRC/.travis.yml uses BUILD_DIR and
SRC_DIR path relative to the current dir, unless these variables
are exported in environment.

Since commit 05790dafef BUILD_DIR is exported in the runner script,
although SRC_DIR is not, so that make docker-travis fails becase
the reference to source dir is wrong. So let's unset both BUILD_DIR
and SRC_DIR before calling the script, given it is executed from
the source dir already (as in Travis).

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20190320221207.11366-3-wainersm@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-03-25 10:35:42 +00:00
Wainer dos Santos Moschetta
e8ced6813d docker: Fix travis.py parser and misc change
Fixed the travis.py script that has failed to parse the current
QEMU_SRC/.travis.yml file. It no longer makes combinations from
env/matrix, instead it uses explicit includes. Also the compiler
can be omitted from matrix/include, so that Travis chooses the
first entry of the global compiler list.

Replaced yaml.load() with yaml.safe_load() so that quieting the
following deprecation warning:
https://github.com/yaml/pyyaml/wiki/PyYAML-yaml.load(input)-Deprecation

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20190320221207.11366-2-wainersm@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-03-25 10:35:42 +00:00
Alex Bennée
d72132c02e tests/tcg: enable cris base user-mode tests
This converts the existing Makefile into a Makefile.target and updates
it so it can be called by the tcg build system. The original Makefile
didn't set -cpu except for the v17 tests however that has broken (I
assume because linux-user is a "max" cpu) so here I force it to be
crisv17.

I've also replicated the GNU simulator targets (run-FOO-on-sim).

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-03-12 17:05:21 +00:00
Alex Bennée
824ad15f1c tests/docker: add fedora-cris-cross compilers
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-03-12 17:05:21 +00:00
Philippe Mathieu-Daudé
c72d9df181 tests/docker: add debian-xtensa-cross image
Xtensa cpu supported:
- dc232b
- dc233c
- csp

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-03-12 17:05:21 +00:00
Alex Bennée
e92a43bb18 tests/docker: peg netmap code to a specific version
Tracking head is always going to be at the whims of the upstream.
Let's use a defined release so things don't magically change under us.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
2019-02-22 09:32:32 +00:00
Alex Bennée
a48a1d1827 tests/docker: squash initial update and install step for debian9
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-02-22 09:32:20 +00:00
Alex Bennée
432d8ad5f6 tests: PEP8 cleanup of docker.py, mostly white space
My editor keeps putting squiggly lines under a bunch of the python
lines to remind me how non-PEP8 compliant it is. Clean that up so it's
easier to spot new errors.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-02-08 17:32:35 +00:00
Alex Bennée
d10404b193 tests: docker.py be even smarter with persistent binfmt_misc
If we have a persistent mapping we don't need the QEMU binary copied
into the container as the kernel has already opened the file and will
pass the fd in. However the support libraries will still need to be
there.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-02-08 17:32:35 +00:00
Alex Bennée
43c898b75a tests: make docker.py check for persistent configs
binfmt_misc configured with the "F" flag opens the interpreter at
config time. This means it can use an already open file-descriptor to
run QEMU so there is no point trying to copy the binary into a
container.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-02-08 17:32:35 +00:00