Running every plugin with every test is getting excessive as well as
not really improving coverage that much. Restrict the plugin tests to
just the MULTIARCH_TESTS which are shared between most architecture
for both system and user-mode. For those that aren't we need to squash
MULTIARCH_TESTS so we don't add them when they are not part of the
TESTS global.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20230424092249.58552-14-alex.bennee@linaro.org>
So you can do:
cd tests/tcg/aarch64-linux-user
make -f ../Makefile.target help
To see the list of tests. You can then run each one individually.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20230315174331.2959-8-alex.bennee@linaro.org>
linux-user blocks all signals while attempting to handle guest
signals (e.g. ABRT), which means that the default TERM sent by timeout
has no effect -- KILL instead.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20230117035701.168514-2-richard.henderson@linaro.org>
[AJB: expanded commit message from cover letter]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20230124180127.1881110-17-alex.bennee@linaro.org>
Failure to set pc_succ_insn may result in a TB covering zero bytes,
which triggers an assert within the code generator.
Cc: qemu-stable@nongnu.org
Resolves: https://gitlab.com/qemu-project/qemu/-/issues/1224
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20221203175744.151365-1-richard.henderson@linaro.org>
[ Changes by AF:
- Add missing run-plugin-test-noc-% line
]
Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
Further decoupling of tests/tcg from the main QEMU Makefile, and making
the build more similar between the cross compiler case and the vetted
container images.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220929114231.583801-25-alex.bennee@linaro.org>
Almost all invocations of run-test have either "$* on $(TARGET_NAME)"
or "$< on $(TARGET_NAME)" as the last argument. So provide a default
test name, while allowing an escape hatch for custom names.
As an additional simplification, remove the need to do shell quoting.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220929114231.583801-24-alex.bennee@linaro.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220929114231.583801-22-alex.bennee@linaro.org>
Remove the DOCKER_SCRIPT and TARGET variable from the Makefile invocation
for tests/tcg. For DOCKER_SCRIPT, resolve the path to docker.py in configure;
for TARGET, move it to config-$(TARGET).mak and use a symbolic link to break
the cycle.
The symbolic link is still needed because tests/tcg includes dummy config files
for targets that are not buildable. Once that is cleaned up, the symbolic link
will go away too.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220929114231.583801-20-alex.bennee@linaro.org>
Tests for correct operation of most x86-64 SSE instructions.
It should cover all combinations of overlapping register and memory
operands on a set of random-ish data.
Results are bit-identical to an Intel i5-8500, with the exception of
the RCPSS and RSQRT approximations where the real CPU gives less accurate
results (the Intel spec allows relative errors up to 1.5 * 2^-12)
Signed-off-by: Paul Brook <paul@nowt.org>
Acked-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20220424220204.2493824-42-paul@nowt.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
System emulation tests do not run in a hosted environment, since they
do not link with libc. They should only use freestanding headers
(float.h, limits.h, stdarg.h, stddef.h, stdbool.h, stdint.h,
stdalign.h, stdnoreturn.h) and should be compiled with -ffreestanding
in order to use the compiler implementation of those headers
rather than the one in libc.
Some tests are using inttypes.h instead of stdint.h, so fix that.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Do not include variables for the QEMU's own compiler, as they
are not necessarily related to the cross compiler used for tests/tcg.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20220401141326.1244422-16-pbonzini@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220419091020.3008144-19-alex.bennee@linaro.org>
Build the "docker.py cc" invocation directly in tests/tcg/configure.sh, and
remove the Makefile.qemu wrapper around Makefile.target. The config-*.mak
files now include the actual variables used when building the tests, rather
than the CROSS_* variables that Makefile.qemu used to "translate".
This is a first step towards generalizing the cross-compilation infrastructure
so that it can be used for firmware as well.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20220401141326.1244422-15-pbonzini@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220419091020.3008144-18-alex.bennee@linaro.org>
Just check the target name instead.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20220401141326.1244422-12-pbonzini@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220419091020.3008144-15-alex.bennee@linaro.org>
This was attempted in commit 533b0a1a41 ("tests/tcg: Fix target-specific
Makefile variables path for user-mode", 2022-01-12) but it also used the
wrong path; default.mak is used for config/devices, not config/targets.
While at it, explain what the inclusion is about.
Cc: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Commit 812b31d3f9 refactor missed to update this path.
Fixes: 812b31d3f9 ("configs: rename default-configs to configs and reorganise")
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20211226001541.3807919-1-f4bug@amsat.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
The cross-i386-tci test has timeouts because we're no longer
applying the timeout that we desired. Hack around it.
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Fixes: 23a77b2d18 ("build-system: clean up TCG/TCI configury")
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
The longest test at the moment seems to be a (slower)
aarch64 host, for which test-mmap takes 64 seconds.
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Acked-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
when trying to run successful short tests from the Makefile timeout would not
terminate. Rather it would wait until the time runs out. Excerpt from the
manpage:
--foreground
when not running timeout directly from a shell prompt,
allow COMMAND to read from the TTY and get TTY signals; in this mode, chil‐
dren of COMMAND will not be timed out
Signed-off-by: Bastian Koppelmann <kbastian@mail.uni-paderborn.de>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210305170045.869437-3-kbastian@mail.uni-paderborn.de>
Message-Id: <20210512102051.12134-13-alex.bennee@linaro.org>
It may be arm-compat-semihosting but more than one architecture uses
it so lets move the tests into the multiarch area. We gate it on the
feature and split the semicall.h header between the arches.
Also clean-up a bit of the Makefile messing about to one common set of
runners.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20210323165308.15244-6-alex.bennee@linaro.org>
Hopefully this will guard against sloppy code getting into our tests.
Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20201210190417.31673-9-alex.bennee@linaro.org>
We have an exploding complexity problem in the testing so lets just
move the more involved plugins into contrib. tests/plugins still exist
for the basic plugins that exercise the API. We restore the old
pre-meson style Makefile for contrib as it also doubles as a guide for
out-of-tree plugin builds.
While we are at it add some examples to the documentation and a
specific plugins build target.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200909112742.25730-11-alex.bennee@linaro.org>
When we make changes to the TCG we sometimes cause regressions that
are deep into the execution cycle of the guest. Debugging this often
requires comparing large volumes of trace information to figure out
where behaviour has diverged.
The lockstep plugin utilises a shared socket so two QEMU's running
with the plugin will write their current execution position and wait
to receive the position of their partner process. When execution
diverges the plugins output where they were and the previous few
blocks before unloading themselves and letting execution continue.
Originally I planned for this to be most useful with -icount but it
turns out you can get divergence pretty quickly due to asynchronous
qemu_cpu_kick_rr_cpus() events causing one side to eventually run into
a short block a few cycles before the other side. For this reason I've
added a bit of tracking and I think the divergence reporting could be
finessed to report only if we really start to diverge in execution.
An example run would be:
qemu-system-sparc -monitor none -parallel none -net none \
-M SS-20 -m 256 -kernel day11/zImage.elf \
-plugin ./tests/plugin/liblockstep.so,arg=lockstep-sparc.sock \
-d plugin,nochain
with an identical command in another window in the same working
directory.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Robert Foley <robert.foley@linaro.org>
Tested-by: Robert Foley <robert.foley@linaro.org>
Cc: Richard Henderson <richard.henderson@linaro.org>
Cc: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Message-Id: <20200610155509.12850-3-alex.bennee@linaro.org>
If you jump back and forth between branches while developing plugins
you end up debugging failures caused by plugins left in the build
directory. Fix this by basing plugins on the source tree instead.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200615141922.18829-2-alex.bennee@linaro.org>
When combined with heavy plugins we occasionally hit the timeouts.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20200225124710.14152-17-alex.bennee@linaro.org>
If CONFIG_PLUGINS is enabled then lets enable testing for all our TCG
targets. This is a simple smoke test that ensure we don't crash or
otherwise barf out by running each plugin against each test.
There is a minor knock on effect for additional runners which need
specialised QEMU_OPTS which will also need to declare a plugin version
of the runner. If this gets onerous we might need to add another
helper.
Checking the results of the plugins is left for a later exercise.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Otherwise clever expanders like the plugins test get unstuck.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Since moving where the tests are run the path to config-host.mak has
been wrong. This doesn't affect much but things like the time fallback
for CONFIG_DEBUG_TCG and will also get in the way of checking for
PLUGINS support.
Fixes: fc76c56d3f
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
This is a generic floating point multiply and accumulate test for
single precision floating point values. I've split of the common float
functions into a helper library so additional tests can use the same
common code.
As I don't have references for all architectures I've allowed some
flexibility for tests to pass without reference files. They can be
added as we get collect them.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
These were missed in the recent de-tangling so have been updated to be
more actuate. I've also built up ARM_TESTS in a manner similar to
AARCH64_TESTS for better consistency.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Avoid the repeated inclusions of config-target.mak, which have
risks of namespace pollution, and instead build minimal configuration
files in a configuration script. The same configuration files can
also be included in Makefile and Makefile.qemu
[AJB 10/09/19]
In the original PR this had inadvertently enabled tests
for ppc64abi32. However as the rest of the multiarch tests work rather
than disabling the otherwise correctly functioning build I've just
skipped the failing linux-test test. For some reason I can't debug it
with TCG so I'm leaving that to the PPC maintainers to look at.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20190807143523.15917-4-pbonzini@redhat.com>
[AJB: s/docker/container/, rm last bits from configure, ppc6432abi hack]
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Cc: Richard Henderson <rth@twiddle.net>
Rename Makefile.probe to Makefile.prereqs and make it actually
define rules for the tests.
Rename Makefile to Makefile.target, since it is not a toplevel
makefile.
Rename Makefile.include to Makefile.qemu and disentangle it
from the QEMU Makefile.target, so that it is invoked recursively
by tests/Makefile.include. Tests are now placed in
tests/tcg/$(TARGET).
Drop the usage of TARGET_BASE_ARCH, which is ignored by everything except
x86_64 and aarch64. Fix x86 tests by using -cpu max and, while
at it, standardize on QEMU_OPTS for aarch64 tests too.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20190807143523.15917-3-pbonzini@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
