When we make changes to the TCG we sometimes cause regressions that
are deep into the execution cycle of the guest. Debugging this often
requires comparing large volumes of trace information to figure out
where behaviour has diverged.
The lockstep plugin utilises a shared socket so two QEMU's running
with the plugin will write their current execution position and wait
to receive the position of their partner process. When execution
diverges the plugins output where they were and the previous few
blocks before unloading themselves and letting execution continue.
Originally I planned for this to be most useful with -icount but it
turns out you can get divergence pretty quickly due to asynchronous
qemu_cpu_kick_rr_cpus() events causing one side to eventually run into
a short block a few cycles before the other side. For this reason I've
added a bit of tracking and I think the divergence reporting could be
finessed to report only if we really start to diverge in execution.
An example run would be:
qemu-system-sparc -monitor none -parallel none -net none \
-M SS-20 -m 256 -kernel day11/zImage.elf \
-plugin ./tests/plugin/liblockstep.so,arg=lockstep-sparc.sock \
-d plugin,nochain
with an identical command in another window in the same working
directory.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Robert Foley <robert.foley@linaro.org>
Tested-by: Robert Foley <robert.foley@linaro.org>
Cc: Richard Henderson <richard.henderson@linaro.org>
Cc: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Message-Id: <20200610155509.12850-3-alex.bennee@linaro.org>
The check-tcg plugins build was failing because some special case
tests that needed -cpu max failed because the plugin variant hadn't
carried across the QEMU_OPTS tweak.
Guests which globally set QEMU_OPTS=-cpu FOO where unaffected.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200615141922.18829-3-alex.bennee@linaro.org>
If you jump back and forth between branches while developing plugins
you end up debugging failures caused by plugins left in the build
directory. Fix this by basing plugins on the source tree instead.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200615141922.18829-2-alex.bennee@linaro.org>
We do this on our other platforms to make it easier to see what has
broken.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Li-Wen Hsu <lwhsu@FreeBSD.org>
Message-Id: <20200612190237.30436-19-alex.bennee@linaro.org>
This reverts commit 12d43b5ae916809aad9ccf8aa2a0a06260527340.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200612190237.30436-18-alex.bennee@linaro.org>
Disable a few tests under CONFIG_TSAN, which
run into a known TSan issue that results in a hang.
https://github.com/google/sanitizers/issues/1116
The disabled tests under TSan include all the qtests as well as
the test-char, test-qga, and test-qdev-global-props.
Signed-off-by: Robert Foley <robert.foley@linaro.org>
Reviewed-by: Emilio G. Cota <cota@braap.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200609200738.445-14-robert.foley@linaro.org>
Message-Id: <20200612190237.30436-17-alex.bennee@linaro.org>
Adds TSan details to testing.rst.
This includes background and reference details on TSan,
and details on how to build and test with TSan
both with and without docker.
Signed-off-by: Robert Foley <robert.foley@linaro.org>
Reviewed-by: Emilio G. Cota <cota@braap.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200609200738.445-13-robert.foley@linaro.org>
Message-Id: <20200612190237.30436-16-alex.bennee@linaro.org>
This allows us to see the name of the thread in tsan
warning reports such as this:
Thread T7 'CPU 1/TCG' (tid=24317, running) created by main thread at:
Signed-off-by: Robert Foley <robert.foley@linaro.org>
Reviewed-by: Emilio G. Cota <cota@braap.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200609200738.445-12-robert.foley@linaro.org>
Message-Id: <20200612190237.30436-15-alex.bennee@linaro.org>
These annotations will allow us to give tsan
additional hints. For example, we can inform
tsan about reads/writes to ignore to silence certain
classes of warnings.
We can also annotate threads so that the proper thread
naming shows up in tsan warning results.
Signed-off-by: Robert Foley <robert.foley@linaro.org>
Reviewed-by: Emilio G. Cota <cota@braap.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200609200738.445-11-robert.foley@linaro.org>
Message-Id: <20200612190237.30436-14-alex.bennee@linaro.org>
Added a new docker for ubuntu 20.04.
This docker has support for Thread Sanitizer
including one patch we need in one of the header files.
https://github.com/llvm/llvm-project/commit/a72dc86cd
This command will build with tsan enabled:
make docker-test-tsan-ubuntu2004 V=1
Also added the TSAN suppresion file to disable certain
cases of TSAN warnings.
Cc: Fam Zheng <fam@euphon.net>
Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Robert Foley <robert.foley@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200609200738.445-10-robert.foley@linaro.org>
Message-Id: <20200612190237.30436-13-alex.bennee@linaro.org>
The radix tree is append-only, but we can fail to insert
a PageDesc if the insertion races with another thread.
Signed-off-by: Emilio G. Cota <cota@braap.org>
Signed-off-by: Robert Foley <robert.foley@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200609200738.445-8-robert.foley@linaro.org>
Message-Id: <20200612190237.30436-11-alex.bennee@linaro.org>
I was after adding qemu_spin_destroy calls, but while at
it I noticed that we are leaking some memory.
Signed-off-by: Emilio G. Cota <cota@braap.org>
Signed-off-by: Robert Foley <robert.foley@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200609200738.445-5-robert.foley@linaro.org>
Message-Id: <20200612190237.30436-8-alex.bennee@linaro.org>
It will be used for TSAN annotations.
Signed-off-by: Emilio G. Cota <cota@braap.org>
Signed-off-by: Robert Foley <robert.foley@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200609200738.445-4-robert.foley@linaro.org>
Message-Id: <20200612190237.30436-7-alex.bennee@linaro.org>
We convert queued work to a QSIMPLEQ, instead of
open-coding it.
While at it, make sure that all accesses to the list are
performed while holding the list's lock.
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Emilio G. Cota <cota@braap.org>
Signed-off-by: Robert Foley <robert.foley@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200609200738.445-3-robert.foley@linaro.org>
Message-Id: <20200612190237.30436-6-alex.bennee@linaro.org>
We tried running QEMU under tsan in 2016, but tsan's lack of support for
longjmp-based fibers was a blocker:
https://groups.google.com/forum/#!topic/thread-sanitizer/se0YuzfWazw
Fortunately, thread sanitizer gained fiber support in early 2019:
https://reviews.llvm.org/D54889
This patch brings tsan support upstream by importing the patch that annotated
QEMU's coroutines as tsan fibers in Android's QEMU fork:
https://android-review.googlesource.com/c/platform/external/qemu/+/844675
Tested with '--enable-tsan --cc=clang-9 --cxx=clang++-9 --disable-werror'
configure flags.
Signed-off-by: Lingfeng Yang <lfy@google.com>
Signed-off-by: Emilio G. Cota <cota@braap.org>
[cota: minor modifications + configure changes]
Signed-off-by: Robert Foley <robert.foley@linaro.org>
[RF: configure changes, coroutine fix + minor modifications]
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200609200738.445-2-robert.foley@linaro.org>
Message-Id: <20200612190237.30436-5-alex.bennee@linaro.org>
dtc submodule update, now call the libfdt target from the new
dtc Makefile, which has been changed to not require bison, flex, etc.
This removes warnings during the build.
scripts/ symlink and tests directory creation are not necessary,
and neither is calling the clean rule explicitly.
Signed-off-by: Claudio Fontana <cfontana@suse.de>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200518160319.18861-2-cfontana@suse.de>
Message-Id: <20200612190237.30436-3-alex.bennee@linaro.org>
We should be keeping this up to date as Fedora goes out of support
quite quickly.
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20200612190237.30436-2-alex.bennee@linaro.org>
This patch adds translation of QEMU to Swedish.
Signed-off-by: Sebastian Rasmussen <sebras@gmail.com>
Signed-off-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>
Reviewed-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>
Message-Id: <20200611114523.15584-2-aleksandar.qemu.devel@gmail.com>
This patch transfers sh4 sections to Yoshinori Sato, who is
best positioned in the community to assume sh4 maintainership.
He is the maintainer of the related target rx as well, which
means that some synergy between the two targets can be expected
in future.
Further adjustments, reorganizations, and improvements of sh4
sections are left to the future maintainer to be devised and
executed, as he deems suitable.
Aurelien and Magnus are deleted as maintainers in some sections
of the MAINTAINERS file with this patch. However, they will not
be deleted from QEMU Hall of Fame, where their names will always
remained carved in stone as QEMU pioneers and granddaddies.
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Acked-by: Aurelien Jarno <aurelien@aurel32.net>
Acked-by: Magnus Damm <magnus.damm@gmail.com>
Acked-by: Yoshinori Sato <ysato@users.sourceforge.jp>
Signed-off-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20200611095316.10133-2-aleksandar.qemu.devel@gmail.com>
Abbreviations of vendor-specific ASEs looks very similiar.
Add comments to explain the full name and vendors of these flags.
Signed-off-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
Reviewed-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>
Signed-off-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>
Message-Id: <20200614080049.31134-3-jiaxun.yang@flygoat.com>
To match the actual status of Loongson insn, we split flags
for LMMI and LEXT from INSN_LOONGSON2F.
As Loongson-2F only implemented interger part of LEXT, we'll
not enable LEXT for the processor, but instead we're still using
INSN_LOONGSON2F as switch flag of these instructions.
All multimedia instructions have been moved to LMMI flag. Loongson-2F
and Loongson-3A are sharing these instructions.
Signed-off-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
Reviewed-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>
Signed-off-by: Aleksandar Markovic <aleksandar.qemu.devel@gmail.com>
Message-Id: <20200614080049.31134-2-jiaxun.yang@flygoat.com>
The git-submodule.sh script is called by make and initialize the
submodules listed in the GIT_SUBMODULES variable generated by
./configure.
SLOF is required for building the s390-ccw firmware on s390x, since
it is using the libnet code from SLOF for network booting.
Add it to the GIT_SUBMODULES when building the s390-ccw firmware.
Reported-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Suggested-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20200615074919.12552-1-f4bug@amsat.org>
[thuth: Tweaked the commit message a little bit]
Signed-off-by: Thomas Huth <thuth@redhat.com>
Now that we can select the second serial console in the acceptance tests
(see commit 746f244d9720 "Allow to use other serial consoles than default"),
we can also test the sh4 image from the QEMU advent calendar 2018.
Message-Id: <20200515164337.4899-1-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
It's either "GNU *Library* General Public License version 2" or "GNU
Lesser General Public License version *2.1*", but there was no "version
2.0" of the "Lesser" license. So assume that version 2.1 is meant here.
Message-Id: <20200605100645.6506-1-thuth@redhat.com>
Reviewed-by: Laurent Vivier <lvivier@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
It is neater to keep this in the QEMU repo, since any change that
requires an update to the oss-fuzz build configuration, can make the
necessary changes in the same series.
Suggested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
Reviewed-by: Darren Kenny <darren.kenny@oracle.com>
Message-Id: <20200612055145.12101-1-alxndr@bu.edu>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
Message-Id: <20200529221450.26673-3-alxndr@bu.edu>
Reviewed-by: Darren Kenny <darren.kenny@oracle.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
The QTest server usually parses ASCII commands from clients. Since we
fuzz within the QEMU process, skip the QTest serialization and server
for most QTest commands. Leave the option to use the ASCII protocol, to
generate readable traces for crash reproducers.
Inspired-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
Message-Id: <20200529221450.26673-2-alxndr@bu.edu>
Reviewed-by: Darren Kenny <darren.kenny@oracle.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
When configure is run with "--disable-tpm", the bios-tables-test
q35/tis test fails with "-tpmdev: invalid option".
Skip the test if CONFIG_TPM is unset.
Signed-off-by: Eric Auger <eric.auger@redhat.com>
Reported-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200615135051.2213-1-eric.auger@redhat.com>
Tested-by: Stefan Berger <stefanb@linux.ibm.com>
Fixes: 5da7c35e25 ("bios-tables-test: Add Q35/TPM-TIS test")
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
According to the gcrypt documentation it's intended that
gcry_check_version() is called with the minimum version of gcrypt
needed by the program, not the version from the <gcrypt.h> header file
that happened to be installed when qemu was compiled. Indeed the
gcrypt.h header says that you shouldn't use the GCRYPT_VERSION macro.
This causes the following failure:
qemu-img: Unable to initialize gcrypt
if a slightly older version of libgcrypt is installed with a newer
qemu, even though the slightly older version works fine. This can
happen with RPM packaging which uses symbol versioning to determine
automatically which libgcrypt is required by qemu, which caused the
following bug in RHEL 8:
https://bugzilla.redhat.com/show_bug.cgi?id=1840485
qemu actually requires libgcrypt >= 1.5.0, so we might put the string
"1.5.0" here. However since 1.5.0 was released in 2011, it hardly
seems we need to check that. So I replaced GCRYPT_VERSION with NULL.
Perhaps in future if we move to requiring a newer version of gcrypt we
could put a literal string here.
Signed-off-by: Richard W.M. Jones <rjones@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
