Store the child objects directly within the sparc32-dma object rather than using
link properties.
Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20200926140216.7368-2-mark.cave-ayland@ilande.co.uk>
Signed-off-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Misono
Set default log level to info
Explicit build option for virtiofsd
Me
xattr name mapping
Stefan
Alternative chroot sandbox method
Max
Submount mechanism
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
-----BEGIN PGP SIGNATURE-----
iQIzBAABCAAdFiEERfXHG0oMt/uXep+pBRYzHrxb/ecFAl+XGGAACgkQBRYzHrxb
/efeDw/8Dz+yKjdV0mqzdOJ/2lg4etlDRqG7P65W9G79ZIthAOZVIH5p3yv+qzdN
a+QTGWquCA/gCCfl29QHU2zC78PcBjP/ugm6icqW126MmJmHR/rOZx+RYC+0W4+0
2YA2HzwtKpolrMuzKoddHwsYoIF2Uw6l1oZK1QOl1hjqL2q47VRDvCs6H7vpvGn1
dGd+xkXMSYCVL4Lq+zAaIg6ZfTtCIlwJ+LMCoT/Wy7eZeB338T9Bz5iLl7BTqF2x
GBv2eDw0xibYw+3d8zX9k76irKdLYPgJiaskjsGNWxLgtSYEOmCrDPzIMDbe34lS
u4JlqRdmc62YoE5X1oI6tF8XSaD+O/PS1CT9O9IttDuHNVctg0zCqjTxONJn9IQk
CmszvoGScnaH4PqWueR47wDjKdFq8p5nryODtmuYILjvAXPYJp0Vt6JDJ6ZZcS+t
YwRYltCK7ToKiteTuosusQ/Vzk2kq4U6znWZsZH1LcyNEVaJNUIoIjYZrxKugG/F
yuAeWinoG37N6gwx5GfUoIO/eRd8UyBmKEaeY7RpJfo+UnFpErg4+uEfZ5gbD1/0
YtQmBXHrXnsZB//wTw0gUob0sDKoII21H5EcA4QiDpgci9Q5saL/XG55TDLNZ62d
uUH7PbIna8KpoKRBjEzz5e71FBTF0sKshzrZFHjfmhZ9HiM6XeE=
=dTCm
-----END PGP SIGNATURE-----
Merge remote-tracking branch 'remotes/dgilbert-gitlab/tags/pull-virtiofs-20201026' into staging
virtiofsd pull 2020-10-26
Misono
Set default log level to info
Explicit build option for virtiofsd
Me
xattr name mapping
Stefan
Alternative chroot sandbox method
Max
Submount mechanism
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
# gpg: Signature made Mon 26 Oct 2020 18:41:36 GMT
# gpg: using RSA key 45F5C71B4A0CB7FB977A9FA90516331EBC5BFDE7
# gpg: Good signature from "Dr. David Alan Gilbert (RH2) <dgilbert@redhat.com>" [full]
# Primary key fingerprint: 45F5 C71B 4A0C B7FB 977A 9FA9 0516 331E BC5B FDE7
* remotes/dgilbert-gitlab/tags/pull-virtiofs-20201026:
tests/acceptance: Add virtiofs_submounts.py
tests/acceptance/boot_linux: Accept SSH pubkey
virtiofsd: Announce sub-mount points
virtiofsd: Store every lo_inode's parent_dev
virtiofsd: Add fuse_reply_attr_with_flags()
virtiofsd: Add attr_flags to fuse_entry_param
virtiofsd: Announce FUSE_ATTR_FLAGS
linux/fuse.h: Pull in from Linux
tools/virtiofsd: xattr name mappings: Simple 'map'
tools/virtiofsd: xattr name mapping examples
tools/virtiofsd: xattr name mappings: Map server xattr names
tools/virtiofsd: xattr name mappings: Map client xattr names
tools/virtiofsd: xattr name mappings: Add option
virtiofsd: add container-friendly -o sandbox=chroot option
virtiofsd: passthrough_ll: set FUSE_LOG_INFO as default log_level
configure: add option for virtiofsd
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Another go at Peter's postcopy fixes
Cleanups from Bihong Yu and Peter Maydell.
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
-----BEGIN PGP SIGNATURE-----
iQIzBAABCAAdFiEERfXHG0oMt/uXep+pBRYzHrxb/ecFAl+W9n8ACgkQBRYzHrxb
/ef2uRAAqWTFLXuBF8+evEd1mMq2SM3ZYTuc7QKTY3MzAH6J/OMvJbZ112itqWOb
iZ5NuuWH4PvzOhlR/PNNf1Yv3hTfv36HinG+OCh6s+6aqVx9yHOAfdBgmJIdYAeg
Sk1jx43dvCyN2FwPs31ir3L6mwsrtfkRsS+2FeyrvRoEl4WE9mOoypCft3vdd9Dw
zZea0Pw7vIs454D4n1vpJiQtq6B4eSAlQKpTLfQbglpTm4MgqLERzGvpT6hbQXJR
eQyTOqRe08viIOZ+oN0B/+RVO6T9jc4Y1bEl2NSak1v4Tf7NNfDkFpLAjFm07V/1
tIhL/NOOsHdzfHQtrZpzKQgwaceb1N5qo0PfxD6/tRf9HlXY54iw6yY75+5c5Y89
UK8VSIYKnM2yXeVDLShxixIr3A1Z+zA41XydDwaLZczjeV7+nwrAXAjO8a+j6Dox
zj4IyN2g5elEOmarC8qkvbDZ+TVvA2tookhWVwoz+D8ChYkcRDKP9eoYomfRwg+e
NKRFuLBkyVPb0eEhyOV6HqJbMfTLpHneTM94v6HGz8tiK8IlMZfTTnC2Mr5gTXuS
/cgOVhsY7+l+pKpxpGJmU3aUCYRk1CuK6MhXgjYEFMh5Siba8s0ZPZVaEm/BUyO1
rD+tVup87xMiJq3xnmLX+opblYE9G+b67hH1KuPc5vZXiSwuTkQ=
=OL0Q
-----END PGP SIGNATURE-----
Merge remote-tracking branch 'remotes/dgilbert/tags/pull-migration-20201026a' into staging
migration pull: 2020-10-26
Another go at Peter's postcopy fixes
Cleanups from Bihong Yu and Peter Maydell.
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
# gpg: Signature made Mon 26 Oct 2020 16:17:03 GMT
# gpg: using RSA key 45F5C71B4A0CB7FB977A9FA90516331EBC5BFDE7
# gpg: Good signature from "Dr. David Alan Gilbert (RH2) <dgilbert@redhat.com>" [full]
# Primary key fingerprint: 45F5 C71B 4A0C B7FB 977A 9FA9 0516 331E BC5B FDE7
* remotes/dgilbert/tags/pull-migration-20201026a:
migration-test: Only hide error if !QTEST_LOG
migration/postcopy: Release fd before going into 'postcopy-pause'
migration: Sync requested pages after postcopy recovery
migration: Maintain postcopy faulted addresses
migration: Introduce migrate_send_rp_message_req_pages()
migration: Pass incoming state into qemu_ufd_copy_ioctl()
migration: using trace_ to replace DPRINTF
migration: Delete redundant spaces
migration: Open brace '{' following function declarations go on the next line
migration: Do not initialise statics and globals to 0 or NULL
migration: Add braces {} for if statement
migration: Open brace '{' following struct go on the same line
migration: Add spaces around operator
migration: Don't use '#' flag of printf format
migration: Do not use C99 // comments
migration: Drop unused VMSTATE_FLOAT64 support
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
This test invokes several shell scripts to create a random directory
tree full of submounts, and then check in the VM whether every submount
has its own ID and the structure looks as expected.
(Note that the test scripts must be non-executable, so Avocado will not
try to execute them as if they were tests on their own, too.)
Because at this commit's date it is unlikely that the Linux kernel on
the image provided by boot_linux.py supports submounts in virtio-fs, the
test will be cancelled if no custom Linux binary is provided through the
vmlinuz parameter. (The on-image kernel can be used by providing an
empty string via vmlinuz=.)
So, invoking the test can be done as follows:
$ avocado run \
tests/acceptance/virtiofs_submounts.py \
-p vmlinuz=/path/to/linux/build/arch/x86/boot/bzImage
This test requires root privileges (through passwordless sudo -n),
because at this point, virtiofsd requires them. (If you have a
timestamp_timeout period for sudoers (e.g. the default of 5 min), you
can provide this by executing something like "sudo true" before invoking
Avocado.)
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200909184028.262297-9-mreitz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Let download_cloudinit() take an optional pubkey, which subclasses of
BootLinux can pass through setUp().
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200909184028.262297-8-mreitz@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: WIllian Rampazzo <willianr@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Whenever we encounter a directory with an st_dev that differs from that
of its parent, we set the FUSE_ATTR_SUBMOUNT flag so the guest can
create a submount for it.
Make this behavior optional, so submounts are only announced to the
guest with the announce_submounts option. Some users may prefer the
current behavior, so that the guest learns nothing about the host mount
structure.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200909184028.262297-7-mreitz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Manual merge
We want to detect mount points in the shared tree. We report them to
the guest by setting the FUSE_ATTR_SUBMOUNT flag in fuse_attr.flags, but
because the FUSE client will create a submount for every directory that
has this flag set, we must do this only for the actual mount points.
We can detect mount points by comparing a directory's st_dev with its
parent's st_dev. To be able to do so, we need to store the parent's
st_dev in the lo_inode object.
Note that mount points need not necessarily be directories; a single
file can be a mount point as well. However, for the sake of simplicity
let us ignore any non-directory mount points for now.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200909184028.262297-6-mreitz@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
The plain fuse_reply_attr() function does not allow setting
fuse_attr.flags, so add this new function that does.
Make fuse_reply_attr() a wrapper around it.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200909184028.262297-5-mreitz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
fuse_entry_param is converted to fuse_attr on the line (by
fill_entry()), so it should have a member that mirrors fuse_attr.flags.
fill_entry() should then copy this fuse_entry_param.attr_flags to
fuse_attr.flags.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200909184028.262297-4-mreitz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
The fuse_attr.flags field is currently just initialized to 0, which is
valid. Thus, there is no reason not to always announce FUSE_ATTR_FLAGS
(when the kernel supports it).
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200909184028.262297-3-mreitz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Update the linux/fuse.h standard header from the kernel development tree
that implements FUSE submounts.
This adds the fuse_attr.flags field, the FUSE_ATTR_FLAGS INIT flag, and
the FUSE_ATTR_SUBMOUNT flag for fuse_attr.flags.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200909184028.262297-2-mreitz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
The mapping rule system implemented in the last few patches is
extremely flexible, but not easy to use. Add a simple
'map' type as a sprinkling of sugar to make it easy.
e.g.
-o xattrmap=":map::user.virtiofs.:"
would be sufficient to prefix all xattr's
or
-o xattrmap=":map:trusted.:user.virtiofs.:"
would just prefix 'trusted.' xattr's and leave
everything else alone.
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <20201023165812.36028-6-dgilbert@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Add a few examples of xattrmaps to the documentation.
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20201023165812.36028-5-dgilbert@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Map xattr names coming from the server, i.e. the host filesystem;
currently this is only from listxattr.
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <20201023165812.36028-4-dgilbert@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Map xattr names originating at the client; from get/set/remove xattr.
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <20201023165812.36028-3-dgilbert@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Add an option to define mappings of xattr names so that
the client and server filesystems see different views.
This can be used to have different SELinux mappings as
seen by the guest, to run the virtiofsd with less privileges
(e.g. in a case where it can't set trusted/system/security
xattrs but you want the guest to be able to), or to isolate
multiple users of the same name; e.g. trusted attributes
used by stacking overlayfs.
A mapping engine is used with 3 simple rules; the rules can
be combined to allow most useful mapping scenarios.
The ruleset is defined by -o xattrmap='rules...'.
This patch doesn't use the rule maps yet.
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <20201023165812.36028-2-dgilbert@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
virtiofsd cannot run in a container because CAP_SYS_ADMIN is required to
create namespaces.
Introduce a weaker sandbox mode that is sufficient in container
environments because the container runtime already sets up namespaces.
Use chroot to restrict path traversal to the shared directory.
virtiofsd loses the following:
1. Mount namespace. The process chroots to the shared directory but
leaves the mounts in place. Seccomp rejects mount(2)/umount(2)
syscalls.
2. Pid namespace. This should be fine because virtiofsd is the only
process running in the container.
3. Network namespace. This should be fine because seccomp already
rejects the connect(2) syscall, but an additional layer of security
is lost. Container runtime-specific network security policies can be
used drop network traffic (except for the vhost-user UNIX domain
socket).
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20201008085534.16070-1-stefanha@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Just noticed that although help message says default log level is INFO,
it is actually 0 (EMRGE) and no mesage will be shown when error occurs.
It's better to follow help message.
Signed-off-by: Misono Tomohiro <misono.tomohiro@jp.fujitsu.com>
Message-Id: <20201008110148.2757734-1-misono.tomohiro@jp.fujitsu.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Currently it is unknown whether virtiofsd will be built at
configuration time. It will be automatically built when dependency
is met. Also, required libraries are not clear.
To make this clear, add configure option --{enable,disable}-virtiofsd.
The default is the same as current (enabled if available) like many
other options. When --enable-virtiofsd is given and dependency is not
met, we get:
ERROR: Problem encountered: virtiofsd requires libcap-ng-devel and seccomp-devel
or
ERROR: Problem encountered: virtiofsd needs tools and vhost-user support
In addition, configuration summary now includes virtiofsd entry:
build virtiofs daemon: YES/NO
Signed-off-by: Misono Tomohiro <misono.tomohiro@jp.fujitsu.com>
Message-Id: <20201008103133.2722903-1-misono.tomohiro@jp.fujitsu.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Manual merge
Avocado documentation referred returns 404 error.
Update the broken links.
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20201010080741.2932406-1-philmd@redhat.com>
Unfortunately the Armbian 19.11.3 image has been removed from the
dl.armbian.com file server. Developers having the artifact cached
can still run the test. Allow them to, until we find a proper
solution to share binaries with the whole community.
This avoids (when file manually added to cache):
BootLinuxConsole.test_arm_orangepi_bionic_19_11: CANCEL: Missing asset https://dl.armbian.com/orangepipc/archive/Armbian_19.11.3_Orangepipc_bionic_current_5.3.9.7z (1.06 s)
Reported-by: Bin Meng <bmeng.cn@gmail.com>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Bin Meng <bin.meng@windriver.com>
Tested-by: Niek Linnenbank <nieklinnenbank@gmail.com>
Reviewed-by: Niek Linnenbank <nieklinnenbank@gmail.com>
Message-Id: <20201023131808.3198005-3-f4bug@amsat.org>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
The current 'virt_kvm' test is restricted to GICv2, but can also
work with a GICv3. Duplicate it but add a GICv3 test which can be
tested on some hardware.
Noticed while running:
$ avocado --show=app run -t machine:virt tests/acceptance/
...
(2/6) tests/acceptance/boot_linux.py:BootLinuxAarch64.test_virt_kvm: ERROR: Unexpected empty reply from server (1.82 s)
The job.log content is:
L0351 DEBUG| Output: 'qemu-system-aarch64: host does not support in-kernel GICv2 emulation\n'
With this patch:
$ avocado --show=app run -t device:gicv3 tests/acceptance/
(1/1) tests/acceptance/boot_linux.py:BootLinuxAarch64.test_virt_kvm_gicv3: PASS (55.10 s)
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20200929224857.1225107-1-philmd@redhat.com>
This test runs Trusted Firmware-A on the Raspberry Pi 3.
We deliberately stop the boot process when the EDK2 UEFI version
is displayed.
The binary is build on AppVeyor CI using Pete Batard repository [1].
ATF v2.1 binary are used (see [2]).
It is very simple and fast:
$ avocado --show=app,console run -t atf tests/acceptance
JOB ID : 1e748d7c9e9011cf0af3250ddc8ebf2389d6204e
JOB LOG : avocado/job-results/job-2020-02-16T18.08-1e748d7/job.log
(1/1) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_aarch64_raspi3_atf:
console: NOTICE: Booting Trusted Firmware
console: NOTICE: BL1: v2.1(release):v2.1
console: NOTICE: BL1: Built : 15:26:06, May 13 2019
console: NOTICE: rpi3: Detected: Raspberry Pi 3 Model B (1GB, Sony, UK) [0x00a02082]
console: NOTICE: BL1: Booting BL2
console: ERROR: rpi3_sdhost: timeout status 0x40
console: NOTICE: BL2: v2.1(release):v2.1
console: NOTICE: BL2: Built : 15:26:01, May 13 2019
console: NOTICE: BL1: Booting BL31
console: NOTICE: BL31: v2.1(release):v2.1
console: NOTICE: BL31: Built : 15:26:04, May 13 2019
console: =UEFI firmware (version UEFI Firmware v1.15 built at 11:58:44 on Feb 14 2020)
PASS (1.54 s)
RESULTS : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB TIME : 1.88 s
[1] https://github.com/pbatard/RPi3#summary
[2] https://github.com/ARM-software/arm-trusted-firmware/blob/v2.1/docs/plat/rpi3.rst
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Luc Michel <luc.michel@greensocs.com>
Message-Id: <20200217103442.30318-1-f4bug@amsat.org>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Commit a81df1b68b ("libqemuutil, qapi, trace: convert to meson")
removed it without explanation and it is useful to be able to run a
script without having to figure out which interpreter to use.
Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20200923103620.1980151-1-anthony.perard@citrix.com>
Tracing can be enabled at the command line or via the
monitor. Command-line trace options are recorded during
trace_opt_parse(), but tracing is not enabled until the various
front-ends later call trace_init_file(). If the user passes a trace
option on the command-line, remember that and enable tracing during
trace_init_file(). Otherwise, trace_init_file() should record the
trace file specified by the frontend and avoid enabling traces
until the user requests them via the monitor.
This fixes 1b7157be3a and also
db25d56c01, by allowing the user
to enable traces on the command line and also avoiding
unwanted trace-<pid> files when the user has not asked for them.
Fixes: 1b7157be3a
Signed-off-by: Josh DuBois <josh@joshdubois.com>
Message-id: 20200816174610.20253-1-josh@joshdubois.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
The errors are very useful when debugging qtest failures, especially when
QTEST_LOG=1 is set. Let's allow override MigrateStart.hide_stderr when
QTEST_LOG=1 is specified, because that means the user wants to be verbose.
Not very nice to introduce the first QTEST_LOG env access in migration-test.c,
however it should be handy. Without this patch, I was hacking error_report()
when debugging such errors. Let's make things easier.
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20201021212721.440373-7-peterx@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Logically below race could trigger with the old code:
test program migration thread
------------ ----------------
wait_until('postcopy-pause')
postcopy_pause()
set_state('postcopy-pause')
do_postcopy_recover()
arm s->to_dst_file with new fd
release s->to_dst_file [1]
Here [1] could have released the just-installed recoverying channel. Then the
migration could hang without really resuming.
Instead, it should be very safe to release the fd before setting the state into
'postcopy-pause', because there's no reason for any other thread to touch it
during 'postcopy-active'.
Dave reported a very rare postcopy recovery hang that the migration-test
program waited for the migration to complete in migrate_postcopy_complete().
We do suspect it's the same thing that we're gonna fix here. Hard to tell.
However since we've noticed this, fix this irrelevant of the hang report.
Cc: Dr. David Alan Gilbert <dgilbert@redhat.com>
Cc: Juan Quintela <quintela@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20201021212721.440373-6-peterx@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
We synchronize the requested pages right after a postcopy recovery happens.
This helps to synchronize the prioritized pages on source so that the faulted
threads can be served faster.
Reported-by: Xiaohui Li <xiaohli@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20201021212721.440373-5-peterx@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Maintain a list of faulted addresses on the destination host for which we're
waiting on. This is implemented using a GTree rather than a real list to make
sure even there're plenty of vCPUs/threads that are faulting, the lookup will
still be fast with O(log(N)) (because we'll do that after placing each page).
It should bring a slight overhead, but ideally that shouldn't be a big problem
simply because in most cases the requested page list will be short.
Actually we did similar things for postcopy blocktime measurements. This patch
didn't use that simply because:
(1) blocktime measurement is towards vcpu threads only, but here we need to
record all faulted addresses, including main thread and external
thread (like, DPDK via vhost-user).
(2) blocktime measurement will require UFFD_FEATURE_THREAD_ID, but here we
don't want to add that extra dependency on the kernel version since not
necessary. E.g., we don't need to know which thread faulted on which
page, we also don't care about multiple threads faulting on the same
page. But we only care about what addresses are faulted so waiting for a
page copying from src.
(3) blocktime measurement is not enabled by default. However we need this by
default especially for postcopy recover.
Another thing to mention is that this patch introduced a new mutex to serialize
the receivedmap and the page_requested tree, however that serialization does
not cover other procedures like UFFDIO_COPY.
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20201021212721.440373-4-peterx@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
This is another layer wrapper for sending a page request to the source VM. The
new migrate_send_rp_message_req_pages() will be used elsewhere in coming
patches.
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20201021212721.440373-3-peterx@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
It'll be used in follow up patches to access more fields out of it. Meanwhile
fetch the userfaultfd inside the function.
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20201021212721.440373-2-peterx@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Bihong Yu <yubihong@huawei.com>
Message-Id: <1603179176-5360-1-git-send-email-yubihong@huawei.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Bihong Yu <yubihong@huawei.com>
Reviewed-by: Chuan Zheng <zhengchuan@huawei.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <1603163448-27122-9-git-send-email-yubihong@huawei.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Bihong Yu <yubihong@huawei.com>
Reviewed-by: Chuan Zheng <zhengchuan@huawei.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <1603163448-27122-8-git-send-email-yubihong@huawei.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Bihong Yu <yubihong@huawei.com>
Reviewed-by: Chuan Zheng <zhengchuan@huawei.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <1603163448-27122-7-git-send-email-yubihong@huawei.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Bihong Yu <yubihong@huawei.com>
Reviewed-by: Chuan Zheng <zhengchuan@huawei.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <1603163448-27122-6-git-send-email-yubihong@huawei.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Bihong Yu <yubihong@huawei.com>
Reviewed-by: Chuan Zheng <zhengchuan@huawei.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <1603163448-27122-5-git-send-email-yubihong@huawei.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Bihong Yu <yubihong@huawei.com>
Reviewed-by: Chuan Zheng <zhengchuan@huawei.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <1603163448-27122-4-git-send-email-yubihong@huawei.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Bihong Yu <yubihong@huawei.com>
Reviewed-by: Chuan Zheng <zhengchuan@huawei.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Message-Id: <1603163448-27122-3-git-send-email-yubihong@huawei.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Bihong Yu <yubihong@huawei.com>
Reviewed-by: Chuan Zheng <zhengchuan@huawei.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Message-Id: <1603163448-27122-2-git-send-email-yubihong@huawei.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Commit ef96e3ae96 in January 2019 removed the last user of the
VMSTATE_FLOAT64* macros. These were used by targets which defined
their floating point register file as an array of 'float64'.
We used to try to maintain a stricter distinction between
'float64' (a type for holding an integer representing an IEEE float)
and 'uint64_t', including having a debug option for 'float64' being
a struct and supposedly mandatory macros for converting between
float64 and uint64_t. We no longer think that's a usefully
strong distinction to draw and we allow ourselves to freely
assume that float64 really is just a 64-bit integer type, so
for new targets we would simply recommend use of the uint64_t type
for a floating point register file. The float64 type remains
as a useful way of documenting in the type signature of helper
functions and the like that they expect to receive an IEEE float
from the TCG generated code rather than an arbitrary integer.
Since the VMSTATE_FLOAT64* macros have no remaining users and
we don't recommend new code uses them, delete them.
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Message-Id: <20201022120830.5938-1-peter.maydell@linaro.org>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
* Fix for Xen dummy cpu loop (which happened due to qtest accel rework)
* Introduction of the generic device fuzzer
* Run more check-acceptance tests in the gitlab-CI
-----BEGIN PGP SIGNATURE-----
iQJFBAABCAAvFiEEJ7iIR+7gJQEY8+q5LtnXdP5wLbUFAl+WmAwRHHRodXRoQHJl
ZGhhdC5jb20ACgkQLtnXdP5wLbUGDA/+Ntog6jDcudxZwrPjB5GUiAyYOHo6MClc
SdBw/abcs7G/EeGvIEexFLkuHTzroynbiE1l2RIwb5s8nB97mpXZYhT+cQ3qGUwg
Uh4TAI0YM9GiEvhIOr9DdDj2XPF5yZ5YIzDSBk5lb5KHcgjrwj5P6EJs0k4RvX36
5yJJeHPi0dDnfcfdowQ0zh7G/4drVLTS78gR+UKIYlbrZnMY2KK2tXwIi0ZE/QUK
e4vnZAbAHFg6p7D/nw+DjDYUGr1ihFtiLLKueBFkpgGkMwFUYLkL548mwiL/ZFdT
0ECiSVJUOFrT0nAbhUzrzBXEKXl6VpP66Dz62wRf8O+RRHihlWotij50ExtHUT+m
6J1G3ycky4WQn4ORaKXG+MhkneiXiovoDEeFxBL7kX8VcTligdZzzjU6nOcE0LC3
HOyCkocKy5u9sXi4rHIS+ui4FLpxZAf+pvYGdYVpekkFZvefeWYm+xUEIFWbTjfb
vH7SeycAWG8KDhwm+DBhY/CC15sC4QGLndS7Kx4l7u9frZhacN48KnC2rfooku/q
9+N/dSpo6Vj0jcq//ZQ9BIV/TkKtptpSKTm2Bd6kaidUpXrcst8CscYkBz7UUnjY
7BcLk37i2yZPnwkGzAhqwgRlfFatDYW7jaWG7R0A7hEp2U3ujvOuxCCls6jcjW2I
PDpsAlt6VnM=
=ZpMy
-----END PGP SIGNATURE-----
Merge remote-tracking branch 'remotes/huth-gitlab/tags/pull-request-2020-10-26' into staging
* qtest fixes (e.g. memory leaks)
* Fix for Xen dummy cpu loop (which happened due to qtest accel rework)
* Introduction of the generic device fuzzer
* Run more check-acceptance tests in the gitlab-CI
# gpg: Signature made Mon 26 Oct 2020 09:34:04 GMT
# gpg: using RSA key 27B88847EEE0250118F3EAB92ED9D774FE702DB5
# gpg: issuer "thuth@redhat.com"
# gpg: Good signature from "Thomas Huth <th.huth@gmx.de>" [full]
# gpg: aka "Thomas Huth <thuth@redhat.com>" [full]
# gpg: aka "Thomas Huth <huth@tuxfamily.org>" [full]
# gpg: aka "Thomas Huth <th.huth@posteo.de>" [unknown]
# Primary key fingerprint: 27B8 8847 EEE0 2501 18F3 EAB9 2ED9 D774 FE70 2DB5
* remotes/huth-gitlab/tags/pull-request-2020-10-26: (31 commits)
tests/acceptance: Use .ppm extention for Portable PixMap files
tests/acceptance: Remove unused import
test/docker/dockerfiles: Add missing packages for acceptance tests
tests/acceptance: Enable AVOCADO_ALLOW_UNTRUSTED_CODE in the gitlab-CI
test/acceptance: Remove the CONTINUOUS_INTEGRATION tags
tests/acceptance/ppc_prep_40p: Fix the URL to the NetBSD-4.0 archive
scripts/oss-fuzz: ignore the generic-fuzz target
scripts/oss-fuzz: use hardlinks instead of copying
fuzz: register predefined generic-fuzz configs
fuzz: add generic-fuzz configs for oss-fuzz
fuzz: add an "opaque" to the FuzzTarget struct
fuzz: Add instructions for using generic-fuzz
scripts/oss-fuzz: Add crash trace minimization script
scripts/oss-fuzz: Add script to reorder a generic-fuzzer trace
fuzz: add a crossover function to generic-fuzzer
fuzz: add a DISABLE_PCI op to generic-fuzzer
fuzz: Add support for custom crossover functions
fuzz: Add fuzzer callbacks to DMA-read functions
fuzz: Declare DMA Read callback function
fuzz: Add DMA support to the generic-fuzzer
...
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
A default value is provided for the variable 'bitmap_name' to avoid
a compiler warning.
The compiler showed the warning:
migration/block-dirty-bitmap.c:1090:13: warning: ‘bitmap_name’
may be used uninitialized in this function [-Wmaybe-uninitialized]
g_strlcpy(s->bitmap_name, bitmap_name, sizeof(s->bitmap_name));
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Reported-by: Euler Robot <euler.robot@huawei.com>
Signed-off-by: Chen Qun <kuhn.chenqun@huawei.com>
Message-Id: <20201014114430.1898684-1-kuhn.chenqun@huawei.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
[eblake: commit message grammar tweaks]
Signed-off-by: Eric Blake <eblake@redhat.com>
sectors_per_chunk is a 64 bit integer, but the calculation is done in 32
bits, leading to an overflow for coarse bitmap granularities.
If that results in the value 0, it leads to a hang where no progress is
made but send_bitmap_bits is constantly called with nr_sectors being 0.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Message-Id: <20201021144456.1072-1-s.reiter@proxmox.com>
Fixes: b35ebdf07 migration: add postcopy migration of dirty bitmaps
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[eblake: Use correct type for 8ULL, use () to avoid overflow]
Signed-off-by: Eric Blake <eblake@redhat.com>
