As the previous commit changes the logic of "qemu-img rebase" (it's using
write alignment now), let's add a couple more test cases which would
ensure it works correctly. In particular, the following scenarios:
024: add test case for rebase within one backing chain when the overlay
cluster size > backings cluster size;
271: add test case for rebase images that contain subclusters. Check
that no extra allocations are being made.
Signed-off-by: Andrey Drobyshev <andrey.drobyshev@virtuozzo.com>
Reviewed-by: Hanna Czenczek <hreitz@redhat.com>
Message-ID: <20230919165804.439110-7-andrey.drobyshev@virtuozzo.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
When rebasing an image from one backing file to another, we need to
compare data from old and new backings. If the diff between that data
happens to be unaligned to the target cluster size, we might end up
doing partial writes, which would lead to copy-on-write and additional IO.
Consider the following simple case (virtual_size == cluster_size == 64K):
base <-- inc1 <-- inc2
qemu-io -c "write -P 0xaa 0 32K" base.qcow2
qemu-io -c "write -P 0xcc 32K 32K" base.qcow2
qemu-io -c "write -P 0xbb 0 32K" inc1.qcow2
qemu-io -c "write -P 0xcc 32K 32K" inc1.qcow2
qemu-img rebase -f qcow2 -b base.qcow2 -F qcow2 inc2.qcow2
While doing rebase, we'll write a half of the cluster to inc2, and block
layer will have to read the 2nd half of the same cluster from the base image
inc1 while doing this write operation, although the whole cluster is already
read earlier to perform data comparison.
In order to avoid these unnecessary IO cycles, let's make sure every
write request is aligned to the overlay subcluster boundaries. Using
subcluster size is universal as for the images which don't have them
this size equals to the cluster size. so in any case we end up aligning
to the smallest unit of allocation.
Signed-off-by: Andrey Drobyshev <andrey.drobyshev@virtuozzo.com>
Message-ID: <20230919165804.439110-6-andrey.drobyshev@virtuozzo.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Add @chsize param to the function which, if non-zero, would represent
the chunk size to be used for comparison. If it's zero, then
BDRV_SECTOR_SIZE is used as default chunk size, which is the previous
behaviour.
In particular, we're going to use this param in img_rebase() to make the
write requests aligned to a predefined alignment value.
Signed-off-by: Andrey Drobyshev <andrey.drobyshev@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Hanna Czenczek <hreitz@redhat.com>
Message-ID: <20230919165804.439110-5-andrey.drobyshev@virtuozzo.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Since commit bb1c05973c ("qemu-img: Use qemu_blockalign"), buffers for
the data read from the old and new backing files are aligned using
BlockDriverState (or BlockBackend later on) referring to the target image.
However, this isn't quite right, because buf_new is only being used for
reading from the new backing, while buf_old is being used for both reading
from the old backing and writing to the target. Let's take that into account
and use more appropriate values as alignments.
Signed-off-by: Andrey Drobyshev <andrey.drobyshev@virtuozzo.com>
Message-ID: <20230919165804.439110-4-andrey.drobyshev@virtuozzo.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Before previous commit, rebase was getting infitely stuck in case of
rebasing within the same backing chain and when overlay_size > backing_size.
Let's add this case to the rebasing test 024 to make sure it doesn't
break again.
Signed-off-by: Andrey Drobyshev <andrey.drobyshev@virtuozzo.com>
Reviewed-by: Denis V. Lunev <den@openvz.org>
Reviewed-by: Hanna Czenczek <hreitz@redhat.com>
Message-ID: <20230919165804.439110-3-andrey.drobyshev@virtuozzo.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
In case when we're rebasing within one backing chain, and when target image
is larger than old backing file, bdrv_is_allocated_above() ends up setting
*pnum = 0. As a result, target offset isn't getting incremented, and we
get stuck in an infinite for loop. Let's detect this case and proceed
further down the loop body, as the offsets beyond the old backing size need
to be explicitly zeroed.
Signed-off-by: Andrey Drobyshev <andrey.drobyshev@virtuozzo.com>
Reviewed-by: Denis V. Lunev <den@openvz.org>
Reviewed-by: Hanna Czenczek <hreitz@redhat.com>
Message-ID: <20230919165804.439110-2-andrey.drobyshev@virtuozzo.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
This let us simplify code of this shape.
qemu_fflush(f);
int ret = qemu_file_get_error(f);
if (ret) {
return ret;
}
into:
int ret = qemu_fflush(f);
if (ret) {
return ret;
}
I updated all callers where there is any error check.
qemu_fclose() don't need to check for f->last_error because
qemu_fflush() returns it at the beggining of the function.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Fabiano Rosas <farosas@suse.de>
Signed-off-by: Juan Quintela <quintela@redhat.com>
Message-ID: <20231025091117.6342-13-quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
After last commit, it is a write only variable.
Reviewed-by: Fabiano Rosas <farosas@suse.de>
Signed-off-by: Juan Quintela <quintela@redhat.com>
Message-ID: <20231025091117.6342-12-quintela@redhat.com>
There are only two differnces with the old value:
- the amount of QEMUFile that hasn't yet been flushed. It can be
discussed what is more exact, the new or the old one.
- the amount of transferred bytes that we forgot to account for (the
newer is better, i.e. exact).
Notice that this two values are used to:
a - present to the user
b - calculate the rate_limit
So a few KB here and there is not going to make a difference.
Reviewed-by: Fabiano Rosas <farosas@suse.de>
Signed-off-by: Juan Quintela <quintela@redhat.com>
Message-ID: <20231025091117.6342-11-quintela@redhat.com>
If we pass a NULL error is the same that returning directly the value.
Reviewed-by: Fabiano Rosas <farosas@suse.de>
Signed-off-by: Juan Quintela <quintela@redhat.com>
Message-ID: <20231025091117.6342-10-quintela@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Juan Quintela <quintela@redhat.com>
Message-ID: <20231025091117.6342-9-quintela@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Juan Quintela <quintela@redhat.com>
Message-ID: <20231025091117.6342-8-quintela@redhat.com>
qemu_file_transferred() don't exist anymore, so we can reuse the name.
Reviewed-by: Fabiano Rosas <farosas@suse.de>
Signed-off-by: Juan Quintela <quintela@redhat.com>
Message-ID: <20231025091117.6342-7-quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
We only use migration_transferred_bytes() to calculate the rate_limit,
for that we don't need to flush whatever is on the qemu_file buffer.
Remember that the buffer is really small (normal case is 32K if we use
iov's can be 64 * TARGET_PAGE_SIZE), so this is not relevant to
calculations.
Reviewed-by: Fabiano Rosas <farosas@suse.de>
Signed-off-by: Juan Quintela <quintela@redhat.com>
Message-ID: <20231025091117.6342-5-quintela@redhat.com>
This way we can read it from any thread.
I checked that it gives the same value as the current one. We never
use two qemu_files at the same time.
Reviewed-by: Fabiano Rosas <farosas@suse.de>
Signed-off-by: Juan Quintela <quintela@redhat.com>
Message-ID: <20231025091117.6342-3-quintela@redhat.com>
We only call qemu_file_transferred_* on the sending side. Remove the
increment at qemu_file_fill_buffer() and add asserts to
qemu_file_transferred* functions.
Reviewed-by: Fabiano Rosas <farosas@suse.de>
Signed-off-by: Juan Quintela <quintela@redhat.com>
Message-ID: <20231025091117.6342-2-quintela@redhat.com>
In multiple places, RDMA errors are handled in a strange way, where it only
sets qemu_file_set_error() but not stop the migration immediately.
It's not obvious what will happen later if there is already an error. Make
all such failures stop migration immediately.
Cc: Zhijian Li (Fujitsu) <lizhijian@fujitsu.com>
Cc: Markus Armbruster <armbru@redhat.com>
Cc: Juan Quintela <quintela@redhat.com>
Cc: Fabiano Rosas <farosas@suse.de>
Reported-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Fabiano Rosas <farosas@suse.de>
Signed-off-by: Juan Quintela <quintela@redhat.com>
Message-ID: <20231024163933.516546-1-peterx@redhat.com>
Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
Acked-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
Message-ID: <20231018115513.2163-6-quintela@redhat.com>
It is obsolete. It is better to use driver-mirror with NBD instead.
CC: Kevin Wolf <kwolf@redhat.com>
CC: Eric Blake <eblake@redhat.com>
CC: Stefan Hajnoczi <stefanha@redhat.com>
CC: Hanna Czenczek <hreitz@redhat.com>
Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
Message-ID: <20231018115513.2163-5-quintela@redhat.com>
Use blocked-mirror with NBD instead.
Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
Message-ID: <20231018115513.2163-4-quintela@redhat.com>
Use blockdev-mirror with NBD instead.
Reviewed-by: Thomas Huth <thuth@redhat.com>
Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
Message-ID: <20231018115513.2163-3-quintela@redhat.com>
Create a new filter that removes the two warnings for test 183.
Reviewed-by: Hanna Czenczek <hreitz@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
Message-ID: <20231018115513.2163-2-quintela@redhat.com>
* Correct minor errors in Cortex-A710 definition
* Implement Neoverse N2 CPU model
* Refactor feature test functions out into separate header
* Fix syndrome for FGT traps on ERET
* Remove 'hw/arm/boot.h' includes from various header files
* pxa2xx: Refactoring/cleanup
* Avoid using 'first_cpu' when first ARM CPU is reachable
* misc/led: LED state is set opposite of what is expected
* hw/net/cadence_gen: clean up to use FIELD macros
* hw/net/cadence_gem: perform PHY access on write only
* hw/net/cadence_gem: enforce 32 bits variable size for CRC
-----BEGIN PGP SIGNATURE-----
iQJNBAABCAA3FiEE4aXFk81BneKOgxXPPCUl7RQ2DN4FAmU7yz0ZHHBldGVyLm1h
eWRlbGxAbGluYXJvLm9yZwAKCRA8JSXtFDYM3n4xEACK4ti+PFSJHVCQ69NzLLBT
ybFGFMsMhXJTSNS30Pzs+KWCKWPP59knYBD4qO43W1iV6pPUhy+skr+BFCCRvBow
se74+Fm1l4LmnuHxgukJzTdvRffI3v37alLn6Y/ioWe8bDpf/IJj8WLj8B1IPoNg
fswJSGDLpPMovaz8NBQRzglUWpfyzxH+uuW779qBS1nuFdPOfIHKrocvvdrfogBP
aO8AeiBzz5STW9Naeq+BIKho8S9LinSB6FHa+rRPUDkWx03lvRIvkgGPzHpXYy8I
zAZ8gUQZyXprHAHMpnoBv8Wcw3Bwc2f+8xx8hnRRki3iBroXKfJA9NkeN0StQmL1
ZHhfYkiKSS5diIFW5pX6ZixKbXHE2a4aH4zPVUNQriNWOevhe7n82mAPNFIYjk97
ciTtd4I2oew48sDLSodMiirGL987Mit7KC23itVGezcNfQ9FnVTDmuGy8Rq52BZm
u4TZjVBrtjQOdMBUcD2hKvXhikQNAdOhArPwNfOr0esSQL44MMEe+6Q5/Cbp0BOE
stAY/xwSP2cY5mIPnAbIBELseEZsV8ySA3M0y1iRCJptjwbyWM+s1TYz0iXcqeOn
l6LfiI6r1BqUeoWLGP4042R4FLyLNh6gU/TiFNLu7JJQjXl/EkRgqVXWYfzy2n51
KKY6iGFi5r41sAU6GIXOkQ==
=szC7
-----END PGP SIGNATURE-----
Merge tag 'pull-target-arm-20231027' of https://git-us.linaro.org/people/pmaydell/qemu-arm into staging
target-arm queue:
* Correct minor errors in Cortex-A710 definition
* Implement Neoverse N2 CPU model
* Refactor feature test functions out into separate header
* Fix syndrome for FGT traps on ERET
* Remove 'hw/arm/boot.h' includes from various header files
* pxa2xx: Refactoring/cleanup
* Avoid using 'first_cpu' when first ARM CPU is reachable
* misc/led: LED state is set opposite of what is expected
* hw/net/cadence_gen: clean up to use FIELD macros
* hw/net/cadence_gem: perform PHY access on write only
* hw/net/cadence_gem: enforce 32 bits variable size for CRC
# -----BEGIN PGP SIGNATURE-----
#
# iQJNBAABCAA3FiEE4aXFk81BneKOgxXPPCUl7RQ2DN4FAmU7yz0ZHHBldGVyLm1h
# eWRlbGxAbGluYXJvLm9yZwAKCRA8JSXtFDYM3n4xEACK4ti+PFSJHVCQ69NzLLBT
# ybFGFMsMhXJTSNS30Pzs+KWCKWPP59knYBD4qO43W1iV6pPUhy+skr+BFCCRvBow
# se74+Fm1l4LmnuHxgukJzTdvRffI3v37alLn6Y/ioWe8bDpf/IJj8WLj8B1IPoNg
# fswJSGDLpPMovaz8NBQRzglUWpfyzxH+uuW779qBS1nuFdPOfIHKrocvvdrfogBP
# aO8AeiBzz5STW9Naeq+BIKho8S9LinSB6FHa+rRPUDkWx03lvRIvkgGPzHpXYy8I
# zAZ8gUQZyXprHAHMpnoBv8Wcw3Bwc2f+8xx8hnRRki3iBroXKfJA9NkeN0StQmL1
# ZHhfYkiKSS5diIFW5pX6ZixKbXHE2a4aH4zPVUNQriNWOevhe7n82mAPNFIYjk97
# ciTtd4I2oew48sDLSodMiirGL987Mit7KC23itVGezcNfQ9FnVTDmuGy8Rq52BZm
# u4TZjVBrtjQOdMBUcD2hKvXhikQNAdOhArPwNfOr0esSQL44MMEe+6Q5/Cbp0BOE
# stAY/xwSP2cY5mIPnAbIBELseEZsV8ySA3M0y1iRCJptjwbyWM+s1TYz0iXcqeOn
# l6LfiI6r1BqUeoWLGP4042R4FLyLNh6gU/TiFNLu7JJQjXl/EkRgqVXWYfzy2n51
# KKY6iGFi5r41sAU6GIXOkQ==
# =szC7
# -----END PGP SIGNATURE-----
# gpg: Signature made Fri 27 Oct 2023 23:37:49 JST
# gpg: using RSA key E1A5C593CD419DE28E8315CF3C2525ED14360CDE
# gpg: issuer "peter.maydell@linaro.org"
# gpg: Good signature from "Peter Maydell <peter.maydell@linaro.org>" [full]
# gpg: aka "Peter Maydell <pmaydell@gmail.com>" [full]
# gpg: aka "Peter Maydell <pmaydell@chiark.greenend.org.uk>" [full]
# gpg: aka "Peter Maydell <peter@archaic.org.uk>" [unknown]
# Primary key fingerprint: E1A5 C593 CD41 9DE2 8E83 15CF 3C25 25ED 1436 0CDE
* tag 'pull-target-arm-20231027' of https://git-us.linaro.org/people/pmaydell/qemu-arm: (41 commits)
hw/net/cadence_gem: enforce 32 bits variable size for CRC
hw/net/cadence_gem: perform PHY access on write only
hw/net/cadence_gem: use FIELD to describe PHYMNTNC register fields
hw/net/cadence_gem: use FIELD to describe DESCONF6 register fields
hw/net/cadence_gem: use FIELD to describe IRQ register fields
hw/net/cadence_gem: use FIELD to describe [TX|RX]STATUS register fields
hw/net/cadence_gem: use FIELD to describe DMACFG register fields
hw/net/cadence_gem: use FIELD to describe NWCFG register fields
hw/net/cadence_gem: use FIELD to describe NWCTRL register fields
hw/net/cadence_gem: use FIELD for screening registers
hw/net/cadence_gem: use REG32 macro for register definitions
misc/led: LED state is set opposite of what is expected
hw/arm: Avoid using 'first_cpu' when first ARM CPU is reachable
hw/arm/pxa2xx: Realize PXA2XX_I2C device before accessing it
hw/intc/pxa2xx: Factor pxa2xx_pic_realize() out of pxa2xx_pic_init()
hw/intc/pxa2xx: Pass CPU reference using QOM link property
hw/intc/pxa2xx: Convert to Resettable interface
hw/pcmcia/pxa2xx: Inline pxa2xx_pcmcia_init()
hw/pcmcia/pxa2xx: Do not open-code sysbus_create_simple()
hw/pcmcia/pxa2xx: Realize sysbus device before accessing it
...
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
This is not ideal, since it requires all cross-compilers
to be present rather than a simple subset. But since it
is only run manually, should be good enough for now.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Acked-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Add support in gen-vdso-elfn.c.inc for the DT_PPC64_OPT
dynamic tag: this is an integer, so does not need relocation.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Requires a relatively recent binutils version in order to avoid
spurious R_LARCH_NONE relocations. The presence of these relocs
are diagnosed by our gen-vdso tool.
Tested-by: Song Gao <gaosong@loongson.cn>
Reviewed-by: Song Gao <gaosong@loongson.cn>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Acked-by: Alistair Francis <alistair.francis@wdc.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
This tool will be used for post-processing the linked vdso image,
turning it into something that is easy to include into elfload.c.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
The vdso image will be pre-processed into a C data array, with
a simple list of relocations to perform, and identifying the
location of signal trampolines.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
There are only a couple of uses of bprm->fd remaining.
Migrate to the other field.
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Aside from the section headers, we're unlikely to hit the
ImageSource cache on guest executables. But the interface
for imgsrc_read_* is better.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Change parse_elf_properties as well, as the bprm_buf argument
ties the two functions closely.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Rearrange the allocation of storage for ehdr between load_elf_image
and load_elf_binary. The same set of copies are done, but we don't
modify bprm_buf, which will be important later.
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reorg the if cases to reduce indentation.
Test for 4 bytes in the file before checking the signatures.
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Introduced and initialized, but not yet really used.
These will tidy the current tests vs BPRM_BUF_SIZE.
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
The arithmetic within the loop was not adjusted properly after SIGRTMIN
was stolen for the guest SIGABRT. The effect was that the guest libc
could not send itself __SIGRTMIN to wake sleeping threads.
Fixes: 38ee0a7dfb ("linux-user: Remap guest SIGABRT")
Resolves: https://gitlab.com/qemu-project/qemu/-/issues/1967
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
commit 13cde50889 ("vmstate: Return error in case of error") sets
QemuFile error to stop reading from it and report to the caller (checked
by unit tests). We should do the same on subsection loading error.
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
Message-ID: <20231024084043.2926316-8-marcandre.lureau@redhat.com>
The function is used on save at this point. The following commits will
use it on load.
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
Message-ID: <20231024084043.2926316-5-marcandre.lureau@redhat.com>
