mirrors/qemu
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
ac4ccac740
qemu/block
History
Jean-Louis Dupond 42a2890a76 qcow2: add discard-no-unref option 
When we for example have a sparse qcow2 image and discard: unmap is enabled,
there can be a lot of fragmentation in the image after some time. Especially on VM's
that do a lot of writes/deletes.
This causes the qcow2 image to grow even over 110% of its virtual size,
because the free gaps in the image get too small to allocate new
continuous clusters. So it allocates new space at the end of the image.

Disabling discard is not an option, as discard is needed to keep the
incremental backup size as low as possible. Without discard, the
incremental backups would become large, as qemu thinks it's just dirty
blocks but it doesn't know the blocks are unneeded.
So we need to avoid fragmentation but also 'empty' the unneeded blocks in
the image to have a small incremental backup.

In addition, we also want to send the discards further down the stack, so
the underlying blocks are still discarded.

Therefor we introduce a new qcow2 option "discard-no-unref".
When setting this option to true, discards will no longer have the qcow2
driver relinquish cluster allocations. Other than that, the request is
handled as normal: All clusters in range are marked as zero, and, if
pass-discard-request is true, it is passed further down the stack.
The only difference is that the now-zero clusters are preallocated
instead of being unallocated.
This will avoid fragmentation on the qcow2 image.

Fixes: https://gitlab.com/qemu-project/qemu/-/issues/1621
Signed-off-by: Jean-Louis Dupond <jean-louis@dupond.be>
Message-Id: <20230605084523.34134-2-jean-louis@dupond.be>
Reviewed-by: Hanna Czenczek <hreitz@redhat.com>
Signed-off-by: Hanna Czenczek <hreitz@redhat.com>
2023-06-05 13:15:42 +02:00
..
export aio: remove aio_disable_external() API 2023-05-30 17:37:26 +02:00
monitor block/monitor: Fix crash when executing HMP commit 2023-04-25 15:11:57 +02:00
accounting.c block: add missed block_acct_setup with new block device init procedure 2022-09-30 18:42:34 +02:00
aio_task.c block/aio_task: assert max_busy_tasks is greater than 0 2021-10-05 18:56:41 +02:00
amend.c block: Mark BlockDriver callbacks for amend job GRAPH_RDLOCK 2023-05-10 14:16:54 +02:00
backup.c Block layer patches 2023-02-24 15:09:39 +00:00
blkdebug.c blkdebug: add missing coroutine_fn annotation 2023-04-25 13:17:28 +02:00
blkio.c block/blkio: use qemu_open() to support fd passing for virtio-blk 2023-06-01 11:08:21 -04:00
blklogwrites.c block: Mark bdrv_co_refresh_total_sectors() and callers GRAPH_RDLOCK 2023-02-23 19:49:33 +01:00
blkreplay.c block: Mark bdrv_co_refresh_total_sectors() and callers GRAPH_RDLOCK 2023-02-23 19:49:33 +01:00
blkverify.c block: Mark bdrv_recurse_can_replace() and callers GRAPH_RDLOCK 2023-05-10 14:16:54 +02:00
block-backend.c block: add blk_io_plug_call() API 2023-06-01 07:34:03 -04:00
block-copy.c block: Mark bdrv_co_pwrite_zeroes() and callers GRAPH_RDLOCK 2023-02-23 19:49:14 +01:00
block-gen.h block-coroutine-wrapper.py: support also basic return types 2022-12-15 16:07:43 +01:00
block-ram-registrar.c block: add BlockRAMRegistrar 2022-10-26 14:56:42 -04:00
bochs.c block: Mark public read/write functions GRAPH_RDLOCK 2023-02-23 19:49:17 +01:00
cloop.c include/block: Untangle inclusion loops 2023-01-20 07:24:28 +01:00
commit.c blockjob: Adhere to rate limit even when reentered early 2023-05-19 19:12:12 +02:00
copy-before-write.c copy-before-write: Fix open with child in iothread 2023-05-30 17:29:35 +02:00
copy-before-write.h block/copy-before-write.h: global state API + assertions 2022-03-04 18:18:25 +01:00
copy-on-read.c block: remove has_variable_length from filters 2023-04-11 16:38:56 +02:00
copy-on-read.h Clean up ill-advised or unusual header guards 2022-05-11 16:50:01 +02:00
coroutines.h nbd: Mark nbd_co_do_establish_connection() and callers GRAPH_RDLOCK 2023-05-10 14:16:53 +02:00
create.c block: Call .bdrv_co_create(_opts) unlocked 2023-05-19 19:12:12 +02:00
crypto.c block: Call .bdrv_co_create(_opts) unlocked 2023-05-19 19:12:12 +02:00
crypto.h nomaintainer: Fix Lesser GPL version number 2020-11-15 17:04:40 +01:00
curl.c aio: remove aio_disable_external() API 2023-05-30 17:37:26 +02:00
dirty-bitmap.c block: Mark bdrv_*_dirty_bitmap() and callers GRAPH_RDLOCK 2023-02-23 19:49:32 +01:00
dmg-bz2.c Include qemu-common.h exactly where needed 2019-06-12 13:20:20 +02:00
dmg-lzfse.c block/dmg: Ignore C99 prototype declaration mismatch from <lzfse.h> 2023-03-30 15:03:36 +02:00
dmg.c block/dmg: Declare a type definition for DMG uncompress function 2023-04-24 13:53:44 -04:00
dmg.h block/dmg: Declare a type definition for DMG uncompress function 2023-04-24 13:53:44 -04:00
file-posix.c block/linux-aio: convert to blk_io_plug_call() API 2023-06-01 07:34:03 -04:00
file-win32.c thread-pool: avoid passing the pool parameter every time 2023-04-25 13:17:28 +02:00
filter-compress.c block: remove has_variable_length from filters 2023-04-11 16:38:56 +02:00
gluster.c cutils: Adjust signature of parse_uint[_full] 2023-06-02 12:27:19 -05:00
graph-lock.c graph-lock: Disable locking for now 2023-05-19 19:16:53 +02:00
io_uring.c block/io_uring: convert to blk_io_plug_call() API 2023-06-01 07:34:03 -04:00
io.c block: Collapse padded I/O vecs exceeding IOV_MAX 2023-06-05 13:11:06 +02:00
iscsi-opts.c modules: add block module annotations 2021-07-09 18:20:27 +02:00
iscsi.c aio: remove aio_disable_external() API 2023-05-30 17:37:26 +02:00
linux-aio.c block/linux-aio: convert to blk_io_plug_call() API 2023-06-01 07:34:03 -04:00
meson.build block: add blk_io_plug_call() API 2023-06-01 07:34:03 -04:00
mirror.c mirror: Hold main AioContext lock for calling bdrv_open_backing_file() 2023-05-30 17:21:23 +02:00
nbd.c nbd: Mark nbd_co_do_establish_connection() and callers GRAPH_RDLOCK 2023-05-10 14:16:53 +02:00
nfs.c cutils: Adjust signature of parse_uint[_full] 2023-06-02 12:27:19 -05:00
null.c block: Convert bdrv_get_allocated_file_size() to co_wrapper 2023-02-01 16:52:32 +01:00
nvme.c block/nvme: convert to blk_io_plug_call() API 2023-06-01 07:34:03 -04:00
parallels-ext.c include/block: Untangle inclusion loops 2023-01-20 07:24:28 +01:00
parallels.c parallels: Incorrect condition in out-of-image check 2023-06-05 13:13:59 +02:00
parallels.h parallels: support bitmap extension for read-only mode 2021-03-08 14:56:55 +01:00
plug.c block: add blk_io_plug_call() API 2023-06-01 07:34:03 -04:00
preallocate.c block: remove has_variable_length from filters 2023-04-11 16:38:56 +02:00
progress_meter.c coroutine: Clean up superfluous inclusion of qemu/lockable.h 2023-01-19 10:18:28 +01:00
qapi-sysemu.c block: Take main AioContext lock when calling bdrv_open() 2023-05-30 17:21:23 +02:00
qapi.c block: add accounting for zone append operation 2023-05-15 08:18:10 -04:00
qcow2-bitmap.c qcow2: mark various functions as coroutine_fn and GRAPH_RDLOCK 2023-04-25 13:17:28 +02:00
qcow2-cache.c include/block: Untangle inclusion loops 2023-01-20 07:24:28 +01:00
qcow2-cluster.c qcow2: add discard-no-unref option 2023-06-05 13:15:42 +02:00
qcow2-refcount.c qcow2: Don't call bdrv_getlength() in coroutine_fns 2023-05-10 14:16:53 +02:00
qcow2-snapshot.c qcow2: mark various functions as coroutine_fn and GRAPH_RDLOCK 2023-04-25 13:17:28 +02:00
qcow2-threads.c thread-pool: avoid passing the pool parameter every time 2023-04-25 13:17:28 +02:00
qcow2.c qcow2: add discard-no-unref option 2023-06-05 13:15:42 +02:00
qcow2.h qcow2: add discard-no-unref option 2023-06-05 13:15:42 +02:00
qcow.c block: Call .bdrv_co_create(_opts) unlocked 2023-05-19 19:12:12 +02:00
qed-check.c block: Mark bdrv_co_flush() and callers GRAPH_RDLOCK 2023-02-23 19:49:12 +01:00
qed-cluster.c
qed-l2-cache.c osdep: Move memalign-related functions to their own header 2022-03-07 13:16:49 +00:00
qed-table.c block: Mark public read/write functions GRAPH_RDLOCK 2023-02-23 19:49:17 +01:00
qed.c block: Call .bdrv_co_create(_opts) unlocked 2023-05-19 19:12:12 +02:00
qed.h block: Mark public read/write functions GRAPH_RDLOCK 2023-02-23 19:49:17 +01:00
quorum.c block: Mark bdrv_recurse_can_replace() and callers GRAPH_RDLOCK 2023-05-10 14:16:54 +02:00
raw-format.c raw-format: Fix open with 'file' in iothread 2023-05-30 17:21:23 +02:00
rbd.c block/rbd: Add support for layered encryption 2023-02-23 19:49:35 +01:00
replication.c block: remove has_variable_length from filters 2023-04-11 16:38:56 +02:00
reqlist.c block/reqlist: add reqlist_wait_all() 2022-03-07 09:33:30 +01:00
snapshot-access.c block: Mark preadv_snapshot/snapshot_block_status GRAPH_RDLOCK 2023-02-23 19:49:21 +01:00
snapshot.c block/snapshot: drop indirection around bdrv_snapshot_fallback_ptr 2022-10-27 20:14:11 +02:00
ssh.c aio: remove aio_disable_external() API 2023-05-30 17:37:26 +02:00
stream.c blockjob: Adhere to rate limit even when reentered early 2023-05-19 19:12:12 +02:00
throttle-groups.c block/throttle-groups: throttle_group_co_io_limits_intercept(): 64bit bytes 2021-02-03 08:14:00 -06:00
throttle.c block: Mark bdrv_co_refresh_total_sectors() and callers GRAPH_RDLOCK 2023-02-23 19:49:33 +01:00
trace-events block/io_uring: convert to blk_io_plug_call() API 2023-06-01 07:34:03 -04:00
trace.h trace: switch position of headers to what Meson requires 2020-08-21 06:18:24 -04:00
vdi.c block: Call .bdrv_co_create(_opts) unlocked 2023-05-19 19:12:12 +02:00
vhdx-endian.c Include qemu-common.h exactly where needed 2019-06-12 13:20:20 +02:00
vhdx-log.c block/vhdx: fix dynamic VHDX BAT corruption 2023-04-11 13:53:03 +02:00
vhdx.c block: Call .bdrv_co_create(_opts) unlocked 2023-05-19 19:12:12 +02:00
vhdx.h block/vhdx: Use IEC binary prefixes for size constants 2019-04-30 15:29:00 +02:00
vmdk.c block: Call .bdrv_co_create(_opts) unlocked 2023-05-19 19:12:12 +02:00
vpc.c block: Call .bdrv_co_create(_opts) unlocked 2023-05-19 19:12:12 +02:00
vvfat.c vvfat: mark various functions as coroutine_fn 2023-04-25 13:17:28 +02:00
win32-aio.c aio: remove aio_disable_external() API 2023-05-30 17:37:26 +02:00
write-threshold.c include/block: Untangle inclusion loops 2023-01-20 07:24:28 +01:00