2015-03-19 15:33:33 +03:00
|
|
|
QA output created by 122
|
|
|
|
|
Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=67108864
|
|
|
|
|
wrote 67108864/67108864 bytes at offset 0
|
|
|
|
|
64 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|
|
|
|
|
|
|
|
|
|
=== Check allocation status regression with -B ===
|
|
|
|
|
|
iotests: Specify explicit backing format where sensible
There are many existing qcow2 images that specify a backing file but
no format. This has been the source of CVEs in the past, but has
become more prominent of a problem now that libvirt has switched to
-blockdev. With older -drive, at least the probing was always done by
qemu (so the only risk of a changed format between successive boots of
a guest was if qemu was upgraded and probed differently). But with
newer -blockdev, libvirt must specify a format; if libvirt guesses raw
where the image was formatted, this results in data corruption visible
to the guest; conversely, if libvirt guesses qcow2 where qemu was
using raw, this can result in potential security holes, so modern
libvirt instead refuses to use images without explicit backing format.
The change in libvirt to reject images without explicit backing format
has pointed out that a number of tools have been far too reliant on
probing in the past. It's time to set a better example in our own
iotests of properly setting this parameter.
iotest calls to create, rebase, and convert are all impacted to some
degree. It's a bit annoying that we are inconsistent on command line
- while all of those accept -o backing_file=...,backing_fmt=..., the
shortcuts are different: create and rebase have -b and -F, while
convert has -B but no -F. (amend has no shortcuts, but the previous
patch just deprecated the use of amend to change backing chains).
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20200706203954.341758-9-eblake@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-07-06 23:39:52 +03:00
|
|
|
Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 backing_file=TEST_DIR/t.IMGFMT.base backing_fmt=IMGFMT
|
2015-03-19 15:33:33 +03:00
|
|
|
wrote 3145728/3145728 bytes at offset 0
|
|
|
|
|
3 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|
|
|
|
|
Offset Length File
|
|
|
|
|
0 0x300000 TEST_DIR/t.IMGFMT.orig
|
|
|
|
|
0x300000 0x3d00000 TEST_DIR/t.IMGFMT.base
|
|
|
|
|
|
|
|
|
|
=== Check that zero clusters are kept in overlay ===
|
|
|
|
|
|
iotests: Specify explicit backing format where sensible
There are many existing qcow2 images that specify a backing file but
no format. This has been the source of CVEs in the past, but has
become more prominent of a problem now that libvirt has switched to
-blockdev. With older -drive, at least the probing was always done by
qemu (so the only risk of a changed format between successive boots of
a guest was if qemu was upgraded and probed differently). But with
newer -blockdev, libvirt must specify a format; if libvirt guesses raw
where the image was formatted, this results in data corruption visible
to the guest; conversely, if libvirt guesses qcow2 where qemu was
using raw, this can result in potential security holes, so modern
libvirt instead refuses to use images without explicit backing format.
The change in libvirt to reject images without explicit backing format
has pointed out that a number of tools have been far too reliant on
probing in the past. It's time to set a better example in our own
iotests of properly setting this parameter.
iotest calls to create, rebase, and convert are all impacted to some
degree. It's a bit annoying that we are inconsistent on command line
- while all of those accept -o backing_file=...,backing_fmt=..., the
shortcuts are different: create and rebase have -b and -F, while
convert has -B but no -F. (amend has no shortcuts, but the previous
patch just deprecated the use of amend to change backing chains).
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20200706203954.341758-9-eblake@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-07-06 23:39:52 +03:00
|
|
|
Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 backing_file=TEST_DIR/t.IMGFMT.base backing_fmt=IMGFMT
|
2015-03-19 15:33:33 +03:00
|
|
|
wrote 3145728/3145728 bytes at offset 0
|
|
|
|
|
3 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|
|
|
|
|
read 3145728/3145728 bytes at offset 0
|
|
|
|
|
3 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|
|
|
|
|
read 3145728/3145728 bytes at offset 0
|
|
|
|
|
3 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|
|
|
|
|
wrote 3145728/3145728 bytes at offset 0
|
|
|
|
|
3 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|
|
|
|
|
read 3145728/3145728 bytes at offset 0
|
|
|
|
|
3 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|
|
|
|
|
read 3145728/3145728 bytes at offset 0
|
|
|
|
|
3 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|
|
|
|
|
|
2018-05-01 19:57:50 +03:00
|
|
|
=== Converting to an overlay larger than its backing file ===
|
|
|
|
|
|
|
|
|
|
Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=268435456
|
iotests: Specify explicit backing format where sensible
There are many existing qcow2 images that specify a backing file but
no format. This has been the source of CVEs in the past, but has
become more prominent of a problem now that libvirt has switched to
-blockdev. With older -drive, at least the probing was always done by
qemu (so the only risk of a changed format between successive boots of
a guest was if qemu was upgraded and probed differently). But with
newer -blockdev, libvirt must specify a format; if libvirt guesses raw
where the image was formatted, this results in data corruption visible
to the guest; conversely, if libvirt guesses qcow2 where qemu was
using raw, this can result in potential security holes, so modern
libvirt instead refuses to use images without explicit backing format.
The change in libvirt to reject images without explicit backing format
has pointed out that a number of tools have been far too reliant on
probing in the past. It's time to set a better example in our own
iotests of properly setting this parameter.
iotest calls to create, rebase, and convert are all impacted to some
degree. It's a bit annoying that we are inconsistent on command line
- while all of those accept -o backing_file=...,backing_fmt=..., the
shortcuts are different: create and rebase have -b and -F, while
convert has -B but no -F. (amend has no shortcuts, but the previous
patch just deprecated the use of amend to change backing chains).
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20200706203954.341758-9-eblake@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-07-06 23:39:52 +03:00
|
|
|
Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=805306368 backing_file=TEST_DIR/t.IMGFMT.base backing_fmt=IMGFMT
|
2018-05-01 19:57:50 +03:00
|
|
|
Offset Length File
|
|
|
|
|
|
|
|
|
|
wrote 1048576/1048576 bytes at offset 267386880
|
|
|
|
|
1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|
|
|
|
|
wrote 1048576/1048576 bytes at offset 629145600
|
|
|
|
|
1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|
|
|
|
|
Offset Length File
|
|
|
|
|
0xff00000 0x100000 TEST_DIR/t.IMGFMT.base
|
|
|
|
|
0x25800000 0x100000 TEST_DIR/t.IMGFMT.orig
|
|
|
|
|
read 1048576/1048576 bytes at offset 267386880
|
|
|
|
|
1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|
|
|
|
|
read 1048576/1048576 bytes at offset 629145600
|
|
|
|
|
1 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|
|
|
|
|
|
2015-03-19 15:33:33 +03:00
|
|
|
=== Concatenate multiple source images ===
|
|
|
|
|
|
|
|
|
|
Formatting 'TEST_DIR/t.IMGFMT.1', fmt=IMGFMT size=4194304
|
|
|
|
|
Formatting 'TEST_DIR/t.IMGFMT.2', fmt=IMGFMT size=4194304
|
|
|
|
|
Formatting 'TEST_DIR/t.IMGFMT.3', fmt=IMGFMT size=4194304
|
|
|
|
|
wrote 65536/65536 bytes at offset 0
|
|
|
|
|
64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|
|
|
|
|
wrote 65536/65536 bytes at offset 0
|
|
|
|
|
64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|
|
|
|
|
wrote 65536/65536 bytes at offset 0
|
|
|
|
|
64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|
|
|
|
|
Offset Length File
|
|
|
|
|
0 0x10000 TEST_DIR/t.IMGFMT
|
|
|
|
|
0x400000 0x10000 TEST_DIR/t.IMGFMT
|
|
|
|
|
0x800000 0x10000 TEST_DIR/t.IMGFMT
|
|
|
|
|
read 65536/65536 bytes at offset 0
|
|
|
|
|
64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|
|
|
|
|
read 65536/65536 bytes at offset 4194304
|
|
|
|
|
64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|
|
|
|
|
read 65536/65536 bytes at offset 8388608
|
|
|
|
|
64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|
qemu-img: Make unallocated part of backing chain obvious in map
The recently-added NBD context qemu:allocation-depth is able to
distinguish between locally-present data (even when that data is
sparse) [shown as depth 1 over NBD], and data that could not be found
anywhere in the backing chain [shown as depth 0]; and the libnbd
project was recently patched to give the human-readable name "absent"
to an allocation-depth of 0. But qemu-img map --output=json predates
that addition, and has the unfortunate behavior that all portions of
the backing chain that resolve without finding a hit in any backing
layer report the same depth as the final backing layer. This makes it
harder to reconstruct a qcow2 backing chain using just 'qemu-img map'
output, especially when using "backing":null to artificially limit a
backing chain, because it is impossible to distinguish between a
QCOW2_CLUSTER_UNALLOCATED (which defers to a [missing] backing file)
and a QCOW2_CLUSTER_ZERO_PLAIN cluster (which would override any
backing file), since both types of clusters otherwise show as
"data":false,"zero":true" (but note that we can distinguish a
QCOW2_CLUSTER_ZERO_ALLOCATED, which would also have an "offset":
listing).
The task of reconstructing a qcow2 chain was made harder in commit
0da9856851 (nbd: server: Report holes for raw images), because prior
to that point, it was possible to abuse NBD's block status command to
see which portions of a qcow2 file resulted in BDRV_BLOCK_ALLOCATED
(showing up as NBD_STATE_ZERO in isolation) vs. missing from the chain
(showing up as NBD_STATE_ZERO|NBD_STATE_HOLE); but now qemu reports
more accurate sparseness information over NBD.
An obvious solution is to make 'qemu-img map --output=json' add an
additional "present":false designation to any cluster lacking an
allocation anywhere in the chain, without any change to the "depth"
parameter to avoid breaking existing clients. The iotests have
several examples where this distinction demonstrates the additional
accuracy.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210701190655.2131223-3-eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
[eblake: fix more iotest fallout]
Signed-off-by: Eric Blake <eblake@redhat.com>
2021-07-01 22:06:55 +03:00
|
|
|
[{ "start": 0, "length": 65536, "depth": 0, "present": true, "zero": false, "data": true},
|
|
|
|
|
{ "start": 65536, "length": 4128768, "depth": 0, "present": false, "zero": true, "data": false},
|
|
|
|
|
{ "start": 4194304, "length": 65536, "depth": 0, "present": true, "zero": false, "data": true},
|
|
|
|
|
{ "start": 4259840, "length": 4128768, "depth": 0, "present": false, "zero": true, "data": false},
|
|
|
|
|
{ "start": 8388608, "length": 65536, "depth": 0, "present": true, "zero": false, "data": true},
|
|
|
|
|
{ "start": 8454144, "length": 4128768, "depth": 0, "present": false, "zero": true, "data": false}]
|
2015-03-19 15:33:33 +03:00
|
|
|
read 65536/65536 bytes at offset 0
|
|
|
|
|
64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|
|
|
|
|
read 65536/65536 bytes at offset 4194304
|
|
|
|
|
64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|
|
|
|
|
read 65536/65536 bytes at offset 8388608
|
|
|
|
|
64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|
2017-04-26 16:46:48 +03:00
|
|
|
qemu-img: Having a backing file for the target makes no sense when concatenating multiple input images
|
|
|
|
|
qemu-img: Having a backing file for the target makes no sense when concatenating multiple input images
|
2015-03-19 15:33:33 +03:00
|
|
|
|
|
|
|
|
=== Compression with misaligned allocations and image sizes ===
|
|
|
|
|
|
|
|
|
|
Formatting 'TEST_DIR/t.IMGFMT.1', fmt=IMGFMT size=1047552
|
|
|
|
|
Formatting 'TEST_DIR/t.IMGFMT.2', fmt=IMGFMT size=1047552
|
|
|
|
|
wrote 16384/16384 bytes at offset 16384
|
|
|
|
|
16 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|
|
|
|
|
wrote 133120/133120 bytes at offset 133120
|
|
|
|
|
130 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|
|
|
|
|
wrote 1024/1024 bytes at offset 1046528
|
|
|
|
|
1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|
|
|
|
|
wrote 1024/1024 bytes at offset 0
|
|
|
|
|
1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|
qemu-img: Make unallocated part of backing chain obvious in map
The recently-added NBD context qemu:allocation-depth is able to
distinguish between locally-present data (even when that data is
sparse) [shown as depth 1 over NBD], and data that could not be found
anywhere in the backing chain [shown as depth 0]; and the libnbd
project was recently patched to give the human-readable name "absent"
to an allocation-depth of 0. But qemu-img map --output=json predates
that addition, and has the unfortunate behavior that all portions of
the backing chain that resolve without finding a hit in any backing
layer report the same depth as the final backing layer. This makes it
harder to reconstruct a qcow2 backing chain using just 'qemu-img map'
output, especially when using "backing":null to artificially limit a
backing chain, because it is impossible to distinguish between a
QCOW2_CLUSTER_UNALLOCATED (which defers to a [missing] backing file)
and a QCOW2_CLUSTER_ZERO_PLAIN cluster (which would override any
backing file), since both types of clusters otherwise show as
"data":false,"zero":true" (but note that we can distinguish a
QCOW2_CLUSTER_ZERO_ALLOCATED, which would also have an "offset":
listing).
The task of reconstructing a qcow2 chain was made harder in commit
0da9856851 (nbd: server: Report holes for raw images), because prior
to that point, it was possible to abuse NBD's block status command to
see which portions of a qcow2 file resulted in BDRV_BLOCK_ALLOCATED
(showing up as NBD_STATE_ZERO in isolation) vs. missing from the chain
(showing up as NBD_STATE_ZERO|NBD_STATE_HOLE); but now qemu reports
more accurate sparseness information over NBD.
An obvious solution is to make 'qemu-img map --output=json' add an
additional "present":false designation to any cluster lacking an
allocation anywhere in the chain, without any change to the "depth"
parameter to avoid breaking existing clients. The iotests have
several examples where this distinction demonstrates the additional
accuracy.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210701190655.2131223-3-eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
[eblake: fix more iotest fallout]
Signed-off-by: Eric Blake <eblake@redhat.com>
2021-07-01 22:06:55 +03:00
|
|
|
[{ "start": 0, "length": 65536, "depth": 0, "present": true, "zero": false, "data": true},
|
|
|
|
|
{ "start": 65536, "length": 65536, "depth": 0, "present": false, "zero": true, "data": false},
|
|
|
|
|
{ "start": 131072, "length": 196608, "depth": 0, "present": true, "zero": false, "data": true},
|
|
|
|
|
{ "start": 327680, "length": 655360, "depth": 0, "present": false, "zero": true, "data": false},
|
|
|
|
|
{ "start": 983040, "length": 65536, "depth": 0, "present": true, "zero": false, "data": true},
|
|
|
|
|
{ "start": 1048576, "length": 1046528, "depth": 0, "present": false, "zero": true, "data": false}]
|
2015-03-19 15:33:33 +03:00
|
|
|
read 16384/16384 bytes at offset 0
|
|
|
|
|
16 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|
|
|
|
|
read 16384/16384 bytes at offset 16384
|
|
|
|
|
16 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|
|
|
|
|
read 100352/100352 bytes at offset 32768
|
|
|
|
|
98 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|
|
|
|
|
read 133120/133120 bytes at offset 133120
|
|
|
|
|
130 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|
|
|
|
|
read 780288/780288 bytes at offset 266240
|
|
|
|
|
762 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|
|
|
|
|
read 1024/1024 bytes at offset 1046528
|
|
|
|
|
1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|
|
|
|
|
read 1024/1024 bytes at offset 1047552
|
|
|
|
|
1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|
|
|
|
|
read 1046528/1046528 bytes at offset 1048576
|
|
|
|
|
1022 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|
|
|
|
|
|
|
|
|
|
=== Full allocation with -S 0 ===
|
|
|
|
|
|
|
|
|
|
Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
|
|
|
|
|
wrote 3145728/3145728 bytes at offset 0
|
|
|
|
|
3 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|
|
|
|
|
wrote 3145728/3145728 bytes at offset 3145728
|
|
|
|
|
3 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|
|
|
|
|
|
|
|
|
|
convert -S 0:
|
|
|
|
|
read 3145728/3145728 bytes at offset 0
|
|
|
|
|
3 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|
|
|
|
|
read 63963136/63963136 bytes at offset 3145728
|
|
|
|
|
61 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|
qemu-img: Make unallocated part of backing chain obvious in map
The recently-added NBD context qemu:allocation-depth is able to
distinguish between locally-present data (even when that data is
sparse) [shown as depth 1 over NBD], and data that could not be found
anywhere in the backing chain [shown as depth 0]; and the libnbd
project was recently patched to give the human-readable name "absent"
to an allocation-depth of 0. But qemu-img map --output=json predates
that addition, and has the unfortunate behavior that all portions of
the backing chain that resolve without finding a hit in any backing
layer report the same depth as the final backing layer. This makes it
harder to reconstruct a qcow2 backing chain using just 'qemu-img map'
output, especially when using "backing":null to artificially limit a
backing chain, because it is impossible to distinguish between a
QCOW2_CLUSTER_UNALLOCATED (which defers to a [missing] backing file)
and a QCOW2_CLUSTER_ZERO_PLAIN cluster (which would override any
backing file), since both types of clusters otherwise show as
"data":false,"zero":true" (but note that we can distinguish a
QCOW2_CLUSTER_ZERO_ALLOCATED, which would also have an "offset":
listing).
The task of reconstructing a qcow2 chain was made harder in commit
0da9856851 (nbd: server: Report holes for raw images), because prior
to that point, it was possible to abuse NBD's block status command to
see which portions of a qcow2 file resulted in BDRV_BLOCK_ALLOCATED
(showing up as NBD_STATE_ZERO in isolation) vs. missing from the chain
(showing up as NBD_STATE_ZERO|NBD_STATE_HOLE); but now qemu reports
more accurate sparseness information over NBD.
An obvious solution is to make 'qemu-img map --output=json' add an
additional "present":false designation to any cluster lacking an
allocation anywhere in the chain, without any change to the "depth"
parameter to avoid breaking existing clients. The iotests have
several examples where this distinction demonstrates the additional
accuracy.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210701190655.2131223-3-eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
[eblake: fix more iotest fallout]
Signed-off-by: Eric Blake <eblake@redhat.com>
2021-07-01 22:06:55 +03:00
|
|
|
[{ "start": 0, "length": 67108864, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}]
|
2015-03-19 15:33:33 +03:00
|
|
|
|
|
|
|
|
convert -c -S 0:
|
|
|
|
|
read 3145728/3145728 bytes at offset 0
|
|
|
|
|
3 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|
|
|
|
|
read 63963136/63963136 bytes at offset 3145728
|
|
|
|
|
61 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|
qemu-img: Make unallocated part of backing chain obvious in map
The recently-added NBD context qemu:allocation-depth is able to
distinguish between locally-present data (even when that data is
sparse) [shown as depth 1 over NBD], and data that could not be found
anywhere in the backing chain [shown as depth 0]; and the libnbd
project was recently patched to give the human-readable name "absent"
to an allocation-depth of 0. But qemu-img map --output=json predates
that addition, and has the unfortunate behavior that all portions of
the backing chain that resolve without finding a hit in any backing
layer report the same depth as the final backing layer. This makes it
harder to reconstruct a qcow2 backing chain using just 'qemu-img map'
output, especially when using "backing":null to artificially limit a
backing chain, because it is impossible to distinguish between a
QCOW2_CLUSTER_UNALLOCATED (which defers to a [missing] backing file)
and a QCOW2_CLUSTER_ZERO_PLAIN cluster (which would override any
backing file), since both types of clusters otherwise show as
"data":false,"zero":true" (but note that we can distinguish a
QCOW2_CLUSTER_ZERO_ALLOCATED, which would also have an "offset":
listing).
The task of reconstructing a qcow2 chain was made harder in commit
0da9856851 (nbd: server: Report holes for raw images), because prior
to that point, it was possible to abuse NBD's block status command to
see which portions of a qcow2 file resulted in BDRV_BLOCK_ALLOCATED
(showing up as NBD_STATE_ZERO in isolation) vs. missing from the chain
(showing up as NBD_STATE_ZERO|NBD_STATE_HOLE); but now qemu reports
more accurate sparseness information over NBD.
An obvious solution is to make 'qemu-img map --output=json' add an
additional "present":false designation to any cluster lacking an
allocation anywhere in the chain, without any change to the "depth"
parameter to avoid breaking existing clients. The iotests have
several examples where this distinction demonstrates the additional
accuracy.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210701190655.2131223-3-eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
[eblake: fix more iotest fallout]
Signed-off-by: Eric Blake <eblake@redhat.com>
2021-07-01 22:06:55 +03:00
|
|
|
[{ "start": 0, "length": 67108864, "depth": 0, "present": true, "zero": false, "data": true}]
|
2015-03-19 15:33:33 +03:00
|
|
|
Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=67108864
|
|
|
|
|
wrote 33554432/33554432 bytes at offset 0
|
|
|
|
|
32 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|
iotests: Specify explicit backing format where sensible
There are many existing qcow2 images that specify a backing file but
no format. This has been the source of CVEs in the past, but has
become more prominent of a problem now that libvirt has switched to
-blockdev. With older -drive, at least the probing was always done by
qemu (so the only risk of a changed format between successive boots of
a guest was if qemu was upgraded and probed differently). But with
newer -blockdev, libvirt must specify a format; if libvirt guesses raw
where the image was formatted, this results in data corruption visible
to the guest; conversely, if libvirt guesses qcow2 where qemu was
using raw, this can result in potential security holes, so modern
libvirt instead refuses to use images without explicit backing format.
The change in libvirt to reject images without explicit backing format
has pointed out that a number of tools have been far too reliant on
probing in the past. It's time to set a better example in our own
iotests of properly setting this parameter.
iotest calls to create, rebase, and convert are all impacted to some
degree. It's a bit annoying that we are inconsistent on command line
- while all of those accept -o backing_file=...,backing_fmt=..., the
shortcuts are different: create and rebase have -b and -F, while
convert has -B but no -F. (amend has no shortcuts, but the previous
patch just deprecated the use of amend to change backing chains).
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20200706203954.341758-9-eblake@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-07-06 23:39:52 +03:00
|
|
|
Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 backing_file=TEST_DIR/t.IMGFMT.base backing_fmt=IMGFMT
|
2015-03-19 15:33:33 +03:00
|
|
|
wrote 3145728/3145728 bytes at offset 0
|
|
|
|
|
3 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|
|
|
|
|
|
|
|
|
|
convert -S 0 with source backing file:
|
|
|
|
|
read 3145728/3145728 bytes at offset 0
|
|
|
|
|
3 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|
|
|
|
|
read 30408704/30408704 bytes at offset 3145728
|
|
|
|
|
29 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|
|
|
|
|
read 33554432/33554432 bytes at offset 33554432
|
|
|
|
|
32 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|
qemu-img: Make unallocated part of backing chain obvious in map
The recently-added NBD context qemu:allocation-depth is able to
distinguish between locally-present data (even when that data is
sparse) [shown as depth 1 over NBD], and data that could not be found
anywhere in the backing chain [shown as depth 0]; and the libnbd
project was recently patched to give the human-readable name "absent"
to an allocation-depth of 0. But qemu-img map --output=json predates
that addition, and has the unfortunate behavior that all portions of
the backing chain that resolve without finding a hit in any backing
layer report the same depth as the final backing layer. This makes it
harder to reconstruct a qcow2 backing chain using just 'qemu-img map'
output, especially when using "backing":null to artificially limit a
backing chain, because it is impossible to distinguish between a
QCOW2_CLUSTER_UNALLOCATED (which defers to a [missing] backing file)
and a QCOW2_CLUSTER_ZERO_PLAIN cluster (which would override any
backing file), since both types of clusters otherwise show as
"data":false,"zero":true" (but note that we can distinguish a
QCOW2_CLUSTER_ZERO_ALLOCATED, which would also have an "offset":
listing).
The task of reconstructing a qcow2 chain was made harder in commit
0da9856851 (nbd: server: Report holes for raw images), because prior
to that point, it was possible to abuse NBD's block status command to
see which portions of a qcow2 file resulted in BDRV_BLOCK_ALLOCATED
(showing up as NBD_STATE_ZERO in isolation) vs. missing from the chain
(showing up as NBD_STATE_ZERO|NBD_STATE_HOLE); but now qemu reports
more accurate sparseness information over NBD.
An obvious solution is to make 'qemu-img map --output=json' add an
additional "present":false designation to any cluster lacking an
allocation anywhere in the chain, without any change to the "depth"
parameter to avoid breaking existing clients. The iotests have
several examples where this distinction demonstrates the additional
accuracy.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210701190655.2131223-3-eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
[eblake: fix more iotest fallout]
Signed-off-by: Eric Blake <eblake@redhat.com>
2021-07-01 22:06:55 +03:00
|
|
|
[{ "start": 0, "length": 67108864, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}]
|
2015-03-19 15:33:33 +03:00
|
|
|
|
|
|
|
|
convert -c -S 0 with source backing file:
|
|
|
|
|
read 3145728/3145728 bytes at offset 0
|
|
|
|
|
3 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|
|
|
|
|
read 30408704/30408704 bytes at offset 3145728
|
|
|
|
|
29 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|
|
|
|
|
read 33554432/33554432 bytes at offset 33554432
|
|
|
|
|
32 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|
qemu-img: Make unallocated part of backing chain obvious in map
The recently-added NBD context qemu:allocation-depth is able to
distinguish between locally-present data (even when that data is
sparse) [shown as depth 1 over NBD], and data that could not be found
anywhere in the backing chain [shown as depth 0]; and the libnbd
project was recently patched to give the human-readable name "absent"
to an allocation-depth of 0. But qemu-img map --output=json predates
that addition, and has the unfortunate behavior that all portions of
the backing chain that resolve without finding a hit in any backing
layer report the same depth as the final backing layer. This makes it
harder to reconstruct a qcow2 backing chain using just 'qemu-img map'
output, especially when using "backing":null to artificially limit a
backing chain, because it is impossible to distinguish between a
QCOW2_CLUSTER_UNALLOCATED (which defers to a [missing] backing file)
and a QCOW2_CLUSTER_ZERO_PLAIN cluster (which would override any
backing file), since both types of clusters otherwise show as
"data":false,"zero":true" (but note that we can distinguish a
QCOW2_CLUSTER_ZERO_ALLOCATED, which would also have an "offset":
listing).
The task of reconstructing a qcow2 chain was made harder in commit
0da9856851 (nbd: server: Report holes for raw images), because prior
to that point, it was possible to abuse NBD's block status command to
see which portions of a qcow2 file resulted in BDRV_BLOCK_ALLOCATED
(showing up as NBD_STATE_ZERO in isolation) vs. missing from the chain
(showing up as NBD_STATE_ZERO|NBD_STATE_HOLE); but now qemu reports
more accurate sparseness information over NBD.
An obvious solution is to make 'qemu-img map --output=json' add an
additional "present":false designation to any cluster lacking an
allocation anywhere in the chain, without any change to the "depth"
parameter to avoid breaking existing clients. The iotests have
several examples where this distinction demonstrates the additional
accuracy.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210701190655.2131223-3-eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
[eblake: fix more iotest fallout]
Signed-off-by: Eric Blake <eblake@redhat.com>
2021-07-01 22:06:55 +03:00
|
|
|
[{ "start": 0, "length": 67108864, "depth": 0, "present": true, "zero": false, "data": true}]
|
2015-03-19 15:33:33 +03:00
|
|
|
|
|
|
|
|
convert -S 0 -B ...
|
|
|
|
|
read 3145728/3145728 bytes at offset 0
|
|
|
|
|
3 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|
|
|
|
|
read 30408704/30408704 bytes at offset 3145728
|
|
|
|
|
29 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|
|
|
|
|
read 33554432/33554432 bytes at offset 33554432
|
|
|
|
|
32 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|
qemu-img: Make unallocated part of backing chain obvious in map
The recently-added NBD context qemu:allocation-depth is able to
distinguish between locally-present data (even when that data is
sparse) [shown as depth 1 over NBD], and data that could not be found
anywhere in the backing chain [shown as depth 0]; and the libnbd
project was recently patched to give the human-readable name "absent"
to an allocation-depth of 0. But qemu-img map --output=json predates
that addition, and has the unfortunate behavior that all portions of
the backing chain that resolve without finding a hit in any backing
layer report the same depth as the final backing layer. This makes it
harder to reconstruct a qcow2 backing chain using just 'qemu-img map'
output, especially when using "backing":null to artificially limit a
backing chain, because it is impossible to distinguish between a
QCOW2_CLUSTER_UNALLOCATED (which defers to a [missing] backing file)
and a QCOW2_CLUSTER_ZERO_PLAIN cluster (which would override any
backing file), since both types of clusters otherwise show as
"data":false,"zero":true" (but note that we can distinguish a
QCOW2_CLUSTER_ZERO_ALLOCATED, which would also have an "offset":
listing).
The task of reconstructing a qcow2 chain was made harder in commit
0da9856851 (nbd: server: Report holes for raw images), because prior
to that point, it was possible to abuse NBD's block status command to
see which portions of a qcow2 file resulted in BDRV_BLOCK_ALLOCATED
(showing up as NBD_STATE_ZERO in isolation) vs. missing from the chain
(showing up as NBD_STATE_ZERO|NBD_STATE_HOLE); but now qemu reports
more accurate sparseness information over NBD.
An obvious solution is to make 'qemu-img map --output=json' add an
additional "present":false designation to any cluster lacking an
allocation anywhere in the chain, without any change to the "depth"
parameter to avoid breaking existing clients. The iotests have
several examples where this distinction demonstrates the additional
accuracy.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210701190655.2131223-3-eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
[eblake: fix more iotest fallout]
Signed-off-by: Eric Blake <eblake@redhat.com>
2021-07-01 22:06:55 +03:00
|
|
|
[{ "start": 0, "length": 67108864, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET}]
|
2015-03-19 15:33:33 +03:00
|
|
|
|
|
|
|
|
convert -c -S 0 -B ...
|
|
|
|
|
read 3145728/3145728 bytes at offset 0
|
|
|
|
|
3 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|
|
|
|
|
read 30408704/30408704 bytes at offset 3145728
|
|
|
|
|
29 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|
|
|
|
|
read 33554432/33554432 bytes at offset 33554432
|
|
|
|
|
32 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|
qemu-img: Make unallocated part of backing chain obvious in map
The recently-added NBD context qemu:allocation-depth is able to
distinguish between locally-present data (even when that data is
sparse) [shown as depth 1 over NBD], and data that could not be found
anywhere in the backing chain [shown as depth 0]; and the libnbd
project was recently patched to give the human-readable name "absent"
to an allocation-depth of 0. But qemu-img map --output=json predates
that addition, and has the unfortunate behavior that all portions of
the backing chain that resolve without finding a hit in any backing
layer report the same depth as the final backing layer. This makes it
harder to reconstruct a qcow2 backing chain using just 'qemu-img map'
output, especially when using "backing":null to artificially limit a
backing chain, because it is impossible to distinguish between a
QCOW2_CLUSTER_UNALLOCATED (which defers to a [missing] backing file)
and a QCOW2_CLUSTER_ZERO_PLAIN cluster (which would override any
backing file), since both types of clusters otherwise show as
"data":false,"zero":true" (but note that we can distinguish a
QCOW2_CLUSTER_ZERO_ALLOCATED, which would also have an "offset":
listing).
The task of reconstructing a qcow2 chain was made harder in commit
0da9856851 (nbd: server: Report holes for raw images), because prior
to that point, it was possible to abuse NBD's block status command to
see which portions of a qcow2 file resulted in BDRV_BLOCK_ALLOCATED
(showing up as NBD_STATE_ZERO in isolation) vs. missing from the chain
(showing up as NBD_STATE_ZERO|NBD_STATE_HOLE); but now qemu reports
more accurate sparseness information over NBD.
An obvious solution is to make 'qemu-img map --output=json' add an
additional "present":false designation to any cluster lacking an
allocation anywhere in the chain, without any change to the "depth"
parameter to avoid breaking existing clients. The iotests have
several examples where this distinction demonstrates the additional
accuracy.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210701190655.2131223-3-eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
[eblake: fix more iotest fallout]
Signed-off-by: Eric Blake <eblake@redhat.com>
2021-07-01 22:06:55 +03:00
|
|
|
[{ "start": 0, "length": 67108864, "depth": 0, "present": true, "zero": false, "data": true}]
|
2015-03-19 15:33:33 +03:00
|
|
|
|
|
|
|
|
=== Non-zero -S ===
|
|
|
|
|
|
|
|
|
|
Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
|
|
|
|
|
wrote 65536/65536 bytes at offset 0
|
|
|
|
|
64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|
|
|
|
|
wrote 1024/1024 bytes at offset 0
|
|
|
|
|
1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|
|
|
|
|
wrote 1024/1024 bytes at offset 8192
|
|
|
|
|
1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|
|
|
|
|
wrote 1024/1024 bytes at offset 17408
|
|
|
|
|
1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|
|
|
|
|
|
|
|
|
|
convert -S 4k
|
qemu-img: Make unallocated part of backing chain obvious in map
The recently-added NBD context qemu:allocation-depth is able to
distinguish between locally-present data (even when that data is
sparse) [shown as depth 1 over NBD], and data that could not be found
anywhere in the backing chain [shown as depth 0]; and the libnbd
project was recently patched to give the human-readable name "absent"
to an allocation-depth of 0. But qemu-img map --output=json predates
that addition, and has the unfortunate behavior that all portions of
the backing chain that resolve without finding a hit in any backing
layer report the same depth as the final backing layer. This makes it
harder to reconstruct a qcow2 backing chain using just 'qemu-img map'
output, especially when using "backing":null to artificially limit a
backing chain, because it is impossible to distinguish between a
QCOW2_CLUSTER_UNALLOCATED (which defers to a [missing] backing file)
and a QCOW2_CLUSTER_ZERO_PLAIN cluster (which would override any
backing file), since both types of clusters otherwise show as
"data":false,"zero":true" (but note that we can distinguish a
QCOW2_CLUSTER_ZERO_ALLOCATED, which would also have an "offset":
listing).
The task of reconstructing a qcow2 chain was made harder in commit
0da9856851 (nbd: server: Report holes for raw images), because prior
to that point, it was possible to abuse NBD's block status command to
see which portions of a qcow2 file resulted in BDRV_BLOCK_ALLOCATED
(showing up as NBD_STATE_ZERO in isolation) vs. missing from the chain
(showing up as NBD_STATE_ZERO|NBD_STATE_HOLE); but now qemu reports
more accurate sparseness information over NBD.
An obvious solution is to make 'qemu-img map --output=json' add an
additional "present":false designation to any cluster lacking an
allocation anywhere in the chain, without any change to the "depth"
parameter to avoid breaking existing clients. The iotests have
several examples where this distinction demonstrates the additional
accuracy.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210701190655.2131223-3-eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
[eblake: fix more iotest fallout]
Signed-off-by: Eric Blake <eblake@redhat.com>
2021-07-01 22:06:55 +03:00
|
|
|
[{ "start": 0, "length": 4096, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET},
|
|
|
|
|
{ "start": 4096, "length": 4096, "depth": 0, "present": false, "zero": true, "data": false},
|
|
|
|
|
{ "start": 8192, "length": 4096, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET},
|
|
|
|
|
{ "start": 12288, "length": 4096, "depth": 0, "present": false, "zero": true, "data": false},
|
|
|
|
|
{ "start": 16384, "length": 4096, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET},
|
|
|
|
|
{ "start": 20480, "length": 67088384, "depth": 0, "present": false, "zero": true, "data": false}]
|
2015-03-19 15:33:33 +03:00
|
|
|
|
|
|
|
|
convert -c -S 4k
|
qemu-img: Make unallocated part of backing chain obvious in map
The recently-added NBD context qemu:allocation-depth is able to
distinguish between locally-present data (even when that data is
sparse) [shown as depth 1 over NBD], and data that could not be found
anywhere in the backing chain [shown as depth 0]; and the libnbd
project was recently patched to give the human-readable name "absent"
to an allocation-depth of 0. But qemu-img map --output=json predates
that addition, and has the unfortunate behavior that all portions of
the backing chain that resolve without finding a hit in any backing
layer report the same depth as the final backing layer. This makes it
harder to reconstruct a qcow2 backing chain using just 'qemu-img map'
output, especially when using "backing":null to artificially limit a
backing chain, because it is impossible to distinguish between a
QCOW2_CLUSTER_UNALLOCATED (which defers to a [missing] backing file)
and a QCOW2_CLUSTER_ZERO_PLAIN cluster (which would override any
backing file), since both types of clusters otherwise show as
"data":false,"zero":true" (but note that we can distinguish a
QCOW2_CLUSTER_ZERO_ALLOCATED, which would also have an "offset":
listing).
The task of reconstructing a qcow2 chain was made harder in commit
0da9856851 (nbd: server: Report holes for raw images), because prior
to that point, it was possible to abuse NBD's block status command to
see which portions of a qcow2 file resulted in BDRV_BLOCK_ALLOCATED
(showing up as NBD_STATE_ZERO in isolation) vs. missing from the chain
(showing up as NBD_STATE_ZERO|NBD_STATE_HOLE); but now qemu reports
more accurate sparseness information over NBD.
An obvious solution is to make 'qemu-img map --output=json' add an
additional "present":false designation to any cluster lacking an
allocation anywhere in the chain, without any change to the "depth"
parameter to avoid breaking existing clients. The iotests have
several examples where this distinction demonstrates the additional
accuracy.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210701190655.2131223-3-eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
[eblake: fix more iotest fallout]
Signed-off-by: Eric Blake <eblake@redhat.com>
2021-07-01 22:06:55 +03:00
|
|
|
[{ "start": 0, "length": 1024, "depth": 0, "present": true, "zero": false, "data": true},
|
|
|
|
|
{ "start": 1024, "length": 7168, "depth": 0, "present": false, "zero": true, "data": false},
|
|
|
|
|
{ "start": 8192, "length": 1024, "depth": 0, "present": true, "zero": false, "data": true},
|
|
|
|
|
{ "start": 9216, "length": 8192, "depth": 0, "present": false, "zero": true, "data": false},
|
|
|
|
|
{ "start": 17408, "length": 1024, "depth": 0, "present": true, "zero": false, "data": true},
|
|
|
|
|
{ "start": 18432, "length": 67090432, "depth": 0, "present": false, "zero": true, "data": false}]
|
2015-03-19 15:33:33 +03:00
|
|
|
|
|
|
|
|
convert -S 8k
|
qemu-img: Make unallocated part of backing chain obvious in map
The recently-added NBD context qemu:allocation-depth is able to
distinguish between locally-present data (even when that data is
sparse) [shown as depth 1 over NBD], and data that could not be found
anywhere in the backing chain [shown as depth 0]; and the libnbd
project was recently patched to give the human-readable name "absent"
to an allocation-depth of 0. But qemu-img map --output=json predates
that addition, and has the unfortunate behavior that all portions of
the backing chain that resolve without finding a hit in any backing
layer report the same depth as the final backing layer. This makes it
harder to reconstruct a qcow2 backing chain using just 'qemu-img map'
output, especially when using "backing":null to artificially limit a
backing chain, because it is impossible to distinguish between a
QCOW2_CLUSTER_UNALLOCATED (which defers to a [missing] backing file)
and a QCOW2_CLUSTER_ZERO_PLAIN cluster (which would override any
backing file), since both types of clusters otherwise show as
"data":false,"zero":true" (but note that we can distinguish a
QCOW2_CLUSTER_ZERO_ALLOCATED, which would also have an "offset":
listing).
The task of reconstructing a qcow2 chain was made harder in commit
0da9856851 (nbd: server: Report holes for raw images), because prior
to that point, it was possible to abuse NBD's block status command to
see which portions of a qcow2 file resulted in BDRV_BLOCK_ALLOCATED
(showing up as NBD_STATE_ZERO in isolation) vs. missing from the chain
(showing up as NBD_STATE_ZERO|NBD_STATE_HOLE); but now qemu reports
more accurate sparseness information over NBD.
An obvious solution is to make 'qemu-img map --output=json' add an
additional "present":false designation to any cluster lacking an
allocation anywhere in the chain, without any change to the "depth"
parameter to avoid breaking existing clients. The iotests have
several examples where this distinction demonstrates the additional
accuracy.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210701190655.2131223-3-eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
[eblake: fix more iotest fallout]
Signed-off-by: Eric Blake <eblake@redhat.com>
2021-07-01 22:06:55 +03:00
|
|
|
[{ "start": 0, "length": 24576, "depth": 0, "present": true, "zero": false, "data": true, "offset": OFFSET},
|
|
|
|
|
{ "start": 24576, "length": 67084288, "depth": 0, "present": false, "zero": true, "data": false}]
|
2015-03-19 15:33:33 +03:00
|
|
|
|
|
|
|
|
convert -c -S 8k
|
qemu-img: Make unallocated part of backing chain obvious in map
The recently-added NBD context qemu:allocation-depth is able to
distinguish between locally-present data (even when that data is
sparse) [shown as depth 1 over NBD], and data that could not be found
anywhere in the backing chain [shown as depth 0]; and the libnbd
project was recently patched to give the human-readable name "absent"
to an allocation-depth of 0. But qemu-img map --output=json predates
that addition, and has the unfortunate behavior that all portions of
the backing chain that resolve without finding a hit in any backing
layer report the same depth as the final backing layer. This makes it
harder to reconstruct a qcow2 backing chain using just 'qemu-img map'
output, especially when using "backing":null to artificially limit a
backing chain, because it is impossible to distinguish between a
QCOW2_CLUSTER_UNALLOCATED (which defers to a [missing] backing file)
and a QCOW2_CLUSTER_ZERO_PLAIN cluster (which would override any
backing file), since both types of clusters otherwise show as
"data":false,"zero":true" (but note that we can distinguish a
QCOW2_CLUSTER_ZERO_ALLOCATED, which would also have an "offset":
listing).
The task of reconstructing a qcow2 chain was made harder in commit
0da9856851 (nbd: server: Report holes for raw images), because prior
to that point, it was possible to abuse NBD's block status command to
see which portions of a qcow2 file resulted in BDRV_BLOCK_ALLOCATED
(showing up as NBD_STATE_ZERO in isolation) vs. missing from the chain
(showing up as NBD_STATE_ZERO|NBD_STATE_HOLE); but now qemu reports
more accurate sparseness information over NBD.
An obvious solution is to make 'qemu-img map --output=json' add an
additional "present":false designation to any cluster lacking an
allocation anywhere in the chain, without any change to the "depth"
parameter to avoid breaking existing clients. The iotests have
several examples where this distinction demonstrates the additional
accuracy.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210701190655.2131223-3-eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
[eblake: fix more iotest fallout]
Signed-off-by: Eric Blake <eblake@redhat.com>
2021-07-01 22:06:55 +03:00
|
|
|
[{ "start": 0, "length": 1024, "depth": 0, "present": true, "zero": false, "data": true},
|
|
|
|
|
{ "start": 1024, "length": 7168, "depth": 0, "present": false, "zero": true, "data": false},
|
|
|
|
|
{ "start": 8192, "length": 1024, "depth": 0, "present": true, "zero": false, "data": true},
|
|
|
|
|
{ "start": 9216, "length": 8192, "depth": 0, "present": false, "zero": true, "data": false},
|
|
|
|
|
{ "start": 17408, "length": 1024, "depth": 0, "present": true, "zero": false, "data": true},
|
|
|
|
|
{ "start": 18432, "length": 67090432, "depth": 0, "present": false, "zero": true, "data": false}]
|
2019-07-24 20:12:38 +03:00
|
|
|
|
|
|
|
|
=== -n to a non-zero image ===
|
|
|
|
|
|
|
|
|
|
Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
|
|
|
|
|
Formatting 'TEST_DIR/t.IMGFMT.orig', fmt=IMGFMT size=67108864
|
|
|
|
|
wrote 65536/65536 bytes at offset 0
|
|
|
|
|
64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
|
|
|
|
|
Images are identical.
|
2020-01-21 18:59:15 +03:00
|
|
|
|
2020-07-21 16:55:20 +03:00
|
|
|
=== -n to an empty image ===
|
|
|
|
|
|
|
|
|
|
Formatting 'TEST_DIR/t.IMGFMT.orig', fmt=IMGFMT size=67108864
|
|
|
|
|
Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
|
qemu-img: Make unallocated part of backing chain obvious in map
The recently-added NBD context qemu:allocation-depth is able to
distinguish between locally-present data (even when that data is
sparse) [shown as depth 1 over NBD], and data that could not be found
anywhere in the backing chain [shown as depth 0]; and the libnbd
project was recently patched to give the human-readable name "absent"
to an allocation-depth of 0. But qemu-img map --output=json predates
that addition, and has the unfortunate behavior that all portions of
the backing chain that resolve without finding a hit in any backing
layer report the same depth as the final backing layer. This makes it
harder to reconstruct a qcow2 backing chain using just 'qemu-img map'
output, especially when using "backing":null to artificially limit a
backing chain, because it is impossible to distinguish between a
QCOW2_CLUSTER_UNALLOCATED (which defers to a [missing] backing file)
and a QCOW2_CLUSTER_ZERO_PLAIN cluster (which would override any
backing file), since both types of clusters otherwise show as
"data":false,"zero":true" (but note that we can distinguish a
QCOW2_CLUSTER_ZERO_ALLOCATED, which would also have an "offset":
listing).
The task of reconstructing a qcow2 chain was made harder in commit
0da9856851 (nbd: server: Report holes for raw images), because prior
to that point, it was possible to abuse NBD's block status command to
see which portions of a qcow2 file resulted in BDRV_BLOCK_ALLOCATED
(showing up as NBD_STATE_ZERO in isolation) vs. missing from the chain
(showing up as NBD_STATE_ZERO|NBD_STATE_HOLE); but now qemu reports
more accurate sparseness information over NBD.
An obvious solution is to make 'qemu-img map --output=json' add an
additional "present":false designation to any cluster lacking an
allocation anywhere in the chain, without any change to the "depth"
parameter to avoid breaking existing clients. The iotests have
several examples where this distinction demonstrates the additional
accuracy.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210701190655.2131223-3-eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
[eblake: fix more iotest fallout]
Signed-off-by: Eric Blake <eblake@redhat.com>
2021-07-01 22:06:55 +03:00
|
|
|
[{ "start": 0, "length": 67108864, "depth": 0, "present": true, "zero": true, "data": false}]
|
2020-07-21 16:55:20 +03:00
|
|
|
Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
|
qemu-img: Make unallocated part of backing chain obvious in map
The recently-added NBD context qemu:allocation-depth is able to
distinguish between locally-present data (even when that data is
sparse) [shown as depth 1 over NBD], and data that could not be found
anywhere in the backing chain [shown as depth 0]; and the libnbd
project was recently patched to give the human-readable name "absent"
to an allocation-depth of 0. But qemu-img map --output=json predates
that addition, and has the unfortunate behavior that all portions of
the backing chain that resolve without finding a hit in any backing
layer report the same depth as the final backing layer. This makes it
harder to reconstruct a qcow2 backing chain using just 'qemu-img map'
output, especially when using "backing":null to artificially limit a
backing chain, because it is impossible to distinguish between a
QCOW2_CLUSTER_UNALLOCATED (which defers to a [missing] backing file)
and a QCOW2_CLUSTER_ZERO_PLAIN cluster (which would override any
backing file), since both types of clusters otherwise show as
"data":false,"zero":true" (but note that we can distinguish a
QCOW2_CLUSTER_ZERO_ALLOCATED, which would also have an "offset":
listing).
The task of reconstructing a qcow2 chain was made harder in commit
0da9856851 (nbd: server: Report holes for raw images), because prior
to that point, it was possible to abuse NBD's block status command to
see which portions of a qcow2 file resulted in BDRV_BLOCK_ALLOCATED
(showing up as NBD_STATE_ZERO in isolation) vs. missing from the chain
(showing up as NBD_STATE_ZERO|NBD_STATE_HOLE); but now qemu reports
more accurate sparseness information over NBD.
An obvious solution is to make 'qemu-img map --output=json' add an
additional "present":false designation to any cluster lacking an
allocation anywhere in the chain, without any change to the "depth"
parameter to avoid breaking existing clients. The iotests have
several examples where this distinction demonstrates the additional
accuracy.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210701190655.2131223-3-eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
[eblake: fix more iotest fallout]
Signed-off-by: Eric Blake <eblake@redhat.com>
2021-07-01 22:06:55 +03:00
|
|
|
[{ "start": 0, "length": 67108864, "depth": 0, "present": false, "zero": true, "data": false}]
|
2020-07-21 16:55:20 +03:00
|
|
|
|
|
|
|
|
=== -n to an empty image with a backing file ===
|
|
|
|
|
|
|
|
|
|
Formatting 'TEST_DIR/t.IMGFMT.orig', fmt=IMGFMT size=67108864
|
|
|
|
|
Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=67108864
|
|
|
|
|
Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 backing_file=TEST_DIR/t.IMGFMT.base backing_fmt=IMGFMT
|
qemu-img: Make unallocated part of backing chain obvious in map
The recently-added NBD context qemu:allocation-depth is able to
distinguish between locally-present data (even when that data is
sparse) [shown as depth 1 over NBD], and data that could not be found
anywhere in the backing chain [shown as depth 0]; and the libnbd
project was recently patched to give the human-readable name "absent"
to an allocation-depth of 0. But qemu-img map --output=json predates
that addition, and has the unfortunate behavior that all portions of
the backing chain that resolve without finding a hit in any backing
layer report the same depth as the final backing layer. This makes it
harder to reconstruct a qcow2 backing chain using just 'qemu-img map'
output, especially when using "backing":null to artificially limit a
backing chain, because it is impossible to distinguish between a
QCOW2_CLUSTER_UNALLOCATED (which defers to a [missing] backing file)
and a QCOW2_CLUSTER_ZERO_PLAIN cluster (which would override any
backing file), since both types of clusters otherwise show as
"data":false,"zero":true" (but note that we can distinguish a
QCOW2_CLUSTER_ZERO_ALLOCATED, which would also have an "offset":
listing).
The task of reconstructing a qcow2 chain was made harder in commit
0da9856851 (nbd: server: Report holes for raw images), because prior
to that point, it was possible to abuse NBD's block status command to
see which portions of a qcow2 file resulted in BDRV_BLOCK_ALLOCATED
(showing up as NBD_STATE_ZERO in isolation) vs. missing from the chain
(showing up as NBD_STATE_ZERO|NBD_STATE_HOLE); but now qemu reports
more accurate sparseness information over NBD.
An obvious solution is to make 'qemu-img map --output=json' add an
additional "present":false designation to any cluster lacking an
allocation anywhere in the chain, without any change to the "depth"
parameter to avoid breaking existing clients. The iotests have
several examples where this distinction demonstrates the additional
accuracy.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210701190655.2131223-3-eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
[eblake: fix more iotest fallout]
Signed-off-by: Eric Blake <eblake@redhat.com>
2021-07-01 22:06:55 +03:00
|
|
|
[{ "start": 0, "length": 67108864, "depth": 0, "present": true, "zero": true, "data": false}]
|
2020-07-21 16:55:20 +03:00
|
|
|
Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 backing_file=TEST_DIR/t.IMGFMT.base backing_fmt=IMGFMT
|
qemu-img: Make unallocated part of backing chain obvious in map
The recently-added NBD context qemu:allocation-depth is able to
distinguish between locally-present data (even when that data is
sparse) [shown as depth 1 over NBD], and data that could not be found
anywhere in the backing chain [shown as depth 0]; and the libnbd
project was recently patched to give the human-readable name "absent"
to an allocation-depth of 0. But qemu-img map --output=json predates
that addition, and has the unfortunate behavior that all portions of
the backing chain that resolve without finding a hit in any backing
layer report the same depth as the final backing layer. This makes it
harder to reconstruct a qcow2 backing chain using just 'qemu-img map'
output, especially when using "backing":null to artificially limit a
backing chain, because it is impossible to distinguish between a
QCOW2_CLUSTER_UNALLOCATED (which defers to a [missing] backing file)
and a QCOW2_CLUSTER_ZERO_PLAIN cluster (which would override any
backing file), since both types of clusters otherwise show as
"data":false,"zero":true" (but note that we can distinguish a
QCOW2_CLUSTER_ZERO_ALLOCATED, which would also have an "offset":
listing).
The task of reconstructing a qcow2 chain was made harder in commit
0da9856851 (nbd: server: Report holes for raw images), because prior
to that point, it was possible to abuse NBD's block status command to
see which portions of a qcow2 file resulted in BDRV_BLOCK_ALLOCATED
(showing up as NBD_STATE_ZERO in isolation) vs. missing from the chain
(showing up as NBD_STATE_ZERO|NBD_STATE_HOLE); but now qemu reports
more accurate sparseness information over NBD.
An obvious solution is to make 'qemu-img map --output=json' add an
additional "present":false designation to any cluster lacking an
allocation anywhere in the chain, without any change to the "depth"
parameter to avoid breaking existing clients. The iotests have
several examples where this distinction demonstrates the additional
accuracy.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210701190655.2131223-3-eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
[eblake: fix more iotest fallout]
Signed-off-by: Eric Blake <eblake@redhat.com>
2021-07-01 22:06:55 +03:00
|
|
|
[{ "start": 0, "length": 67108864, "depth": 0, "present": true, "zero": false, "data": true, "offset": 327680}]
|
2020-07-21 16:55:20 +03:00
|
|
|
|
2020-01-21 18:59:15 +03:00
|
|
|
=== -n -B to an image without a backing file ===
|
|
|
|
|
|
|
|
|
|
Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=67108864
|
|
|
|
|
Formatting 'TEST_DIR/t.IMGFMT.orig', fmt=IMGFMT size=67108864
|
2020-07-06 23:39:46 +03:00
|
|
|
|
|
|
|
|
=== -n incompatible with -o ===
|
|
|
|
|
|
|
|
|
|
qemu-img: -o has no effect when skipping image creation
|
2015-03-19 15:33:33 +03:00
|
|
|
*** done
|