This new test case checks that streaming completes successfully when the
backing file is smaller than the image file.
Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
When the qemu-io --nocache option is used the 039 test case cannot abort
QEMU at a point where the image is dirty. Skip the test case.
Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Image formats with a dirty bit, like qed and qcow2, repair dirty image
files upon open with BDRV_O_RDWR. Performing automatic repair when
qemu-img check runs is not ideal because the bdrv_open() call repairs
the image before the actual bdrv_check() call from qemu-img.c.
Fix this "double repair" since it leads to confusing output from
qemu-img check. Tell the block driver that this image is being opened
just for bdrv_check(). This skips automatic repair and qemu-img.c can
invoke it manually with bdrv_check().
Update the golden output for qemu-iotests 039 to reflect the new
qemu-img check output.
Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Instead of building a huge pipeline, just pass all expressions to a
single sed process.
Suggested-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
qemu-iotests already filters out image creation options that may be
present or not in order to get the same output in both cases. However,
often it only considers the default value of the option. Cover all valid
values instead so that ./check -o name=value can be used successfull for
all of them.
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
This tests establishes the basic post-conditions of the qcow2 lazy
refcounts features:
1. If the image was closed normally, it is marked clean.
2. If an allocating write was performed and the image was not closed
normally, then it is marked dirty.
a. Written data can be read back successfully.
b. The image file can be repaired and will be marked clean again.
c. The image file is automatically repaired when opened read/write.
Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Lazy refcounts is a performance optimization for qcow2 that postpones
refcount metadata updates and instead marks the image dirty. In the
case of crash or power failure the image will be left in a dirty state
and repaired next time it is opened.
Reducing metadata I/O is important for cache=writethrough and
cache=directsync because these modes guarantee that data is on disk
after each write (hence we cannot take advantage of caching updates in
RAM). Refcount metadata is not needed for guest->file block address
translation and therefore does not need to be on-disk at the time of
write completion - this is the motivation behind the lazy refcount
optimization.
The lazy refcount optimization must be enabled at image creation time:
qemu-img create -f qcow2 -o compat=1.1,lazy_refcounts=on a.qcow2 10G
qemu-system-x86_64 -drive if=virtio,file=a.qcow2,cache=writethrough
Update qemu-iotests 031 and 036 since the extension header size changes
when we add feature bit table entries.
Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Hide the default lazy_refcounts=off output from qemu-img like we do with
other image creation options. This ensures that existing golden outputs
continue to pass despite the new option that has been added.
Note that this patch applies before the one that actually introduces the
lazy_refcounts=on|off option. This ensures git-bisect(1) continues to
work.
Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
This patch adds an incompatible feature bit to mark images that have not
been closed cleanly. When a dirty image file is opened a consistency
check and repair is performed.
Update qemu-iotests 031 and 036 since the extension header size changes
when we add feature bit table entries.
Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
The qed.py utility can inspect and manipulate QED image files. It can
be used for testing to see the state of image metadata and also to
inject corruptions into the image file. It also has a scrubbing feature
to copy just the metadata out of an image file, allowing users to share
broken image files without revealing data in bug reports.
This has lived in my local repo for a long time but could be useful
to others.
Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
check -valgrind wraps all qemu-io calls with valgrind. This makes it a
bit easier to debug problems that occur somewhere deep in a test case.
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
This new test validates the autoclear feature bit behavior. When QEMU
opens a qcow2v3 image file with an unknown autoclear feature bit the bit
should be cleared in the image file header.
Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
This new command sets feature bits in the image file header:
qcow2.py set-feature-bit incompatible|compatible|autoclear <bit>
The bit number must be in the range [0, 64).
Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
This one is a bit more interesting. The COW operation isn't performed
completely synchronously, and therefore dependencies must be handled
correctly when multiple requests write to the same unallocated cluster.
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Looks like we're still missing these very basic tests for backing file
handling.
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
This way, they will not execute any VM code at all. However, right now
the cancellation test is "relying" on being slowed down by TCG executing
BIOS code. So, change the timeouts.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
The TestStreamStop test case is racy; if the job completes before we can
cancel it, it fails. If we remove the sleep the job will be canceled
before it has even started, and the test succeeds but it is also not
testing anything interesting.
But if the image is left sparse, then the job has really nothing to do.
For qcow2 it will read one L2-table, for raw it will issue a bunch of
ioctls. This also falls under "not testing anything interesting", and
this may be happening right now (depending on the filesystem) since the
file protocol got an is_allocated method.
Filling the test image with data ensures that the test covers the
intended case. It also slows down the test, which will be particularly
important after the next patch.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
The 035 parallel aio write test relies on knowledge of qcow2 metadata
layout to stress parallel L2 table accesses. This only works for qcow2
unless we add additional calculations for qed or other formats.
Mark this test as qcow2-only.
Note that the test is strictly speaking non-deterministic although the
output produced is reliable with qcow2. This is because the aio_write
command returns before the aio write request has completed. Completions
can occur at any time afterwards and cause a message to be printed.
Therefore the exact output of this test is not deterministic but we seem
to get away with it for qcow2 (maybe due to coroutine and main loop
scheduling).
Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Unallocated sectors should really never be accessed by the guest,
so there's no need to copy them during the streaming process.
If they are read by the guest during streaming, guest-initiated
copy-on-read will copy them (we're in the base == NULL case, which
enables copy on read). If they are read after we disconnect the
image from the base, they will read as zeroes anyway.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
The test on sectors not allocated can fail if the L1/L2 tables are
not on disk yet. Allow tests to shutdown the VM early.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
qemu-io's cvtstr function sometimes will incorrectly omit the
decimal part of the number, and sometimes will incorrectly include
it. This patch fixes both. The former is more serious, and can
be seen in the patches to 027.out and 033.out.
The changes to all other files were scripted with sed, so there were
no "surprises" beyond 027.out and 033.out.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
A trailing space is left when qemu-img has no arguments, for example if
-nocache is not used. This becomes an empty argument after split()
and causes qemu-io to fail.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
This test case manages to let qcow2 abort because its cache is used up
and it can't find free cache entries for new requests any more.
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
We added image fragmentation statistics functions to qemu-img several days
ago, those patches will cause "./check -qed" failed. This patch will ignore
fragmentation statistics information of qed format, and then "./check -qed"
will work.
Signed-off-by: Dong Xu Wang <wdongxu@linux.vnet.ibm.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Add tests to exercise the InvalidParameter 'speed' error code path, as
well as the regular success case for setting the speed. The
block-stream 'speed' parameter allows the speed limit of the job to be
applied immediately when the job starts instead of issuing a separate
block-job-set-speed command later. If the parameter has an invalid
value we expect to get an error and the job is not created.
It turns out that cancelling a block job is a common operation in these
test cases, let's extract a cancel_and_wait() function instead of
duplicating the QMP commands.
Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Acked-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
The block streaming and job commands used '_' instead of '-' for reasons
of compatibility with libvirt, which already included support for the
'_' naming. However, the semantics of block_job_cancel have changed and
libvirt now needs to handle the new semantics.
Since the old semantics were never in a QEMU release we can still rename
the commands to use '-' instead of '_'. Libvirt is also happy because
the new name can be used to distinguish QEMU binaries that support the
latest block-job-cancel semantics from those that include a downstream
block_job_cancel command.
Therefore, let's apply the QAPI/QMP naming rules to the block streaming
and job commands. QEMU 1.1 will be the first release with these
commands so no upstream users can break.
Note that HMP commands are left with '_' because that is the convention
there.
Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
qcow2.py must be updated to work with version 3 images at all, the
output has changed since the feature table extension has been added, and
version 2 and version 3 images can't possibly have the same test output.
Change the test case to completely ignore IMGOPTS and run the test for
both compat=1.1 and compat=0.10 regardless of the ./check command line.
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
This adds an -o option to qemu-iotests, which is an option string that
is passed through to qemu-img create -o... This allows testing different
subformat with a command like './check -qcow2 -o compat=0.10'.
For qcow2, if no compat option is specified, compat=1.1 is the new
default.
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
`hostname -s` may output an errror:
hostname: Name or service not known
This causes all tests to fail for `make check-block`.
Suppress such error messages, letting the tests succeed.
Signed-off-by: Andreas Färber <afaerber@suse.de>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
According comment, we should not read again, we will write.
Signed-off-by: Dong Xu Wang <wdongxu@linux.vnet.ibm.com>
Reviewed-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
If the BlockDriverState is closed/freed without draining the AIO
requests first, the request coroutines may work on invalid data and file
descriptors or have some dangling pointers that cause segfaults.
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Some image formats do have a cluster size, others don't, but there are
tests that work with both sets of images and currently we get failures
because the qemu-img create output doesn't mention the cluster size for
some formats.
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
qemu-io requires options first, then fixed parameters.
GNU getopt also allows options at the end, but POSIX getopt
doesn't. Try "export POSIXLY_CORRECT=y" to get the POSIX
behaviour with GNU getopt, too.
Cc: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Stefan Weil <sw@weilnetz.de>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
qemu-img requires first options, then file name, then size.
GNU getopt also allows options at the end, but POSIX getopt
doesn't. Try "export POSIXLY_CORRECT=y" to get the POSIX
behaviour with GNU getopt, too.
Cc: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Stefan Weil <sw@weilnetz.de>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
The immportant thing here is that header extensions don't get silently
dropped when the header is rewritten, e.g. during a rebase.
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
This adds a tool that is meant to inspect and edit qcow2 files in a
low-level way, that wouldn't be possible with qemu-img/io, for example
by adding yet unknown extensions or flags. This way we can test whether
qemu deals properly with future backwards compatible extensions.
For now, let's start with the image header and header extensions.
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
This creates a new test group 'quick' for some test case that take at
most a couple of seconds each, so that the group can be run during a
quick 'make check'
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
This tests that qemu-img rebase doesn't assume that the backing file has
the same size as the image, but considers that it can be smaller.
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
qemu-iotests supports the -nocache option which makes the tests run with
cache=none. For blkdebug tests with qcow2 this means that we may see
test results that differ from cache=writethrough. This patch makes the
diff a bit smaller and therefore easier to review.
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
This one makes it possible to run qemu-iotests on a Windows build using Wine
and get somewhat meaningful results.
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
This patch adds a test suite for the image streaming feature. It
exercises the 'block_stream', 'block_job_cancel', 'block_job_set_speed',
and 'query-block-jobs' QMP commands.
Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Block layer tests that involve QMP commands rather than qemu-img or
qemu-io are not well-suited for shell scripting. This patch adds a
Python module which allows tests to be written in Python instead.
The basic API is:
VM - class for launching and interacting with a VM
QMPTestCase - abstract base class for tests that use QMP
qemu_img() - wrapper function for invoking qemu-img
qemu_io() - wrapper function for invoking qemu-io
imgfmt - the image format under test (e.g. qcow2, qed)
test_dir - scratch directory path for temporary files
main() - entry point for running tests
Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Since qemu-iotests may need to create large image files it is possible
to specify the test directory. The TEST_DIR variable needs to be
exported so non-bash tests can make use of it.
Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Since we might want to test arbitrary qemu, qemu-img and
qemu-io paths, allow users to specify environment variable
values for QEMU_PROG, QEMU_IMG_PROG and QEMU_IO_PROG so
the testsuite will use those values rather than find them
on PATH. Obviously, if such env variables are not set
prior to script execution, normal detection mechanism
takes place.
Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
Signed-off-by: Christoph Hellwig <hch@lst.de>
Print the paths of the programs under test
(qemu, qemu-img and qemu-io).
Signed-off-by: Lucas Meneghel Rodrigues <lmr@redhat.com>
Signed-off-by: Christoph Hellwig <hch@lst.de>
