Commit Graph

64 Commits

Author SHA1 Message Date
Thomas Huth
733dafe491 iotests: Select a default machine for the rx and avr targets
If you are building only with either the new rx-softmmu or avr-softmmu
target, "make check-block" fails a couple of tests since there is no
default machine defined in these new targets. We have to select a machine
in the "check" script for these, just like we already do for the arm- and
tricore-softmmu targets.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20200722161908.25383-1-thuth@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-07-27 12:37:25 +02:00
Marc-André Lureau
a08464521c Remove VXHS block device
The vxhs code doesn't compile since v2.12.0. There's no point in fixing
and then adding CI for a config that our users have demonstrated that
they do not use; better to just remove it.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20200711065926.2204721-1-marcandre.lureau@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-07-17 14:20:57 +02:00
Kevin Wolf
c1eafd27b1 iotests: vmdk: Enable zeroed_grained=on by default
In order to avoid bitrot in the zero cluster code in VMDK, enable
zeroed_grain=on by default for the tests.

059 now unsets the default options because zeroed_grain=on works only
with some subformats and the test case tests many different subformats,
including those for which it doesn't work.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20200430133007.170335-7-kwolf@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-05-08 13:26:35 +02:00
Philippe Mathieu-Daudé
351aa2706e tests/qemu-iotests/check: Only check for Python 3 interpreter
All the iotests Python scripts have been converted to search for
the Python 3 interpreter. Update the ./check script accordingly.

Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20200130163232.10446-13-philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:15:16 +01:00
Philippe Mathieu-Daudé
15b015690b tests/qemu-iotests/check: Allow use of python3 interpreter
As we want to enforce a unique and explicit Python 3 interpreter,
we need let this script handle 'python3' too.

Suggested-by: Kevin Wolf <kwolf@redhat.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20200130163232.10446-3-philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:12:48 +01:00
Thomas Huth
cd2058289b iotests: Skip Python-based tests if QEMU does not support virtio-blk
We are going to enable some of the python-based tests in the "auto" group,
and these tests require virtio-blk to work properly. Running iotests
without virtio-blk likely does not make too much sense anyway, so instead
of adding a check for the availability of virtio-blk to each and every
test (which does not sound very appealing), let's rather add a check for
this a central spot in the "check" script instead (so that it is still
possible to run "make check" for qemu-system-tricore for example).

Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-id: 20200121095205.26323-6-thuth@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-02-06 13:47:45 +01:00
Aarushi Mehta
7156ca4888 tests/qemu-iotests: enable testing with aio options
Signed-off-by: Aarushi Mehta <mehta.aaru20@gmail.com>
Acked-by: Stefano Garzarella <sgarzare@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 20200120141858.587874-15-stefanha@redhat.com
Message-Id: <20200120141858.587874-15-stefanha@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2020-01-30 21:01:40 +00:00
Philippe Mathieu-Daudé
75ab574b4c tests/qemu-iotests: Update tests to recent desugarized -accel option
Commit 6f6e1698a6 desugarized "-machine accel=" to a list
of "-accel" options. Since now "-machine accel" and "-accel"
became incompatible, update the iotests to the new format.

Error reported here:
https://gitlab.com/qemu-project/qemu/-/jobs/385801004#L3400

Reported-by: GitLab CI
Fixes: 6f6e1698a6 (vl: configure accelerators from -accel options)
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-id: 20200106130951.29873-1-philmd@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-01-06 14:14:41 +01:00
Kevin Wolf
d44abcc0f7 iotests: Fix "no qualified output" error path
The variable for error messages to be displayed is $results, not
$reason. Fix 'check' to print the "no qualified output" error message
again instead of having a failure without any message telling the user
why it failed.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
2019-11-18 17:50:08 +01:00
Max Reitz
c7df3f19d2 iotests: Introduce $SOCK_DIR
Unix sockets generally have a maximum path length.  Depending on your
$TEST_DIR, it may be exceeded and then all tests that create and use
Unix sockets there may fail.

Circumvent this by adding a new scratch directory specifically for
Unix socket files.  It defaults to a temporary directory (mktemp -d)
that is completely removed after the iotests are done.

(By default, mktemp -d creates a /tmp/tmp.XXXXXXXXXX directory, which
should be short enough for our use cases.)

Use mkdir -p to create the directory (because it seems right), and do
the same for $TEST_DIR (because there is no reason for that to be
created in any different way).

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-id: 20191017133155.5327-2-mreitz@redhat.com
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-10-28 11:22:30 +01:00
Kevin Wolf
c69719fcad iotests: Require Python 3.6 or later
Running iotests is not required to build QEMU, so we can have stricter
version requirements for Python here and can make use of new features
and drop compatibility code earlier.

This makes qemu-iotests skip all Python tests if a Python version before
3.6 is used for the build.

Suggested-by: Eduardo Habkost <ehabkost@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
2019-10-04 11:59:16 +02:00
Thomas Huth
4ee5f4be4f tests/qemu-iotests/check: Replace "tests" with "iotests" in final status text
When running "make check -j8" or something similar, the iotests are
running in parallel with the other tests. So when they are printing
out "Passed all xx tests" or a similar status message at the end,
it might not be quite clear that this message belongs to the iotests,
since the output might be mixed with the other tests. Thus change the
word "tests" here to "iotests" instead to avoid confusion.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-id: 20190906113920.11271-1-thuth@redhat.com
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-09-16 14:48:30 +02:00
Max Reitz
9da126fc2e iotests: Add -display none to the qemu options
Without this argument, qemu will print an angry message about not being
able to connect to a display server if $DISPLAY is not set.  For me,
that breaks iotests.supported_formats() because it thus only sees
["Could", "not", "connect"] as the supported formats.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-id: 20190819201851.24418-2-mreitz@redhat.com
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-09-03 14:56:06 +02:00
Thomas Huth
bf75d75275 tests/qemu-iotests/check: Allow tests without groups
The regular expressions in the "check" script currently expect that there
is always a space after the test number in the group file, so you can't
have a test in there without a group unless the line still ends with a
space - which is quite error prone since some editors might remove spaces
at the end of lines automatically.
Thus let's fix the regular expressions so that it is also possible to
have lines with one test number only in the group file.

Suggested-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20190717111947.30356-2-thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-07-23 15:53:25 +01:00
Max Reitz
835d689d25 iotests: Hide timestamps for skipped tests
Currently, the "thistime" variable is not reinitialized on every loop
iteration.  This leads to tests that do not yield a run time (because
they failed or were skipped) printing the run time of the previous test
that did.  Fix that by reinitializing "thistime" for every test.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-06-18 16:41:09 +02:00
Kevin Wolf
11ba81c3cd iotests: Fix duplicated diff output on failure
Commit 70ff5b07 wanted to move the diff between actual and reference
output to the end after printing the test result line. It really only
copied it, though, so the diff is now displayed twice. Remove the old
one.

Fixes: 70ff5b07fc
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-06-04 16:55:58 +02:00
Alex Bennée
70ff5b07fc tests/qemu-iotests: re-format output to for make check-block
This attempts to clean-up the output to better match the output of the
rest of the QEMU check system when called with -makecheck. This includes:

  - formatting as "  TEST    iotest-FMT: nnn"
  - only dumping config on failure (when -makecheck enabled)

The non-make check output has been cleaned up as well:

  - line re-displayed (\r) at the end
  - fancy colours for pass/fail/skip
  - timestamps always printed (option removed)

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20190503143904.31211-1-alex.bennee@linaro.org>
Tested-by: Thomas Huth <thuth@redhat.com>
2019-05-28 10:28:51 +01:00
Thomas Huth
4a715461c8 tests/qemu-iotests/check: Pick a default machine if necessary
qemu-system-arm, qemu-system-aarch64 and qemu-system-tricore do not have
a default machine, so when running the qemu-iotests with such a binary,
lots of tests are failing. Fix it by picking a default machine in the
"check" script instead.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Acked-by: Christian Borntraeger <borntraeger@de.ibm.com>
Message-Id: <20190502084506.8009-3-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2019-05-21 10:07:29 +02:00
Philippe Mathieu-Daudé
11a82d1429 qemu-iotests: Improve portability by searching bash in the $PATH
Bash is not always installed as /bin/bash. In particular on OpenBSD,
the package installs it in /usr/local/bin.
Use the 'env' shebang to search bash in the $PATH.

Patch created mechanically by running:

  $ git grep -lE '#! ?/bin/bash' -- tests/qemu-iotests \
    | while read f; do \
      sed -i 's|^#!.\?/bin/bash$|#!/usr/bin/env bash|' $f; \
    done

Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-03-08 12:26:45 +01:00
Andrey Shinkevich
57ed557f03 iotests: ask QEMU for supported formats
Supported formats listed by 'qemu' may differ from those listed by
'qemu-img' due to whitelists. Some test cases require specific formats
that may be used with qemu. They can be inquired directly by running
'qemu -drive format=help'. The response takes whitelists into account.
The method supported_formats() serves for that. The method decorator
skip_if_unsupported() checks if all requested formats are whitelisted.
If not, the test case will be skipped. That has been implemented in
the 'check' file in the way similar to the 'test notrun' mechanism.

Suggested-by: Roman Kagan <rkagan@virtuozzo.com>
Suggested-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Suggested-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-03-08 12:26:45 +01:00
yuchenlin
76f1cf0a5e qemu-iotests: add test case for dmg
Recently, some bugs in dmg file have been fixed. To prevent reading dmg
is broken someday in the future, add a simple test which ensures the
conversion from dmg to raw should not hang or face any I/O error.

Signed-off-by: yuchenlin <npes87184@gmail.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2019-02-01 13:46:44 +01:00
Mao Zhongyi
4a9e751f61 qemu-iotests: Modern shell scripting (use $() instead of ``)
Various shell files contain a mix between obsolete ``
and modern $(); It would be nice to convert to using
$() everywhere.  For now, just do the qemu-iotests directory.

Cc: kwolf@redhat.com
Cc: mreitz@redhat.com
Cc: eblake@redhat.com
Suggested-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Mao Zhongyi <maozhongyi@cmss.chinamobile.com>
Message-Id: <20181024094051.4470-4-maozhongyi@cmss.chinamobile.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[eblake: tweak commit message]
Signed-off-by: Eric Blake <eblake@redhat.com>
2018-11-19 10:08:19 -06:00
Mao Zhongyi
e8d81a61e1 qemu-iotests: convert pwd and $(pwd) to $PWD
POSIX requires $PWD to be reliable, and we expect all
shells used by qemu scripts to be relatively close to
POSIX.  Thus, it is smarter to avoid forking the pwd
executable for something that is already available in
the environment.

So replace it with the following:

sed -i 's/\(`pwd`\|\$(pwd)\)/$PWD/g' $(git grep -l pwd)

Then delete a pointless line assigning PWD to itself.

Cc: kwolf@redhat.com
Cc: mreitz@redhat.com
Cc: eblake@redhat.com
Suggested-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Mao Zhongyi <maozhongyi@cmss.chinamobile.com>
Message-Id: <20181024094051.4470-2-maozhongyi@cmss.chinamobile.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[eblake: touch up commit message, reorder series, tweak a couple more files]
Signed-off-by: Eric Blake <eblake@redhat.com>
2018-11-19 10:08:19 -06:00
Lukáš Doktor
96914159b7 qemu-iotests: Use ppc64 qemu_arch on ppc64le host
The qemu target does not always correspond to the host machine type. For
example ppc64le machine target is ppc64. Let's introduce "qemu_arch"
variable to store the matching qemu architecture related to the current
architecture and use it when auto-detecting the default qemu binary.

Signed-off-by: Lukáš Doktor <ldoktor@redhat.com>
Message-id: 20180329112053.5399-2-ldoktor@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2018-04-03 17:39:37 +02:00
Eric Blake
63ca8406be iotests: Avoid realpath, for CentOS 6
CentOS 6 lacks a realpath binary on the base install, which makes
all iotests runs fail since the 2.11 release:

001         - output mismatch (see 001.out.bad)
./check: line 815: realpath: command not found
diff: missing operand after `/home/dummy/qemu/tests/qemu-iotests/001.out'
diff: Try `diff --help' for more information.

Many of the uses of 'realpath' in the check script were being
used on the output of 'type -p' - but that is already an
absolute file name.  While a canonical name can often be
shorter (realpath gets rid of /../), it can also be longer (due
to symlink expansion); and we really don't care if the name is
canonical, merely that it was an executable file with an
absolute path.  These were broken in commit cceaf1db.

The remaining use of realpath was to convert a possibly relative
filename into an absolute one before calling diff to make it
easier to copy-and-paste the filename for moving the .bad file
into place as the new reference file even when running iotests
out-of-tree (see commit 93e53fb6), but $PWD can achieve the same
purpose.

Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Jeff Cody <jcody@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2018-03-19 12:01:39 +01:00
Kevin Wolf
e1473133f7 qemu-iotests: Enable write tests for parallels
Originally we added parallels as a read-only format to qemu-iotests
where we did just some tests with a binary image. Since then, write and
image creation support has been added to the driver, so we can now
enable it in _supported_fmt generic.

The driver doesn't support migration yet, though, so we need to add it
to the list of exceptions in 181.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Jeff Cody <jcody@redhat.com>
2018-03-19 12:01:39 +01:00
Eric Blake
8803714b53 iotests: Restore stty settings on completion
Executing qemu with a terminal as stdin will temporarily alter stty
settings on that terminal (for example, disabling echo), because of
how we run both the monitor and any multiplexing with guest input.
Normally, qemu restores the original settings on exit; but if an
iotest triggers qemu to abort in the middle, we can be left with
the altered terminal setup.  This can make life very annoying when
debugging an iotest failure (not everyone remembers the trick of
blind-typing 'stty sane' without echo, and some people prefer
terminal settings that are slightly different than the defaults
picked by 'stty sane').

It is possible to avoid qemu corrupting the terminal by not passing
a terminal to qemu's stdin in the first place (as in, use
'./check ... </dev/null'), but that's extra typing to have to
remember.  But running 'exec </dev/null' in the harness seems like
it might be too heavy of a hammer.  So I instead went the the
solution of saving and restoring the stty settings, only when the
harness detects that it is run interactively.

I tested this patch by forcing an allocation failure (I can't
guarantee that this particular limit will work on all setups, but
it shows the idea):
 $ (ulimit -S -v 500000; ./check -qcow2 1)

Signed-off-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2017-10-06 16:28:58 +02:00
Paolo Bonzini
09d653e617 qemu-iotests: merge "check" and "common"
"check" is full of qemu-iotests--specific details.  Separating it
from "common" does not make much sense anymore.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2017-10-06 16:28:58 +02:00
Paolo Bonzini
4e670492ef qemu-iotests: get rid of $iam
The variable is almost unused, and one of the two uses is actually
uninitialized.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2017-10-06 16:28:58 +02:00
Paolo Bonzini
8f4dcaba9b qemu-iotests: fix uninitialized variable
The variable is used in "common" but defined only after the file
is sourced.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2017-10-06 16:28:58 +02:00
Paolo Bonzini
3817ce03bf qemu-iotests: do not include common.rc in "check"
It only provides functions used by the test programs.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2017-10-06 16:28:58 +02:00
Paolo Bonzini
cceaf1db6f qemu-iotests: cleanup and fix search for programs
Instead of ./check failing when a binary is missing, we try each test
case now and each one fails with tons of test case diffs.  Also, all the
variables were initialized by "check" prior to "common" being sourced,
and then (uselessly) checked for emptiness again in "check".

Centralize the search for programs in "common" (which will soon be
one with "check"), including the "realpath" invocation which can be done
just once in "check" rather than in the tests.

For qnio_server, move the detection to "common", simplifying
set_prog_path to stop handling the unused second argument, and
embedding the "realpath" pass.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2017-10-06 16:28:58 +02:00
Paolo Bonzini
9ee4b6f803 qemu-iotests: get rid of AWK_PROG
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2017-10-06 16:28:58 +02:00
Paolo Bonzini
f06a8dcfc6 qemu-iotests: remove dead code
This includes shell function, shell variables and command line options
(randomize.awk does not exist).

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2017-10-06 16:28:58 +02:00
Fam Zheng
93e53fb695 iotests: Print full path of bad output if mismatch
So it is easier to copy paste the path.

Signed-off-by: Fam Zheng <famz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2017-09-26 14:46:23 +02:00
Cleber Rosa
a3e08c2dbd qemu-iotests: remove comment about root privileges requirement
The check script contains a commented out root user requirement,
probably because of its xfstests heritage.  This requirement doesn't
apply to qemu-iotests, so it better be gone.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>
2017-08-14 13:06:54 +03:00
Cleber Rosa
80758ec57d qemu-iotests: remove commented out variables
The variables FULL_MKFS_OPTIONS and FULL_MOUNT_OPTIONS are commented
out, never used, and even refer to functions that do exist.  The last
time these were touched was around 8 years ago, so I guess it's safe
to assume outputting such information on test execution is still on the
radar.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>
2017-08-14 13:06:54 +03:00
Cleber Rosa
657c572afe qemu-iotests: get rid of _full_imgproto_details()
Although this function is used, its implementation does nothing
besides echoing a variable name.  There's no need to wrap this
functionality in a function, and based on the one usage it has, it's
not even required to adhere to a convention or code style.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>
2017-08-14 13:06:54 +03:00
Stefan Hajnoczi
217a0683b7 qemu-iotests: support per-format golden output files
Some tests produce format-dependent output.  Either the difference is
filtered out and ignored, or the test case is format-specific so we
don't need to worry about per-format output differences.

There is a third case: the test script is the same for all image formats
and the format-dependent output is relevant.  An ugly workaround is to
copy-paste the test into multiple per-format test cases.  This
duplicates code and is not maintainable.

This patch allows test cases to add per-format golden output files so a
single test case can work correctly when format-dependent output must be
checked:

  123.out.qcow2
  123.out.raw
  123.out.vmdk
  ...

This naming scheme is not composable with 123.out.nocache or 123.pc.out,
two other scenarios where output files are split.  I don't think it
matters since few test cases need these features.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Message-id: 20170705125738.8777-9-stefanha@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-07-11 17:45:01 +02:00
Eric Blake
b43671f80c tests: Avoid non-portable 'echo -ARG'
POSIX says that backslashes in the arguments to 'echo', as well as
any use of 'echo -n' and 'echo -e', are non-portable; it recommends
people should favor 'printf' instead.  This is definitely true where
we do not control which shell is running (such as in makefile snippets
or in documentation examples).  But even for scripts where we
require bash (and therefore, where echo does what we want by default),
it is still possible to use 'shopt -s xpg_echo' to change bash's
behavior of echo.  And setting a good example never hurts when we are
not sure if a snippet will be copied from a bash-only script to a
general shell script (although I don't change the use of non-portable
\e for ESC when we know the running shell is bash).

Replace 'echo -n "..."' with 'printf %s "..."', and 'echo -e "..."'
with 'printf %b "...\n"', with the optimization that the %s/%b
argument can be omitted if the string being printed is a strict
literal with no '%', '$', or '`' (we could technically also make
this optimization when there are $ or `` substitutions but where
we can prove their results will not be problematic, but proving
that such substitutions are safe makes the patch less trivial
compared to just being consistent).

In the qemu-iotests check script, fix unusual shell quoting
that would result in word-splitting if 'date' outputs a space.

In test 051, take an opportunity to shorten the line.

In test 068, get rid of a pointless second invocation of bash.

CC: qemu-trivial@nongnu.org
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-id: 20170703180950.9895-1-eblake@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-07-11 17:45:00 +02:00
Daniel P. Berrange
36bd422812 iotests: record separate timings per format,protocol pair
The 'check' program records timings for each test that
is run. These timings are only valid, however, for a
particular format/protocol combination. So if frequently
running 'check' with a variety of different formats or
protocols, the times printed can be very misleading.

Instead of having a single 'check.time' file, maintain
multiple 'check.time-$IMGPROTO-$IMGFMT' files.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Message-id: 20170103160556.9895-1-berrange@redhat.com
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2017-02-12 00:47:42 +01:00
Sascha Silbe
0145b4e130 qemu-iotests: check: don't place files with predictable names in /tmp
Placing files with predictable or even hard-coded names in /tmp is a
security risk and can prevent or disturb operation on a multi-user
machine. Place them inside the "scratch" directory instead, as we
already do for most other test-related files.

Signed-off-by: Sascha Silbe <silbe@linux.vnet.ibm.com>
Reviewed-by: Bo Tu <tubo@linux.vnet.ibm.com>
Message-id: 1459848109-29756-2-git-send-email-silbe@linux.vnet.ibm.com
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2016-04-12 18:07:39 +02:00
Max Reitz
934659c460 iotests: Do not suppress segfaults in bash tests
Currently, if a qemu/qemu-io/qemu-img/qemu-nbd invocation receives a
segmentation fault, that message is invisible in most cases since the
output is generally filtered and bash suppresses the segmentation fault
notice for any but the last element of a pipe.

Most of the time, the test will then fail anyway because of missing
output, but not necessarily (as happened with test 82 recently).

Fix this by making the corresponding environment variables point to
wrapper functions which execute the respective command in a subshell.

Giving options to qemu/qemu-io/qemu-img and path names with spaces were
broken for the Python tests; this patch "accidentally" fixes that.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2015-09-04 20:59:48 +02:00
Bo Tu
e166b41482 qemu-iotests: qemu machine type support
This patch adds qemu machine type support to the io test suite.
Based on the qemu default machine type and alias of the default machine
type the reference output file can now vary from the default to a
machine specific output file if necessary. When using a machine specific
reference file if the default machine has an alias then use the alias as the output
file name otherwise use the default machine name as the output file name.

Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Michael Mueller <mimu@linux.vnet.ibm.com>
Reviewed-by: Sascha Silbe <silbe@linux.vnet.ibm.com>
Signed-off-by: Xiao Guang Chen <chenxg@linux.vnet.ibm.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2015-09-04 20:59:48 +02:00
Fam Zheng
aa4f592a1d qemu-iotests: Make debugging python tests easier
Adding "-d" option. The output goes to "tee" so it appears in your
console. Also, raise the verbosity of unnitest runner.

When testing a topic branch, it's possible that a bug introduced by a
code change makes the python test case hang, with debug output, it is
much easier to locate the problem.

This can also be helpful if you want to watch the progress of a python
test, it offers you a way to sense the speed of each test case method
you're writing.

Note: because there is no easy way to get *both* the verbose output and
the output expected by ./check comparison, the case would always fail
with an "output mismatch". The sole purpose of using this option is
giving developers a quick way to debug when things go wrong.

Signed-off-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2015-05-22 17:08:01 +02:00
Fam Zheng
b8aff7d6bf qemu-iotests: Speed up make check-block
Using /tmp, which is usually mounted as tmpfs, the quick group can be
quicker.

On my laptop (Lenovo T430s with Fedora 20), this reduces the time from
50s to 30s.

Signed-off-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2015-01-13 11:47:55 +00:00
Max Reitz
ea81ca9de1 iotests: Use $PYTHON for Python scripts
Instead of invoking Python scripts directly via ./, use $PYTHON to
obtain the correct Python interpreter command.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Fam Zheng <famz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2014-06-27 20:00:00 +02:00
Max Reitz
7fed1a49ff iotests: Source common.env
Source common.env in the iotests' check script.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2014-06-27 20:00:00 +02:00
Max Reitz
e8f8624d3b iotests: Allow out-of-tree run
As out-of-tree builds are preferred for qemu, running the qemu-iotests
in that out-of-tree build should be supported as well. To do so, a
symbolic link has to be created pointing to the check script in the
source directory. That script will check whether it has been run through
a symlink, and if so, will assume it is run in the build tree. All
output and temporary operations performed by iotests are then redirected
here and, unless specified otherwise by the user, QEMU_PROG etc. will be
set to paths appropriate for the build tree.

Also, drop making every test case executable if it is not yet, as this
would modify the source tree which is not desired for out-of-tree runs
and should be fixed in the repository anyway.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2014-06-27 20:00:00 +02:00
Peter Maydell
7d983531c6 Revert "iotests: Use configured python"
This reverts commit f915db07ef.

This commit is broken because it does not account for the
build tree and the source tree being different, and can cause
build failures for out-of-tree builds. Revert it until we can
identify a better solution to the problem.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Message-id: 1400153676-30180-1-git-send-email-peter.maydell@linaro.org
Acked-by: Kevin Wolf <kwolf@redhat.com>
2014-05-15 13:41:19 +01:00