Commit Graph

5967 Commits

Author SHA1 Message Date
Stefan Hajnoczi
8c6b0356b5 util/async: make bh_aio_poll() O(1)
The ctx->first_bh list contains all created BHs, including those that
are not scheduled.  The list is iterated by the event loop and therefore
has O(n) time complexity with respected to the number of created BHs.

Rewrite BHs so that only scheduled or deleted BHs are enqueued.
Only BHs that actually require action will be iterated.

One semantic change is required: qemu_bh_delete() enqueues the BH and
therefore invokes aio_notify().  The
tests/test-aio.c:test_source_bh_delete_from_cb() test case assumed that
g_main_context_iteration(NULL, false) returns false after
qemu_bh_delete() but it now returns true for one iteration.  Fix up the
test case.

This patch makes aio_compute_timeout() and aio_bh_poll() drop from a CPU
profile reported by perf-top(1).  Previously they combined to 9% CPU
utilization when AioContext polling is commented out and the guest has 2
virtio-blk,num-queues=1 and 99 virtio-blk,num-queues=32 devices.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Message-id: 20200221093951.1414693-1-stefanha@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2020-02-22 08:26:47 +00:00
Paolo Bonzini
8c3570e339 rcu_queue: add QSLIST functions
QSLIST is the only family of lists for which we do not have RCU-friendly accessors,
add them.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 20200220103828.24525-1-pbonzini@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2020-02-22 08:26:47 +00:00
Peter Maydell
9e6b7f7eb8 Block patches:
- qemu-img convert: New --target-is-zero parameter
 - qcow2: Specify non-default compression type flag
 - optionally flat output for query-named-block-nodes
 - some fixes
 - pseudo-creation of images on block devices is now done by a generic
   block layer function
 -----BEGIN PGP SIGNATURE-----
 
 iQFGBAABCAAwFiEEkb62CjDbPohX0Rgp9AfbAGHVz0AFAl5Ork4SHG1yZWl0ekBy
 ZWRoYXQuY29tAAoJEPQH2wBh1c9A7XYH/RFs+kxf+v9V1qE1rwdbJCXdA36/JIoF
 AW6CtPk47vEodvssjgPBygs/GBqQDdkpZKyA42ZxrE3dPeyRCusncYKRRFPbP4Oq
 pXj/ZYy9YBn0LIqK41kmXgMdm9HZ113WcG5/rNuh5DOtwX7e688tmHXOHbT8MKAR
 q/nBK1AVNVxFyBRnRow/YK1jGXmA5rIoONVd6Spkj/5xmKZJ6RtpQb1+WnD7HI3j
 TVvDcEnKGCT/qZ73rJ6UFRIUdllAQ7fJAOLnXYFwcwkErq3fMEwLpq3syuiuks+B
 KoFlUzCsM3xiASGnz5MgZI7EVhjGzruneVoBrSyhjlRqNaOB2pYeBwQ=
 =4/ZP
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2020-02-20' into staging

Block patches:
- qemu-img convert: New --target-is-zero parameter
- qcow2: Specify non-default compression type flag
- optionally flat output for query-named-block-nodes
- some fixes
- pseudo-creation of images on block devices is now done by a generic
  block layer function

# gpg: Signature made Thu 20 Feb 2020 16:05:34 GMT
# gpg:                using RSA key 91BEB60A30DB3E8857D11829F407DB0061D5CF40
# gpg:                issuer "mreitz@redhat.com"
# gpg: Good signature from "Max Reitz <mreitz@redhat.com>" [full]
# Primary key fingerprint: 91BE B60A 30DB 3E88 57D1  1829 F407 DB00 61D5 CF40

* remotes/maxreitz/tags/pull-block-2020-02-20:
  iotests: Test snapshot -l field separation
  block: Fix VM size field width in snapshot dump
  iotests: Test convert -n -B to backing-less target
  qemu-img: Fix convert -n -B for backing-less targets
  iotests: Add test for image creation fallback
  iscsi: Drop iscsi_co_create_opts()
  file-posix: Drop hdev_co_create_opts()
  block: Generic file creation fallback
  block/nbd: Fix hang in .bdrv_close()
  iotests/279: Fix for non-qcow2 formats
  block/backup-top: fix flags handling
  block: always fill entire LUKS header space with zeros
  qemu-img: Add --target-is-zero to convert
  qapi: Allow getting flat output from 'query-named-block-nodes'
  iotests/147: Fix drive parameters
  iotests: Remove the superfluous 2nd check for the availability of quorum
  docs: qcow2: introduce compression type feature
  docs: improve qcow2 spec about extending image header

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-02-21 11:24:57 +00:00
Peter Maydell
7afee874f1 Fix memory leak with fdt
cosmetic change in code and logs
 update mailmap
 -----BEGIN PGP SIGNATURE-----
 
 iQJGBAABCAAwFiEEzS913cjjpNwuT1Fz8ww4vT8vvjwFAl5NCtwSHGxhdXJlbnRA
 dml2aWVyLmV1AAoJEPMMOL0/L748GMMQAIX/UNIKHIIYUYM/5mrBQjQzk/OTZ0F8
 7GqsYkOJ3U189iZsbMTCkCo3uQjP1+qKBRGhEq/hWozItTpvKwwfg6V5e8/8UUYn
 CREUZVBnSj2GcBHCSyK0+h6carG/ThuRc5m6QFPG4OMKDR6FTbvleeUTWhgPAJ3t
 /jhIj9+MlKYFJ9VmMRYs47ZMGoX0KDX1m9L6fx7pllwHM3DbFF5L1GieHrdnnDYu
 7az+KZsFkvIu2yuyonzIApitriRjPmb5DM2udxRoX2ZxhkZU0OARzQVxfx+rsilJ
 puuD1mc8PwQTbdAFi0y/5IvL7OkX7lV09cxnBk4wLoxKsJFn+2F0YN+iMVkcY+B3
 BKr2uwikvolb5BQgfpZg7bJboHABuHy+PGBiX3EC/F93k11bIIF/XM2bEz/Kwt3K
 cY7DLDxEQfEcyuHeFdmT23qE0xyrQytkgIU6Vefp2YuOvpeNNVC7KNnrnIfo32dt
 TvoRFCoW2s3ALevoBHBHmbbZzFGjvBV2377kiA8yDTb0aHsnzXeptVYojFVG76Td
 qpy6e/OWnhh08FB4HTTeWx2VlHMUjC6XiYsbcxvMcmKLd5gNh+fQ5HNYKXseyJ7J
 I0AYCH6l9lYKFp0/NxzW6m8bPso9m5IxltPy+5Y6HaXjGg6RJgwx2aM/U+GYkrY6
 2oN84MPK/PZT
 =HsOM
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/vivier2/tags/trivial-branch-pull-request' into staging

Fix memory leak with fdt
cosmetic change in code and logs
update mailmap

# gpg: Signature made Wed 19 Feb 2020 10:15:56 GMT
# gpg:                using RSA key CD2F75DDC8E3A4DC2E4F5173F30C38BD3F2FBE3C
# gpg:                issuer "laurent@vivier.eu"
# gpg: Good signature from "Laurent Vivier <lvivier@redhat.com>" [full]
# gpg:                 aka "Laurent Vivier <laurent@vivier.eu>" [full]
# gpg:                 aka "Laurent Vivier (Red Hat) <lvivier@redhat.com>" [full]
# Primary key fingerprint: CD2F 75DD C8E3 A4DC 2E4F  5173 F30C 38BD 3F2F BE3C

* remotes/vivier2/tags/trivial-branch-pull-request:
  hw/xtensa/xtfpga:fix leak of fdevice tree blob
  hw/nios2:fix leak of fdevice tree blob
  hw/net/rocker: Report unimplemented feature with qemu_log_mask(UNIMP)
  hw/block/pflash_cfi02: Remove unneeded variable assignment
  hw/display/qxl: Remove unneeded variable assignment
  contrib/rdmacm-mux: Remove superfluous semicolon
  tests/qtest/libqos/qgraph: Remove superfluous semicolons
  target/i386/whpx: Remove superfluous semicolon
  ui/input-barrier: Remove superfluous semicolon
  hw/vfio/display: Remove superfluous semicolon
  hw/scsi/esp: Remove superfluous semicolon
  hw/m68k/next-cube: Remove superfluous semicolon
  hw/arm/xlnx-versal: Remove superfluous semicolon
  audio/alsaaudio: Remove superfluous semicolons
  scripts/checkpatch.pl: Detect superfluous semicolon in C code
  Report stringified errno in VFIO related errors
  mailmap: Add entry for Yu-Chen Lin

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-02-20 16:51:19 +00:00
Max Reitz
dff8d44c96 iotests: Test snapshot -l field separation
Add a test that all fields in "qemu-img snapshot -l"s output are
separated by spaces.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200117105859.241818-3-mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[mreitz: Renamed test from 284 to 286]
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-02-20 16:43:42 +01:00
Max Reitz
f535cc90b5 iotests: Test convert -n -B to backing-less target
This must not crash.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200121155915.98232-3-mreitz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-02-20 16:43:42 +01:00
Max Reitz
4dddeac115 iotests: Add test for image creation fallback
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200122164532.178040-6-mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Maxim Levitsky <mlevitsk@redhat.com>
[mreitz: Added a note that NBD does not support resizing, which is why
         the second case is expected to fail]
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-02-20 16:43:42 +01:00
Max Reitz
b7e9eae98c iotests/279: Fix for non-qcow2 formats
First, driver=qcow2 will not work so well with non-qcow2 formats (and
this test claims to support qcow, qed, and vmdk).

Second, vmdk will always report the backing file format to be vmdk.
Filter that out so the output looks like for all other formats.

Third, the flat vmdk subformats do not support backing files, so they
will not work with this test.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20191219144243.1763246-1-mreitz@redhat.com>
Tested-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-02-20 16:43:42 +01:00
Daniel P. Berrangé
087ab8e775 block: always fill entire LUKS header space with zeros
When initializing the LUKS header the size with default encryption
parameters will currently be 2068480 bytes. This is rounded up to
a multiple of the cluster size, 2081792, with 64k sectors. If the
end of the header is not the same as the end of the cluster we fill
the extra space with zeros. This was forgetting that not even the
space allocated for the header will be fully initialized, as we
only write key material for the first key slot. The space left
for the other 7 slots is never written to.

An optimization to the ref count checking code:

  commit a5fff8d4b4 (refs/bisect/bad)
  Author: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
  Date:   Wed Feb 27 16:14:30 2019 +0300

    qcow2-refcount: avoid eating RAM

made the assumption that every cluster which was allocated would
have at least some data written to it. This was violated by way
the LUKS header is only partially written, with much space simply
reserved for future use.

Depending on the cluster size this problem was masked by the
logic which wrote zeros between the end of the LUKS header and
the end of the cluster.

$ qemu-img create --object secret,id=cluster_encrypt0,data=123456 \
   -f qcow2 -o cluster_size=2k,encrypt.iter-time=1,\
               encrypt.format=luks,encrypt.key-secret=cluster_encrypt0 \
               cluster_size_check.qcow2 100M
  Formatting 'cluster_size_check.qcow2', fmt=qcow2 size=104857600
    encrypt.format=luks encrypt.key-secret=cluster_encrypt0
    encrypt.iter-time=1 cluster_size=2048 lazy_refcounts=off refcount_bits=16

$ qemu-img check --object secret,id=cluster_encrypt0,data=redhat \
    'json:{"driver": "qcow2", "encrypt.format": "luks", \
           "encrypt.key-secret": "cluster_encrypt0", \
           "file.driver": "file", "file.filename": "cluster_size_check.qcow2"}'
ERROR: counting reference for region exceeding the end of the file by one cluster or more: offset 0x2000 size 0x1f9000
Leaked cluster 4 refcount=1 reference=0
...snip...
Leaked cluster 130 refcount=1 reference=0

1 errors were found on the image.
Data may be corrupted, or further writes to the image may corrupt it.

127 leaked clusters were found on the image.
This means waste of disk space, but no harm to data.
Image end offset: 268288

The problem only exists when the disk image is entirely empty. Writing
data to the disk image payload will solve the problem by causing the
end of the file to be extended further.

The change fixes it by ensuring that the entire allocated LUKS header
region is fully initialized with zeros. The qemu-img check will still
fail for any pre-existing disk images created prior to this change,
unless at least 1 byte of the payload is written to.

Fully writing zeros to the entire LUKS header is a good idea regardless
as it ensures that space has been allocated on the host filesystem (or
whatever block storage backend is used).

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20200207135520.2669430-1-berrange@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-02-20 16:43:42 +01:00
Max Reitz
b0c4cf21b0 iotests/147: Fix drive parameters
8dff69b94 added an aio parameter to the drive parameter but forgot to
add a comma before, thus breaking the test.  Fix it again.

Fixes: 8dff69b941
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200206130812.612960-1-mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Tested-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-02-20 16:43:42 +01:00
Thomas Huth
6756696e38 iotests: Remove the superfluous 2nd check for the availability of quorum
Commit d9df28e7b0 ("iotests: check whitelisted formats") added the
modern @iotests.skip_if_unsupported() to the functions in this test,
so we don't need the old explicit test here anymore.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20200129141751.32652-1-thuth@redhat.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Reviewed-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Tested-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2020-02-20 16:43:42 +01:00
Philippe Mathieu-Daudé
58bcdda915 tests/qtest/libqos/qgraph: Remove superfluous semicolons
Fixes: fc281c8020
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Message-Id: <20200218094402.26625-13-philmd@redhat.com>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
2020-02-18 20:20:49 +01:00
Max Reitz
c45a88f442 iotests: Check that @replaces can replace filters
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20200218103454.296704-20-mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-02-18 14:52:16 +01:00
Max Reitz
a1da187860 iotests: Add tests for invalid Quorum @replaces
Add two tests to see that you cannot replace a Quorum child with the
mirror job while the child is in use by a different parent.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20200218103454.296704-19-mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-02-18 14:52:16 +01:00
Max Reitz
89e2194524 iotests: Use self.image_len in TestRepairQuorum
041's TestRepairQuorum has its own image_len, no need to refer to
TestSingleDrive.  (This patch allows commenting out TestSingleDrive to
speed up 041 during test testing.)

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20200218103454.296704-18-mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-02-18 11:55:41 +01:00
Max Reitz
c351afd6f3 iotests: Resolve TODOs in 041
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20200218103454.296704-17-mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-02-18 11:55:41 +01:00
Max Reitz
5d016a69e3 iotests/041: Drop superfluous shutdowns
All tearDowns in 041 shutdown the VM.  Thus, test cases do not need to
do it themselves (unless they need the VM to be down for some
post-operation check).

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20200218103454.296704-16-mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-02-18 11:55:41 +01:00
Max Reitz
6a3d0f1e3f iotests: Add VM.assert_block_path()
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200218103454.296704-15-mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-02-18 11:55:41 +01:00
Max Reitz
6644d0e619 iotests: Use complete_and_wait() in 155
This way, we get to see errors during the completion phase.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20200218103454.296704-14-mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-02-18 11:55:40 +01:00
Max Reitz
f718ca147d iotests: Let 041 use -blockdev for quorum children
Using -drive with default options means that a virtio-blk drive will be
created that has write access to the to-be quorum children.  Quorum
should have exclusive write access to them, so we should use -blockdev
instead.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20200218103454.296704-5-mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-02-18 11:55:39 +01:00
Kevin Wolf
d439848941 iotests: Test error handling policies with block-commit
This tests both read failure (from the top node) and write failure (to
the base node) for on-error=report/stop/ignore.

As block-commit actually starts two different types of block jobs
(mirror.c for committing the active later, commit.c for intermediate
layers), all tests are run for both cases.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20200214200812.28180-8-kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-02-18 10:53:56 +01:00
Kevin Wolf
a0cf8daf77 iotests: Test copy offloading with external data file
This adds a test for 'qemu-img convert' with copy offloading where the
target image has an external data file. If the test hosts supports it,
it tests both the case where copy offloading is supported and the case
where it isn't (otherwise we just test unsupported twice).

More specifically, the case with unsupported copy offloading tests
qcow2_alloc_cluster_abort() with external data files.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20200211094900.17315-4-kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2020-02-18 10:53:56 +01:00
Kevin Wolf
fa4dcf577e qapi: Split control.json off misc.json
misc.json contains definitions that are related to the system emulator,
so it can't be used for other tools like the storage daemon. This patch
moves basic functionality that is shared between all tools (and mostly
related to the monitor itself) into a new control.json, which could be
used in tools as well.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20200129102239.31435-3-kwolf@redhat.com>
[Commit message tweaked]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2020-02-17 13:53:47 +01:00
Peter Maydell
b29c3e23f6 Migration pull request
-----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEEGJn/jt6/WMzuA0uC9IfvGFhy1yMFAl5FSWsACgkQ9IfvGFhy
 1yMywA/+M47cU7LqrI0VpIIOTggkgobTv+RZyJrPDTy5UMAvclCj+4D+khCieFlS
 Yuj9U1dNHMCB4yLWCl6NDsiJCRAqyEiRlQU08wiNIsGbp2tNvUccQHOhovkXsgyK
 JKRzarUXPa0Cgvrt2UJXJMgYiMSB7T6YBJ6HyYuaTM4A5fnuiGqH33YTJSznMCFD
 VswxqMlQ+6iKrqI8yHQOxDCyl4KA7CVt+qcOKt0bj+UZsoDyqbyP9bXB578rtNua
 a/4j4Uqqo5NCaGDhAs7gLWtbeIqHz2nok5NpcXt/WGKbLz7WmcHo++YZKiECLbH8
 b6hrIUsj4yKzhG7VijPpFVtxl4Erm0CknxnWglgfYVp+BwBqJMbycPtCbUo3hC61
 o2VOlUpKLmJRGSqOm9heWHNPjp5jXi0aW2PxrpxP3FQKIuw8aldzqrzfPlDk/4Z9
 9s7IecoXNx5vLhcbkLoXSGVoIRrL8yJfr6yQdCeapMBviyU5duZYWCFyiG3g0Wb0
 RD5dqEzKaTtA1XRHJLSZUlBgTIjTwFRUXqSqcc8dJPeCTbmkUbisX0sXxcyqTZWS
 myepB8QVVPghT1+zoyUUxZfY33ROp3EsfOucOCjQVMqns1qWC7Qiw/XbjqB5hApQ
 peZ9bS9i1Ns1nNJqBDWpJez5fobvJ7a8lUNm5/s39pU5GLbUYvM=
 =fWBO
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/juanquintela/tags/pull-migration-pull-request' into staging

Migration pull request

# gpg: Signature made Thu 13 Feb 2020 13:04:43 GMT
# gpg:                using RSA key 1899FF8EDEBF58CCEE034B82F487EF185872D723
# gpg: Good signature from "Juan Quintela <quintela@redhat.com>" [full]
# gpg:                 aka "Juan Quintela <quintela@trasno.org>" [full]
# Primary key fingerprint: 1899 FF8E DEBF 58CC EE03  4B82 F487 EF18 5872 D723

* remotes/juanquintela/tags/pull-migration-pull-request:
  git: Make submodule check only needed modules
  migration-test: fix some memleaks in migration-test
  tests/migration: Add some slack to auto converge
  migration/rdma: rdma_accept_incoming_migration fix error handling
  migration: Optimization about wait-unplug migration state
  migration: Maybe VM is paused when migration is cancelled

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-02-14 17:57:15 +00:00
Peter Maydell
bc882694a3 target-arm queue:
* i.MX: Fix inverted sense of register bits in watchdog timer
  * i.MX: Add support for WDT on i.MX6
  * arm/virt: cleanups to ACPI tables
  * Implement ARMv8.1-VMID16 extension
  * Implement ARMv8.1-PAN
  * Implement ARMv8.2-UAO
  * Implement ARMv8.2-ATS1E1
  * ast2400/2500/2600: Wire up EHCI controllers
  * hw/char/exynos4210_uart: Fix memleaks in exynos4210_uart_init
  * hw/arm/raspi: Clean up the board code
 -----BEGIN PGP SIGNATURE-----
 
 iQJNBAABCAA3FiEE4aXFk81BneKOgxXPPCUl7RQ2DN4FAl5FX+IZHHBldGVyLm1h
 eWRlbGxAbGluYXJvLm9yZwAKCRA8JSXtFDYM3kSGEACPhcIl+ChhXd6M9iJASiXn
 txg1a5ww6I99G9z9PFEuCv6lHBBcmfux9slu03HPyot1ycimx62NfjrOF/d0wq2p
 OEw5Zs1BA9NqT6/aaB4JMV0RxkODP9xT6kKGVyufNJK1JS81V098R1EcSmaX+22+
 LTa/IvQF128uqGqRwh68oU2YQ4FyP+Ow5AvGZ0JzhyDrtPxLVC5hVBqXK6iZhddk
 6o6CtMRYP6v50dq6njScoAa0DOYm+FL/cOATtlZCozt2uGqkmuYmy4zP5j5JeP3c
 SCQZW8MnF3duG1tX4nsxFEJEljjB1JrEpllm1Nml9wXe7XrQQfFXrvXARq900Xax
 +hY6xMmNx/WyijXXbm3W8GVfVfzcE+/kzNb3wRcbXK4T2AB2B7qT7ShDLD84NLo4
 5l+asl2bjfArcp4JQSHZbe3ZG848+qb+XIoXU7CMOswmJvKnoaXD0xXeFe84Qgnl
 n/bhNJPNskNcfXGhXdGjmoTRN8gEpappiimCc5nzajNuJiAFJ9mcstV+ZGMHwOui
 2Y/FMm3JgtKK9ecO1p3DZHrN+rqe3/WE4LIr2v2OoEdp9mlsAsq6cO3tzxxuKKTz
 pyLnCW3q6t+rYDxqw6gvYudINtCSrmEHncEOAoqPxxFmc3YDJ6P3TfmiTKX3bioo
 jwBnZGEuPINW1MZvCdCWUQ==
 =Mspo
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/pmaydell/tags/pull-target-arm-20200213' into staging

target-arm queue:
 * i.MX: Fix inverted sense of register bits in watchdog timer
 * i.MX: Add support for WDT on i.MX6
 * arm/virt: cleanups to ACPI tables
 * Implement ARMv8.1-VMID16 extension
 * Implement ARMv8.1-PAN
 * Implement ARMv8.2-UAO
 * Implement ARMv8.2-ATS1E1
 * ast2400/2500/2600: Wire up EHCI controllers
 * hw/char/exynos4210_uart: Fix memleaks in exynos4210_uart_init
 * hw/arm/raspi: Clean up the board code

# gpg: Signature made Thu 13 Feb 2020 14:40:34 GMT
# gpg:                using RSA key E1A5C593CD419DE28E8315CF3C2525ED14360CDE
# gpg:                issuer "peter.maydell@linaro.org"
# gpg: Good signature from "Peter Maydell <peter.maydell@linaro.org>" [ultimate]
# gpg:                 aka "Peter Maydell <pmaydell@gmail.com>" [ultimate]
# gpg:                 aka "Peter Maydell <pmaydell@chiark.greenend.org.uk>" [ultimate]
# Primary key fingerprint: E1A5 C593 CD41 9DE2 8E83  15CF 3C25 25ED 1436 0CDE

* remotes/pmaydell/tags/pull-target-arm-20200213: (46 commits)
  target/arm: Implement ARMv8.1-VMID16 extension
  hw/arm/raspi: Extract the cores count from the board revision
  hw/arm/raspi: Use a unique raspi_machine_class_init() method
  hw/arm/raspi: Extract the board model from the board revision
  hw/arm/raspi: Set default RAM size to size encoded in board revision
  hw/arm/raspi: Let class_init() directly call raspi_machine_init()
  hw/arm/raspi: Make board_rev a field of RaspiMachineClass
  hw/arm/raspi: Make machines children of abstract RaspiMachineClass
  hw/arm/raspi: Trivial code movement
  hw/arm/raspi: Extract the processor type from the board revision
  hw/arm/raspi: Extract the RAM size from the board revision
  hw/arm/raspi: Extract the version from the board revision
  hw/arm/raspi: Correct the board descriptions
  hw/arm/raspi: Use BCM2708 machine type with pre Device Tree kernels
  hw/char/exynos4210_uart: Fix memleaks in exynos4210_uart_init
  hw/arm: ast2600: Wire up EHCI controllers
  hw/arm: ast2400/ast2500: Wire up EHCI controllers
  target/arm: Enable ARMv8.2-UAO in -cpu max
  target/arm: Implement UAO semantics
  target/arm: Update MSR access to UAO
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-02-14 15:10:33 +00:00
Heyi Guo
979a89023f virt/acpi: update golden masters for DSDT update
Differences between disassembled ASL files:

@@ -5,13 +5,13 @@
  *
  * Disassembling to symbolic ASL+ operators
  *
- * Disassembly of DSDT, Thu Jan 23 16:00:04 2020
+ * Disassembly of DSDT.new, Thu Jan 23 16:47:12 2020
  *
  * Original Table Header:
  *     Signature        "DSDT"
- *     Length           0x0000481E (18462)
+ *     Length           0x000014BB (5307)
  *     Revision         0x02
- *     Checksum         0x60
+ *     Checksum         0xD1
  *     OEM ID           "BOCHS "
  *     OEM Table ID     "BXPCDSDT"
  *     OEM Revision     0x00000001 (1)
@@ -43,7 +43,6 @@ DefinitionBlock ("", "DSDT", 2, "BOCHS ", "BXPCDSDT", 0x00000001)
                     0x00000021,
                 }
             })
-            Name (_ADR, 0x09000000)  // _ADR: Address
         }

         Device (FLS0)
@@ -668,11 +667,10 @@ DefinitionBlock ("", "DSDT", 2, "BOCHS ", "BXPCDSDT", 0x00000001)
             Name (_CID, "PNP0A03" /* PCI Bus */)  // _CID: Compatible ID
             Name (_SEG, Zero)  // _SEG: PCI Segment
             Name (_BBN, Zero)  // _BBN: BIOS Bus Number
-            Name (_ADR, Zero)  // _ADR: Address
             Name (_UID, "PCI0")  // _UID: Unique ID
             Name (_STR, Unicode ("PCIe 0 Device"))  // _STR: Description String
             Name (_CCA, One)  // _CCA: Cache Coherency Attribute
-            Name (_PRT, Package (0x0400)  // _PRT: PCI Routing Table
+            Name (_PRT, Package (0x80)  // _PRT: PCI Routing Table
             {
                 Package (0x04)
                 {
@@ -1696,7174 +1694,6 @@ DefinitionBlock ("", "DSDT", 2, "BOCHS ", "BXPCDSDT", 0x00000001)
                     0x03,
                     GSI2,
                     Zero
-                },
-
-                Package (0x04)
-                {
-                    0x0020FFFF,
-                    Zero,
-                    GSI0,
-                    Zero
-                },
-
-                *Omit the other (4 * (256 - 32) - 2) packages*
-
-                Package (0x04)
-                {
-                    0x00FFFFFF,
-                    0x03,
-                    GSI2,
-                    Zero
                 }
             })
             Device (GSI0)
@@ -8892,7 +1722,7 @@ DefinitionBlock ("", "DSDT", 2, "BOCHS ", "BXPCDSDT", 0x00000001)
             Device (GSI1)
             {
                 Name (_HID, "PNP0C0F" /* PCI Interrupt Link Device */)  // _HID: Hardware ID
-                Name (_UID, Zero)  // _UID: Unique ID
+                Name (_UID, One)  // _UID: Unique ID
                 Name (_PRS, ResourceTemplate ()  // _PRS: Possible Resource Settings
                 {
                     Interrupt (ResourceConsumer, Level, ActiveHigh, Exclusive, ,, )
@@ -8915,7 +1745,7 @@ DefinitionBlock ("", "DSDT", 2, "BOCHS ", "BXPCDSDT", 0x00000001)
             Device (GSI2)
             {
                 Name (_HID, "PNP0C0F" /* PCI Interrupt Link Device */)  // _HID: Hardware ID
-                Name (_UID, Zero)  // _UID: Unique ID
+                Name (_UID, 0x02)  // _UID: Unique ID
                 Name (_PRS, ResourceTemplate ()  // _PRS: Possible Resource Settings
                 {
                     Interrupt (ResourceConsumer, Level, ActiveHigh, Exclusive, ,, )
@@ -8938,7 +1768,7 @@ DefinitionBlock ("", "DSDT", 2, "BOCHS ", "BXPCDSDT", 0x00000001)
             Device (GSI3)
             {
                 Name (_HID, "PNP0C0F" /* PCI Interrupt Link Device */)  // _HID: Hardware ID
-                Name (_UID, Zero)  // _UID: Unique ID
+                Name (_UID, 0x03)  // _UID: Unique ID
                 Name (_PRS, ResourceTemplate ()  // _PRS: Possible Resource Settings
                 {
                     Interrupt (ResourceConsumer, Level, ActiveHigh, Exclusive, ,, )
@@ -8965,37 +1795,6 @@ DefinitionBlock ("", "DSDT", 2, "BOCHS ", "BXPCDSDT", 0x00000001)

             Method (_CRS, 0, NotSerialized)  // _CRS: Current Resource Settings
             {
-                Name (RBUF, ResourceTemplate ()
-                {
-                    WordBusNumber (ResourceProducer, MinFixed, MaxFixed, PosDecode,
-                        0x0000,             // Granularity
-                        0x0000,             // Range Minimum
-                        0x00FF,             // Range Maximum
-                        0x0000,             // Translation Offset
-                        0x0100,             // Length
-                        ,, )
-                    DWordMemory (ResourceProducer, PosDecode, MinFixed, MaxFixed, NonCacheable, ReadWrite,
-                        0x00000000,         // Granularity
-                        0x10000000,         // Range Minimum
-                        0x3EFEFFFF,         // Range Maximum
-                        0x00000000,         // Translation Offset
-                        0x2EFF0000,         // Length
-                        ,, , AddressRangeMemory, TypeStatic)
-                    DWordIO (ResourceProducer, MinFixed, MaxFixed, PosDecode, EntireRange,
-                        0x00000000,         // Granularity
-                        0x00000000,         // Range Minimum
-                        0x0000FFFF,         // Range Maximum
-                        0x3EFF0000,         // Translation Offset
-                        0x00010000,         // Length
-                        ,, , TypeStatic, DenseTranslation)
-                    QWordMemory (ResourceProducer, PosDecode, MinFixed, MaxFixed, NonCacheable, ReadWrite,
-                        0x0000000000000000, // Granularity
-                        0x0000008000000000, // Range Minimum
-                        0x000000FFFFFFFFFF, // Range Maximum
-                        0x0000000000000000, // Translation Offset
-                        0x0000008000000000, // Length
-                        ,, , AddressRangeMemory, TypeStatic)
-                })
                 Return (ResourceTemplate ()
                 {
                     WordBusNumber (ResourceProducer, MinFixed, MaxFixed, PosDecode,
@@ -9080,11 +1879,6 @@ DefinitionBlock ("", "DSDT", 2, "BOCHS ", "BXPCDSDT", 0x00000001)
                 })
             }

-            Device (RP0)
-            {
-                Name (_ADR, Zero)  // _ADR: Address
-            }
-
             Device (RES0)
             {
                 Name (_HID, "PNP0C02" /* PNP Motherboard Resources */)  // _HID: Hardware ID
@@ -9131,7 +1925,6 @@ DefinitionBlock ("", "DSDT", 2, "BOCHS ", "BXPCDSDT", 0x00000001)
         Device (PWRB)
         {
             Name (_HID, "PNP0C0C" /* Power Button Device */)  // _HID: Hardware ID
-            Name (_ADR, Zero)  // _ADR: Address
             Name (_UID, Zero)  // _UID: Unique ID
         }
     }

The differences between the two versions of DSDT.memhp are almost the
same as the above, except for total length and checksum.

DSDT.numamem binary is just the same with DSDT on virt machine, so we
don't show the differences again.

Signed-off-by: Heyi Guo <guoheyi@huawei.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Message-id: 20200204014325.16279-8-guoheyi@huawei.com
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-02-13 14:14:53 +00:00
Heyi Guo
4ac637ef84 bios-tables-test: prepare to change ARM virt ACPI DSDT
We are going to change ARM virt ACPI DSDT table, which will cause make
check to fail, so temporarily add related golden masters to ignore
list.

Signed-off-by: Heyi Guo <guoheyi@huawei.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Message-id: 20200204014325.16279-2-guoheyi@huawei.com
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-02-13 14:14:53 +00:00
Peter Maydell
81f49abaaa * various small fixes and cleanups
* fixes for the ucode revision patch from the previous pull request
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v2.0.22 (GNU/Linux)
 
 iQEcBAABAgAGBQJeRBoQAAoJEL/70l94x66DYOUIAJKM5PDAhOkJpSqoSz2mMVJN
 IOq2YhmbQHSVojUDCUEFnKd2Zja858vsOoAslgce0nbgj+ZNn9WZEVIKeHqkW6cg
 gWybMy1deT5o0U05e6/EIL0id8N5FVMLdUZRdGCFk4qtILFa1kRYJmlwNOMxSVkB
 MSwOy60//+po6lW0WSKD2aTf9qVvZJOQjZozqFrGlIPG/SnqrPZCMRxR/XIrrJuF
 GZZyiAsRXFU1mabr7HafdUj+MYf0DWSvATmihz4wsI/LlXx712uVm9eZ48Rc6PyF
 wEKh17/2bVPoL1g9OeTKYnX9aKt/7C0D98n3W0foG3IpzZzPLX4NaNKPihb5n7I=
 =VlyM
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into staging

* various small fixes and cleanups
* fixes for the ucode revision patch from the previous pull request

# gpg: Signature made Wed 12 Feb 2020 15:30:24 GMT
# gpg:                using RSA key BFFBD25F78C7AE83
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>" [full]
# gpg:                 aka "Paolo Bonzini <pbonzini@redhat.com>" [full]
# Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4  E2F7 7E15 100C CD36 69B1
#      Subkey fingerprint: F133 3857 4B66 2389 866C  7682 BFFB D25F 78C7 AE83

* remotes/bonzini/tags/for-upstream:
  target/i386: enable monitor and ucode revision with -cpu max
  target/i386: check for availability of MSR_IA32_UCODE_REV as an emulated MSR
  target/i386: fix TCG UCODE_REV access
  build: move TARGET_GPROF to config-host.mak
  exec: do not define use_icount for user-mode emulation
  minikconf: accept alnum identifiers
  Remove support for CLOCK_MONOTONIC not being defined
  seqlock: fix seqlock_write_unlock_impl function
  vl: Don't mismatch g_strsplit()/g_free()

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-02-13 13:40:59 +00:00
Pan Nengyuan
e022d47388 migration-test: fix some memleaks in migration-test
spotted by asan, 'check-qtest-aarch64' runs fail if sanitizers is enabled.

Reported-by: Euler Robot <euler.robot@huawei.com>
Signed-off-by: Pan Nengyuan <pannengyuan@huawei.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Laurent Vivier <lvivier@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
2020-02-13 11:28:54 +01:00
Dr. David Alan Gilbert
6e1f837a14 tests/migration: Add some slack to auto converge
There's an assert in autoconverge that checks that we quit the
iteration when we go below the expected threshold.  Philippe
saw a case where this assert fired with the measured value
slightly over the threshold. (about 3k out of a few million).

I can think of two reasons:
  a) Rounding errors
  b) That after we make the decision to quit iteration we do one
    more sync and that sees a few more dirty pages.

So add 1% slack to the assertion, that should cover a and
most cases of b, probably all we'll see for the test.

Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
2020-02-13 10:57:35 +01:00
Paolo Bonzini
4cc600d229 build: move TARGET_GPROF to config-host.mak
TARGET_GPROF is the same for all targets, write it to
config-host.mak instead.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-id: <20200204161104.21077-1-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-02-12 16:29:27 +01:00
Peter Maydell
2b8a51cdb3 9p patches:
- some more protocol sanity checks
 - qtest for readdir
 - Christian Schoenebeck now official reviewer
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEEtIKLr5QxQM7yo0kQcdTV5YIvc9YFAl4+c3IACgkQcdTV5YIv
 c9Z0eA/8Dpl4KpT2GosgsCj9iqGnT25B0UYiVdLL3aYsJfCgoD7MV0b/v6cK30SI
 XyJS2sAMMdGUfOhy+dRUVzSwwKtQOUR9s8zudD5zZ95CtDBGd0c8KdX6TktXRv7U
 pnxej9zL29b9+OK6zTv7jzPmpiTkJ1AXa4Rs59DccBaGn3AItgN9VRkOWylMiNir
 m4EJh/v5BBTvO9754nExKwgsQHt9eX6jmp+uiRwC2bFirIIbt9bQdwUlX8km03Ml
 Qm8gdpJaBgBubFwH1LvN/DPX0JnShGbGonO1zq8SS9NUkZJuSPtI3LY+X1oC9Wgi
 ZWLnGc9yTAeaABeg5Hk2jGapxtSL3qEEZUKitRJFrnmSe+R0PcAdjuN9DZEl56rP
 zOj0+YUtUggoRd0iwzlZkpga8mKnGM63jovT6cOOTXuLWoh4VlsvBnZSguxKjT2x
 rwuEjNA/ctsT5sl6Q++iM8AlrQmrajKphFpT19MtalUN9pNlk2C8PPBQ3xPVTlwT
 V3YO4x49hkKbJIUoWqNRwwTQUVsFr3vkhf4IJwq7MBC++wPBoq9W1kQsjcxiZfDP
 m1drND5AM9d7E8PAgGsvNO2C41rH7funoZo+NVrrz+Sb01JJI+pE1A2mKrIsTvmh
 uxAEv+U6dqS/EjCTY1lyAiM5nLN0qZ8IQiPO1+yxJmkbDJIuJgM=
 =oMrh
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/gkurz/tags/9p-next-2020-02-08' into staging

9p patches:
- some more protocol sanity checks
- qtest for readdir
- Christian Schoenebeck now official reviewer

# gpg: Signature made Sat 08 Feb 2020 08:38:10 GMT
# gpg:                using RSA key B4828BAF943140CEF2A3491071D4D5E5822F73D6
# gpg: Good signature from "Greg Kurz <groug@kaod.org>" [full]
# gpg:                 aka "Gregory Kurz <gregory.kurz@free.fr>" [full]
# gpg:                 aka "[jpeg image of size 3330]" [full]
# Primary key fingerprint: B482 8BAF 9431 40CE F2A3  4910 71D4 D5E5 822F 73D6

* remotes/gkurz/tags/9p-next-2020-02-08:
  MAINTAINERS: 9pfs: Add myself as reviewer
  tests/virtio-9p: added readdir test
  hw/9pfs/9p-synth: added directory for readdir test
  9pfs: validate count sent by client with T_readdir
  9pfs: require msize >= 4096
  tests/virtio-9p: add terminating null in v9fs_string_read()

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-02-10 16:07:29 +00:00
Peter Maydell
73d336510c - Python 3 cleanups:
. Remove text about Python 2 in qemu-deprecated (Thomas)
   . Remove shebang header (Paolo, Philippe)
   . scripts/checkpatch.pl now allows Python 3 interpreter (Philippe)
   . Explicit usage of Python 3 interpreter in scripts (Philippe)
   . Fix Python scripts permissions (Paolo, Philippe)
   . Drop 'from __future__ import print_function' (Paolo)
   . Specify minimum python requirements in ReadTheDocs configuration (Alex)
 - Test UNIX/EXEC transports with migration (Oksana)
 - Added extract_from_rpm helper, improved extract_from_deb (Liam)
 - Allow to use other serial consoles than default one (Philippe)
 - Various improvements in QEMUMonitorProtocol (Wainer)
 - Fix kvm_available() on ppc64le (Wainer)
 -----BEGIN PGP SIGNATURE-----
 
 iQIyBAABCAAdFiEE+qvnXhKRciHc/Wuy4+MsLN6twN4FAl49e+QACgkQ4+MsLN6t
 wN7E6A/3dzCLP0QDoT/C2rC1uu2yXBpXsSUSGeRCA+NLGEOpkyWiEK/vCQp/WxGO
 YU3ToO5NIGMxjVGhy14D1U4NrmiUhxmkGlJQJ60WKAO/oJBElHz8aR2keGlbkBp3
 mh/Amyw+ubM3aBYcAmBbgn0y7yr+lV62mD1JmffQebwIMsT+6aL4XhZ6Lf8PAEkY
 0pCK30O0j8Mwk8ppwKjoRT0WhhcKInmaLElbcOstoqVjQerGV7ho7VjlS9v2BPeg
 uQbDbv5lask7XVoJzortQ9nP2Tq3R+T7hNc0RPa0/tuqZ/6ucq6eYbFgbr/S7r3/
 Lem3ByA1EPlNzjq1HZoU3zx0/Ydq6qPyiudO9fLOCDIGVG6ZKXmKVq71lWHotcAw
 Oir+RPSVhPmOekhOPXAEi3mwAqRX3iSUGSsm+aLgq9I1DJfdE8dG2n5ZKVSWClmF
 xwb8Dhlz1T5SIlLderfVFryLzV2/LoHQOpr37phw6Ec5gox6UR0i4ozKEaY1zmZV
 THvweFraoBUGwIN7ULfP8jxs8hjQY9HIxhYjNvPxqAQekH6hO97qkjbs44ujvNrS
 D/yL4Gk4XgDjGP37aeBoY0zCUMM3cpWZdVfycDiXe6r949ifOoF0gZSny7ryRTGh
 s1RvQAM9VaYcGTERkLLCC+QMuz24nN/cEg9H3SFlfHBGVBxaBw==
 =8xBQ
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/philmd-gitlab/tags/python-next-20200207' into staging

- Python 3 cleanups:
  . Remove text about Python 2 in qemu-deprecated (Thomas)
  . Remove shebang header (Paolo, Philippe)
  . scripts/checkpatch.pl now allows Python 3 interpreter (Philippe)
  . Explicit usage of Python 3 interpreter in scripts (Philippe)
  . Fix Python scripts permissions (Paolo, Philippe)
  . Drop 'from __future__ import print_function' (Paolo)
  . Specify minimum python requirements in ReadTheDocs configuration (Alex)
- Test UNIX/EXEC transports with migration (Oksana)
- Added extract_from_rpm helper, improved extract_from_deb (Liam)
- Allow to use other serial consoles than default one (Philippe)
- Various improvements in QEMUMonitorProtocol (Wainer)
- Fix kvm_available() on ppc64le (Wainer)

# gpg: Signature made Fri 07 Feb 2020 15:01:56 GMT
# gpg:                using RSA key FAABE75E12917221DCFD6BB2E3E32C2CDEADC0DE
# gpg: Good signature from "Philippe Mathieu-Daudé (F4BUG) <f4bug@amsat.org>" [full]
# Primary key fingerprint: FAAB E75E 1291 7221 DCFD  6BB2 E3E3 2C2C DEAD C0DE

* remotes/philmd-gitlab/tags/python-next-20200207: (46 commits)
  .readthedocs.yml: specify some minimum python requirements
  drop "from __future__ import print_function"
  make all Python scripts executable
  scripts/signrom: remove Python 2 support, add shebang
  tests/qemu-iotests/check: Only check for Python 3 interpreter
  scripts: Explicit usage of Python 3 (scripts without __main__)
  tests/qemu-iotests: Explicit usage of Python3 (scripts without __main__)
  tests/vm: Remove shebang header
  tests/acceptance: Remove shebang header
  scripts/tracetool: Remove shebang header
  scripts/minikconf: Explicit usage of Python 3
  scripts: Explicit usage of Python 3 (scripts with __main__)
  tests: Explicit usage of Python 3
  tests/qemu-iotests: Explicit usage of Python 3 (scripts with __main__)
  tests/qemu-iotests/check: Allow use of python3 interpreter
  scripts/checkpatch.pl: Only allow Python 3 interpreter
  tests/acceptance/migration: Default to -nodefaults
  tests/acceptance/migration: Add the 'migration' tag
  tests/acceptance/migration: Test EXEC transport when migrating
  tests/acceptance/migration: Test UNIX transport when migrating
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-02-10 12:07:26 +00:00
Christian Schoenebeck
4829469fd9 tests/virtio-9p: added readdir test
The first readdir test simply checks the amount of directory
entries returned by 9pfs server, according to the created amount
of virtual files on 9pfs synth driver side. Then the subsequent
readdir test also checks whether all directory entries have the
expected file names (as created on 9pfs synth driver side),
ignoring their precise order in result list though.

Signed-off-by: Christian Schoenebeck <qemu_oss@crudebyte.com>
Message-Id: <e0b4402722a877178f8fb6a8ad7b64bb20150613.1579567020.git.qemu_oss@crudebyte.com>
Reviewed-by: Greg Kurz <groug@kaod.org>
Signed-off-by: Greg Kurz <groug@kaod.org>
2020-02-08 09:29:04 +01:00
Christian Schoenebeck
2e2293c238 tests/virtio-9p: add terminating null in v9fs_string_read()
The 9p protocol sends strings in general without null termination
over the wire. However for future use of this functions it is
beneficial for the delivered string to be null terminated though
for being able to use the string with standard C functions which
often rely on strings being null terminated.

Signed-off-by: Christian Schoenebeck <qemu_oss@crudebyte.com>
Reviewed-by: Greg Kurz <groug@kaod.org>
Message-Id: <52c84e2ce3bcafc2a38eed13b8c8e23bc1a8ecb9.1579567019.git.qemu_oss@crudebyte.com>
Signed-off-by: Greg Kurz <groug@kaod.org>
2020-02-08 09:24:19 +01:00
Paolo Bonzini
423edd9a31 drop "from __future__ import print_function"
This is only needed for Python 2, which we do not support anymore.

Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Acked-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20200204160604.19883-1-pbonzini@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:15:16 +01:00
Paolo Bonzini
6dd360ce3f make all Python scripts executable
Scripts that have a Python shebang are meant to be executed directly from the
shell; give them 755 permissions.

Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200204160237.16889-1-pbonzini@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:15:16 +01:00
Philippe Mathieu-Daudé
351aa2706e tests/qemu-iotests/check: Only check for Python 3 interpreter
All the iotests Python scripts have been converted to search for
the Python 3 interpreter. Update the ./check script accordingly.

Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20200130163232.10446-13-philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:15:16 +01:00
Philippe Mathieu-Daudé
7c47752608 tests/qemu-iotests: Explicit usage of Python3 (scripts without __main__)
Use the program search path to find the Python 3 interpreter.

Patch created mechanically by running:

  $ sed -i "s,^#\!/usr/bin/\(env\ \)\?python$,#\!/usr/bin/env python3," \
      $(git grep -lF '#!/usr/bin/env python' \
      | xargs grep -L 'if __name__.*__main__')

Reported-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Suggested-by: Daniel P. Berrangé <berrange@redhat.com>
Suggested-by: Stefan Hajnoczi <stefanha@redhat.com>
Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20200130163232.10446-11-philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:15:16 +01:00
Philippe Mathieu-Daudé
629d5edfaa tests/vm: Remove shebang header
Patch created mechanically by running:

  $ chmod 644 $(git grep -lF '#!/usr/bin/env python' \
      | xargs grep -L 'if __name__.*__main__')
  $ sed -i "/^#\!\/usr\/bin\/\(env\ \)\?python.\?$/d" \
      $(git grep -lF '#!/usr/bin/env python' \
      | xargs grep -L 'if __name__.*__main__')

Reported-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Suggested-by: Stefan Hajnoczi <stefanha@redhat.com>
Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20200130163232.10446-10-philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:12:48 +01:00
Philippe Mathieu-Daudé
273954e782 tests/acceptance: Remove shebang header
Patch created mechanically by running:

  $ chmod 644 $(git grep -lF '#!/usr/bin/env python' \
      | xargs grep -L 'if __name__.*__main__')
  $ sed -i "/^#\!\/usr\/bin\/\(env\ \)\?python.\?$/d" \
      $(git grep -lF '#!/usr/bin/env python' \
      | xargs grep -L 'if __name__.*__main__')

Reported-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Suggested-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20200130163232.10446-8-philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:12:48 +01:00
Philippe Mathieu-Daudé
c88ee46cdb tests: Explicit usage of Python 3
Use the program search path to find the Python 3 interpreter.

Patch created mechanically by running:

  $ sed -i "s,^#\!/usr/bin/\(env\ \)\?python$,#\!/usr/bin/env python3," \
       $(git grep -l 'if __name__.*__main__')

Reported-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Suggested-by: Daniel P. Berrangé <berrange@redhat.com>
Suggested-by: Stefan Hajnoczi <stefanha@redhat.com>
Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20200130163232.10446-5-philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:12:48 +01:00
Philippe Mathieu-Daudé
903cb1bf39 tests/qemu-iotests: Explicit usage of Python 3 (scripts with __main__)
Use the program search path to find the Python 3 interpreter.

Patch created mechanically by running:

  $ sed -i "s,^#\!/usr/bin/\(env\ \)\?python$,#\!/usr/bin/env python3," \
       $(git grep -l 'if __name__.*__main__')

Reported-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Suggested-by: Daniel P. Berrangé <berrange@redhat.com>
Suggested-by: Stefan Hajnoczi <stefanha@redhat.com>
Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20200130163232.10446-4-philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:12:48 +01:00
Philippe Mathieu-Daudé
15b015690b tests/qemu-iotests/check: Allow use of python3 interpreter
As we want to enforce a unique and explicit Python 3 interpreter,
we need let this script handle 'python3' too.

Suggested-by: Kevin Wolf <kwolf@redhat.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20200130163232.10446-3-philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:12:48 +01:00
Philippe Mathieu-Daudé
5427ecd4ab tests/acceptance/migration: Default to -nodefaults
We don't need the default options to run this test.

This fixes errors when running a binary built with
--without-default-devices such:

  ERROR: qemu-system-arm: Unsupported NIC model: virtio-net-pci

Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20200129212345.20547-28-philmd@redhat.com>
[PMD: Rebased]
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:12:48 +01:00
Philippe Mathieu-Daudé
a091864943 tests/acceptance/migration: Add the 'migration' tag
By using an Avocado tag, we can run all tests described by that
tag as once:

  $ avocado --show=app run -t migration tests/acceptance/
  JOB ID     : 165477737509503fcfa6d7108057a0a18f2a6559
  JOB LOG    : avocado/job-results/job-2020-02-04T17.29-1654777/job.log
   (1/3) tests/acceptance/migration.py:Migration.test_migration_with_tcp_localhost: PASS (0.38 s)
   (2/3) tests/acceptance/migration.py:Migration.test_migration_with_unix: PASS (0.33 s)
   (3/3) tests/acceptance/migration.py:Migration.test_migration_with_exec: PASS (0.07 s)
  RESULTS    : PASS 3 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0

Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20200204163304.14616-1-philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:12:48 +01:00
Oksana Vohchana
2e768cb682 tests/acceptance/migration: Test EXEC transport when migrating
Along with VM migration via TCP, we can use migration through
the EXEC transport protocol

Signed-off-by: Oksana Vohchana <ovoshcha@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200203111631.18796-3-ovoshcha@redhat.com>
[PMD: Split patch in 2, reworded subject and description]
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:12:48 +01:00
Oksana Vohchana
b2cf8d4704 tests/acceptance/migration: Test UNIX transport when migrating
Along with VM migration via TCP, we can use migration through
the UNIX transport protocol

Signed-off-by: Oksana Vohchana <ovoshcha@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200203111631.18796-3-ovoshcha@redhat.com>
[PMD: Split patch in 2, reworded subject and description]
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:12:48 +01:00
Oksana Vohchana
63adf16d4f tests/acceptance/migration: Factor out do_migrate()
We are going to reuse this code when testing different transport
methods, so factor it out first.

Signed-off-by: Oksana Vohchana <ovoshcha@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200203111631.18796-2-ovoshcha@redhat.com>
[PMD: Split patch in 2, reworded subject and description]
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:12:48 +01:00
Oksana Vohchana
d7c9a83344 tests/acceptance/migration: Factor out assert_migration()
We are going to reuse this code when testing different transport
methods, so factor it out first

Signed-off-by: Oksana Vohchana <ovoshcha@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200203111631.18796-2-ovoshcha@redhat.com>
[PMD: Split patch in 2, reworded subject and description]
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:12:48 +01:00