Commit Graph

69676 Commits

Author SHA1 Message Date
Vladimir Sementsov-Ogievskiy
403bb8185e iotests: test external snapshot with bitmap copying
This test shows that external snapshots and incremental backups are
friends.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-id: 20190517152111.206494-3-vsementsov@virtuozzo.com
Signed-off-by: John Snow <jsnow@redhat.com>
2019-05-28 19:33:31 -04:00
Vladimir Sementsov-Ogievskiy
eff0829b07 qapi: support external bitmaps in block-dirty-bitmap-merge
Add new optional parameter making possible to merge bitmaps from
different nodes. It is needed to maintain external snapshots during
incremental backup chain history.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-id: 20190517152111.206494-2-vsementsov@virtuozzo.com
Signed-off-by: John Snow <jsnow@redhat.com>
2019-05-28 19:33:31 -04:00
John Snow
592203e7cf migration/dirty-bitmaps: change bitmap enumeration method
Shift from looking at every root BDS to *every* BDS. This will migrate
bitmaps that are attached to blockdev created nodes instead of just ones
attached to emulated storage devices.

Note that this will not migrate anonymous or internal-use bitmaps, as
those are defined as having no name.

This will also fix the Coverity issues Peter Maydell has been asking
about for the past several releases, as well as fixing a real bug.

Reported-by: Peter Maydell <peter.maydell@linaro.org>
Reported-by: Coverity 😅
Reported-by: aihua liang <aliang@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20190514201926.10407-1-jsnow@redhat.com
Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1652490
Fixes: Coverity CID 1390625
CC: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
2019-05-28 19:33:31 -04:00
John Snow
a2d665c1bc blockdev: loosen restrictions on drive-backup source node
We mandate that the source node must be a root node; but there's no reason
I am aware of that it needs to be restricted to such. In some cases, we need
to make sure that there's a medium present, but in the general case we can
allow the backup job itself to do the graph checking.

This patch helps improve the error message when you try to backup from
the same node more than once, which is reflected in the change to test
056.

For backups with bitmaps, it will also show a better error message that
the bitmap is in use instead of giving you something cryptic like "need
a root node."

Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1707303
Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20190521210053.8864-1-jsnow@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-05-28 20:30:55 +02:00
Andrey Shinkevich
6388903e7c qcow2-bitmap: initialize bitmap directory alignment
Valgrind detects multiple issues in QEMU iotests when the memory is
used without being initialized. Valgrind may dump lots of unnecessary
reports what makes the memory issue analysis harder. Particularly,
that is true for the aligned bitmap directory and can be seen while
running the iotest #169. Padding the aligned space with zeros eases
the pain.

Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Message-id: 1558961521-131620-1-git-send-email-andrey.shinkevich@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-05-28 20:30:55 +02:00
Anton Nefedov
c8bb23cbdb qcow2: skip writing zero buffers to empty COW areas
If COW areas of the newly allocated clusters are zeroes on the backing
image, efficient bdrv_write_zeroes(flags=BDRV_REQ_NO_FALLBACK) can be
used on the whole cluster instead of writing explicit zero buffers later
in perform_cow().

iotest 060:
write to the discarded cluster does not trigger COW anymore.
Use a backing image instead.

Signed-off-by: Anton Nefedov <anton.nefedov@virtuozzo.com>
Message-id: 20190516142749.81019-2-anton.nefedov@virtuozzo.com
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-05-28 20:30:55 +02:00
Sam Eiderman
330c729571 qemu-img: rebase: Reuse in-chain BlockDriverState
If a chain was detected, don't open a new BlockBackend from the target
backing file which will create a new BlockDriverState. Instead, create
an empty BlockBackend and attach the already open BlockDriverState.

Permissions for blk_new() were copied from blk_new_open() when
flags = 0.

Reviewed-by: Karl Heubaum <karl.heubaum@oracle.com>
Reviewed-by: Eyal Moscovici <eyal.moscovici@oracle.com>
Signed-off-by: Sagi Amit <sagi.amit@oracle.com>
Co-developed-by: Sagi Amit <sagi.amit@oracle.com>
Signed-off-by: Sam Eiderman <shmuel.eiderman@oracle.com>
Message-id: 20190523163337.4497-4-shmuel.eiderman@oracle.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-05-28 20:30:55 +02:00
Sam Eiderman
863cc78f1b qemu-img: rebase: Reduce reads on in-chain rebase
In the following case:

(base) A <- B <- C (tip)

when running:

    qemu-img rebase -b A C

QEMU would read all sectors not allocated in the file being rebased (C)
and compare them to the new base image (A), regardless of whether they
were changed or even allocated anywhere along the chain between the new
base and the top image (B). This causes many unneeded reads when
rebasing an image which represents a small diff of a large disk, as it
would read most of the disk's sectors.

Instead, use bdrv_is_allocated_above() to reduce the number of
unnecessary reads.

Reviewed-by: Karl Heubaum <karl.heubaum@oracle.com>
Signed-off-by: Sam Eiderman <shmuel.eiderman@oracle.com>
Signed-off-by: Eyal Moscovici <eyal.moscovici@oracle.com>
Message-id: 20190523163337.4497-3-shmuel.eiderman@oracle.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-05-28 20:30:55 +02:00
Sam Eiderman
4ebe061726 qemu-img: rebase: Reuse parent BlockDriverState
In safe mode we open the entire chain, including the parent backing
file of the rebased file.
Do not open a new BlockBackend for the parent backing file, which
saves opening the rest of the chain twice, which for long chains
saves many "pricy" bdrv_open() calls.

Permissions for blk_new() were copied from blk_new_open() when
flags = 0.

Reviewed-by: Karl Heubaum <karl.heubaum@oracle.com>
Reviewed-by: Eyal Moscovici <eyal.moscovici@oracle.com>
Signed-off-by: Sagi Amit <sagi.amit@oracle.com>
Co-developed-by: Sagi Amit <sagi.amit@oracle.com>
Signed-off-by: Sam Eiderman <shmuel.eiderman@oracle.com>
Message-id: 20190523163337.4497-2-shmuel.eiderman@oracle.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-05-28 20:30:55 +02:00
Alberto Garcia
b441dc71c0 block: Make bdrv_root_attach_child() unref child_bs on failure
A consequence of the previous patch is that bdrv_attach_child()
transfers the reference to child_bs from the caller to parent_bs,
which will drop it on bdrv_close() or when someone calls
bdrv_unref_child().

But this only happens when bdrv_attach_child() succeeds. If it fails
then the caller is responsible for dropping the reference to child_bs.

This patch makes bdrv_attach_child() take the reference also when
there is an error, freeing the caller for having to do it.

A similar situation happens with bdrv_root_attach_child(), so the
changes on this patch affect both functions.

Signed-off-by: Alberto Garcia <berto@igalia.com>
Message-id: 20dfb3d9ccec559cdd1a9690146abad5d204a186.1557754872.git.berto@igalia.com
[mreitz: Removed now superfluous BdrvChild * variable in
         bdrv_open_child()]
Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-05-28 20:30:55 +02:00
Alberto Garcia
dd4118c792 block: Use bdrv_unref_child() for all children in bdrv_close()
bdrv_unref_child() does the following things:

  - Updates the child->bs->inherits_from pointer.
  - Calls bdrv_detach_child() to remove the BdrvChild from bs->children.
  - Calls bdrv_unref() to unref the child BlockDriverState.

When bdrv_unref_child() was introduced in commit 33a604075c it was not
used in bdrv_close() because the drivers that had additional children
(like quorum or blkverify) had already called bdrv_unref() on their
children during their own close functions.

This was changed later (in 0bd6e91a7e for quorum, in 3e586be0b2 for
blkverify) so there's no reason not to use bdrv_unref_child() in
bdrv_close() anymore.

After this there's also no need to remove bs->backing and bs->file
separately from the rest of the children, so bdrv_close() can be
simplified.

Now bdrv_close() unrefs all children (before this patch it was only
bs->file and bs->backing). As a result, none of the callers of
brvd_attach_child() should remove their reference to child_bs (because
this function effectively steals that reference). This patch updates a
couple of tests that were doing their own bdrv_unref().

Signed-off-by: Alberto Garcia <berto@igalia.com>
Message-id: 6d1d5feaa53aa1ab127adb73d605dc4503e3abd5.1557754872.git.berto@igalia.com
[mreitz: s/where/were/]
Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-05-28 20:30:55 +02:00
Vladimir Sementsov-Ogievskiy
ae6b12fa4c block/backup: refactor: split out backup_calculate_cluster_size
Split out cluster_size calculation. Move copy-bitmap creation above
block-job creation, as we are going to share it with upcoming
backup-top filter, which also should be created before actual block job
creation.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-id: 20190429090842.57910-6-vsementsov@virtuozzo.com
[mreitz: Dropped a paragraph from the commit message that was left over
         from a previous version]
Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-05-28 20:30:55 +02:00
Vladimir Sementsov-Ogievskiy
c334e897d0 block/backup: unify different modes code path
Do full, top and incremental mode copying all in one place. This
unifies the code path and helps further improvements.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190429090842.57910-5-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-05-28 20:30:55 +02:00
Vladimir Sementsov-Ogievskiy
9eb5a248f3 block/backup: refactor and tolerate unallocated cluster skipping
Split allocation checking to separate function and reduce nesting.
Consider bdrv_is_allocated() fail as allocated area, as copying more
than needed is not wrong (and we do it anyway) and seems better than
fail the whole job. And, most probably we will fail on the next read,
if there are real problem with source.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190429090842.57910-4-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-05-28 20:30:55 +02:00
Vladimir Sementsov-Ogievskiy
a8389e315e block/backup: move to copy_bitmap with granularity
We are going to share this bitmap between backup and backup-top filter
driver, so let's share something more meaningful. It also simplifies
some calculations.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190429090842.57910-3-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-05-28 20:30:55 +02:00
Vladimir Sementsov-Ogievskiy
c2da3413c0 block/backup: simplify backup_incremental_init_copy_bitmap
Simplify backup_incremental_init_copy_bitmap using the function
bdrv_dirty_bitmap_next_dirty_area.

Note: move to job->len instead of bitmap size: it should not matter but
less code.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190429090842.57910-2-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-05-28 20:30:55 +02:00
Vladimir Sementsov-Ogievskiy
8ac0f15f33 qcow2: do encryption in threads
Do encryption/decryption in threads, like it is already done for
compression. This improves asynchronous encrypted io.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190506142741.41731-9-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-05-28 20:30:55 +02:00
Vladimir Sementsov-Ogievskiy
5447c3a03f qcow2: bdrv_co_pwritev: move encryption code out of the lock
Encryption will be done in threads, to take benefit of it, we should
move it out of the lock first.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190506142741.41731-8-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-05-28 20:30:55 +02:00
Vladimir Sementsov-Ogievskiy
f24196d388 qcow2: qcow2_co_preadv: improve locking
Background: decryption will be done in threads, to take benefit of it,
we should move it out of the lock first.

But let's go further: it turns out, that only
qcow2_get_cluster_offset() needs locking, so reduce locking to it.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-id: 20190506142741.41731-7-vsementsov@virtuozzo.com
Reviewed-by: Alberto Garcia <berto@igalia.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-05-28 20:30:55 +02:00
Vladimir Sementsov-Ogievskiy
6f13a316dd qcow2-threads: split out generic path
Move generic part out of qcow2_co_do_compress, to reuse it for
encryption and rename things that would be shared with encryption path.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190506142741.41731-6-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-05-28 20:30:55 +02:00
Vladimir Sementsov-Ogievskiy
0f5636c51c qcow2-threads: qcow2_co_do_compress: protect queuing by mutex
Drop dependence on AioContext lock.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190506142741.41731-5-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-05-28 20:30:55 +02:00
Vladimir Sementsov-Ogievskiy
269062efc8 qcow2-threads: use thread_pool_submit_co
Use thread_pool_submit_co, instead of reinventing it here. Note, that
thread_pool_submit_aio() never returns NULL, so checking it was an
extra thing.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190506142741.41731-4-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-05-28 20:30:55 +02:00
Vladimir Sementsov-Ogievskiy
56e2f1d898 qcow2: add separate file for threaded data processing functions
Move compression-on-threads to separate file. Encryption will be in it
too.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190506142741.41731-3-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-05-28 20:30:55 +02:00
Vladimir Sementsov-Ogievskiy
9353db47c5 qcow2.h: add missing include
qcow2.h depends on block_int.h. Compilation isn't broken currently only
due to block_int.h always included before qcow2.h. Though, it seems
better to directly include block_int.h in qcow2.h.

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 20190506142741.41731-2-vsementsov@virtuozzo.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2019-05-28 20:30:55 +02:00
Peter Maydell
8c1ecb5904 Various testing updates
- semihosting re-factor (used in system tests)
   - aarch64 and alpha system tests
   - editorconfig tweak for .S
   - some docker image updates
   - iotests clean-up (without make check inclusion)
 -----BEGIN PGP SIGNATURE-----
 
 iQEzBAABCgAdFiEEZoWumedRZ7yvyN81+9DbCVqeKkQFAlztYToACgkQ+9DbCVqe
 KkQU9wf/Uv5qBgDn9MwcCt8tzHTX/i21QHwFLBbCmFoUwZjSridZ2KV6Ma3ig4mF
 xY+8Cr5oZT186V+aD39K6KCZKqZRulIpRVNkOKXEfAAklUoAyQs95Wa8F8LtO1eG
 vOtOYEdkXQQiAnlnQ+eaGiZQ2mpbCbREa10JrBhxp6iXh0PYcvtD7iAlOldqIvd2
 hDRwOgTtYoiiKh6UdediAgQsRvv6oNPHFUOjWgrGxfhPWKbjCVKl7VS4furg9zux
 j/S0E0xYKhj+JNq3arjiMUMl19TauCBQLrbQpphd1jOl1s7bELRjAuaKM60TVIbW
 Hd2/PYbGnkpyUcJQh0Pr1cb4RMcznw==
 =lvtu
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/stsquad/tags/pull-testing-next-280519-2' into staging

Various testing updates

  - semihosting re-factor (used in system tests)
  - aarch64 and alpha system tests
  - editorconfig tweak for .S
  - some docker image updates
  - iotests clean-up (without make check inclusion)

# gpg: Signature made Tue 28 May 2019 17:26:34 BST
# gpg:                using RSA key 6685AE99E75167BCAFC8DF35FBD0DB095A9E2A44
# gpg: Good signature from "Alex Bennée (Master Work Key) <alex.bennee@linaro.org>" [full]
# Primary key fingerprint: 6685 AE99 E751 67BC AFC8  DF35 FBD0 DB09 5A9E 2A44

* remotes/stsquad/tags/pull-testing-next-280519-2: (27 commits)
  tests/qemu-iotests: re-format output to for make check-block
  tests/qemu-iotests/group: Re-use the "auto" group for tests that can always run
  Makefile.target: support per-target coverage reports
  Makefile: include per-target build directories in coverage report
  Makefile: fix coverage-report reference to BUILD_DIR
  .travis.yml: enable aarch64-softmmu and alpha-softmmu tcg tests
  tests/tcg/alpha: add system boot.S
  tests/tcg/multiarch: expand system memory test to cover more
  tests/tcg/minilib: support %c format char
  tests/tcg/multiarch: move the system memory test
  tests/tcg/aarch64: add system boot.S
  editorconfig: add settings for .s/.S files
  tests/tcg/multiarch: add hello world system test
  tests/tcg/multiarch: add support for multiarch system tests
  tests/docker: Test more components on the Fedora default image
  tests/docker: add ubuntu 18.04
  MAINTAINERS: update for semihostings new home
  target/mips: convert UHI_plog to use common semihosting code
  target/mips: only build mips-semi for softmmu
  target/arm: correct return values for WRITE/READ in arm-semi
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-05-28 17:38:32 +01:00
Peter Maydell
4a1d38c440 MIPS queue for May 19th, 2019 - v3
-----BEGIN PGP SIGNATURE-----
 Version: GnuPG v1
 
 iQEcBAABAgAGBQJc6rmrAAoJENSXKoln91pleiIH/iII48dveAe846j+leRJzvq0
 xwykZ+uc1gM8vwHjJmQL2iz8BSx9irYg68jNgZNw+0/7pOqKVkyDt7T9CMpNIgI/
 xsbaX3ta90Rhw6Cnw9h99SsWJAg2Oq4Phv7yCIgDMoNmph/caZyd/95ThmmQ2MXr
 BDOehxODcDyJAO3KIMODccuD6R0ofM2JKXUk2S5s7v1FjYm5fxfzOcx8P4P4lGP3
 AB/k+iVMaEaZJWDHhPXte/5tF9aW8s39CUoe0BE/Cqje3ehLEY/6XxMamXiaWf/G
 2taLvJ6lZ1T5nRmcTCBFfO4Sk3BEBeGlqF72nIlzeCJCe9SvISfwEhsHNkfsqiE=
 =hSSg
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/amarkovic/tags/mips-queue-may-19-2019-v3' into staging

MIPS queue for May 19th, 2019 - v3

# gpg: Signature made Sun 26 May 2019 17:07:07 BST
# gpg:                using RSA key D4972A8967F75A65
# gpg: Good signature from "Aleksandar Markovic <amarkovic@wavecomp.com>" [unknown]
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg:          There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 8526 FBF1 5DA3 811F 4A01  DD75 D497 2A89 67F7 5A65

* remotes/amarkovic/tags/mips-queue-may-19-2019-v3:
  BootLinuxSshTest: Test some userspace commands on Malta
  target/mips: realign comments to fix checkpatch warnings
  target/mips: add or remove space to fix checkpatch errors
  linux-user: fix __NR_semtimedop undeclared error
  mips: Decide to map PAGE_EXEC in map_address
  target/mips: Refactor and fix INSERT.<B|H|W|D> instructions
  target/mips: Refactor and fix COPY_U.<B|H|W> instructions
  target/mips: Refactor and fix COPY_S.<B|H|W|D> instructions
  target/mips: Fix MSA instructions ST.<B|H|W|D> on big endian host
  target/mips: Fix MSA instructions LD.<B|H|W|D> on big endian host
  target/mips: Make the results of MOD_<U|S>.<B|H|W|D> the same as on hardware
  target/mips: Make the results of DIV_<U|S>.<B|H|W|D> the same as on hardware

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-05-28 12:25:20 +01:00
Peter Maydell
4bade28288 RISC-V Patches for the 4.1 Soft Freeze, Part 1
This tag contains a handful of patches that I'd like to target for 4.1:
 
 * An emulation for SiFive's GPIO device.
 * A fix to disallow sfence.vma from userspace.
 * Additional decodetree cleanups that should have no functional impact.
 * C extension emulation fidelity fixes that were noticed as part of that
   cleanup process.
 * A new "spike" target, along with the deprecation of a handful of old
   targets and CPUs.
 * Some initial infastructure related to the hypervisor extension.
 * An emulation fidelity fix that prevents prevents arbitrary bits in the
   SIP CSR from being set.
 * A small performance improvement that avoids excessive TLB flushing
   when the ASID does not change.
 
 This time I've used a new testing workflow: I've tested on both 32-bit
 and 64-bit builds of OpenEmbedded, via the default OpenSBI-based boot
 flow.
 -----BEGIN PGP SIGNATURE-----
 
 iQJHBAABCAAxFiEEAM520YNJYN/OiG3470yhUCzLq0EFAlzohuUTHHBhbG1lckBk
 YWJiZWx0LmNvbQAKCRDvTKFQLMurQfcKD/9xSyQld3g41e4qrcq/MHHQhIou0Mpj
 FdJ9jb5k8ndlhJix1RCJorXQ4ld7KPco5ywdQGebEzrSlsI0hanlWkrtBDibR9dU
 OoNjrBRqKAjTQOkIsmgQMW961n9HnfSZNlmm/L1n94TC/6W4/g+hlvDXQNJjyDKX
 u8QalbfNFlolSv11vOxSlVFTo8Cb6/++yBitvbLdNnquWcDgxid6FYWKMEmYB4jK
 pOTzlb0ZDdvS7rGLdMkTB2pTqrvEykP1uLvXpF7qOurGgoeHul2lqKiRuVcFFU46
 6IOsaiwzDtMoUHzuFZypEJfnm0uJ4FEGEK8ZQBXlQhl2c0MDLXzXhIekKZIs/BmG
 K1Anio30zgfJscA6Zv6ZPataeEbPQ/l0LeAgS/Q8n7zIN5oRi3D9kdNTlo1MoSpD
 RLXESf0uXaTxkOAiyQ2hDj/NuPqja4TFD6UKj91eoWIhcga8i4olqsvLmsTsxhCK
 aaaiLGpYnmm/yhmBjMOAeBnu03w0ApTFS2UoPoMvTnrhZI4OXhm9XABEASpzdh57
 5mJL/U8eJ098QSxBui2Sjcpn8mxfy1FHqAQl+urDCAb6/P/94+729LVzyuvEV08U
 IHYMFjO8v4e56Sf6eys6rcPYpPV/jrLnyL8uFODN1PsXY9+OyH/KLCGVb1vyAgsP
 nhE2uPKTnIrPqA==
 =7LvI
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/palmer/tags/riscv-for-master-4.1-sf0' into staging

RISC-V Patches for the 4.1 Soft Freeze, Part 1

This tag contains a handful of patches that I'd like to target for 4.1:

* An emulation for SiFive's GPIO device.
* A fix to disallow sfence.vma from userspace.
* Additional decodetree cleanups that should have no functional impact.
* C extension emulation fidelity fixes that were noticed as part of that
  cleanup process.
* A new "spike" target, along with the deprecation of a handful of old
  targets and CPUs.
* Some initial infastructure related to the hypervisor extension.
* An emulation fidelity fix that prevents prevents arbitrary bits in the
  SIP CSR from being set.
* A small performance improvement that avoids excessive TLB flushing
  when the ASID does not change.

This time I've used a new testing workflow: I've tested on both 32-bit
and 64-bit builds of OpenEmbedded, via the default OpenSBI-based boot
flow.

# gpg: Signature made Sat 25 May 2019 01:05:57 BST
# gpg:                using RSA key 00CE76D1834960DFCE886DF8EF4CA1502CCBAB41
# gpg:                issuer "palmer@dabbelt.com"
# gpg: Good signature from "Palmer Dabbelt <palmer@dabbelt.com>" [unknown]
# gpg:                 aka "Palmer Dabbelt <palmer@sifive.com>" [unknown]
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg:          There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 00CE 76D1 8349 60DF CE88  6DF8 EF4C A150 2CCB AB41

* remotes/palmer/tags/riscv-for-master-4.1-sf0: (29 commits)
  target/riscv: Only flush TLB if SATP.ASID changes
  target/riscv: More accurate handling of `sip` CSR
  target/riscv: Add checks for several RVC reserved operands
  target/riscv: Add the HGATP register masks
  target/riscv: Add the HSTATUS register masks
  target/riscv: Add Hypervisor CSR macros
  target/riscv: Allow setting mstatus virtulisation bits
  target/riscv: Add the MPV and MTL mstatus bits
  target/riscv: Improve the scause logic
  target/riscv: Trigger interrupt on MIP update asynchronously
  target/riscv: Mark privilege level 2 as reserved
  riscv: spike: Add a generic spike machine
  target/riscv: Deprecate the generic no MMU CPUs
  target/riscv: Add a base 32 and 64 bit CPU
  target/riscv: Create settable CPU properties
  riscv: virt: Allow specifying a CPU via commandline
  linux-user/riscv: Add the CPU type as a comment
  target/riscv: Remove unused include of riscv_htif.h for virt board riscv
  target/riscv: Remove spaces from register names
  target/riscv: Split gen_arith_imm into functional and temp
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-05-28 11:52:53 +01:00
Peter Maydell
2b01c1b382 Machine Core queue, 2019-05-24
* Display more helpful message when an object type is missing
   (Philippe Mathieu-Daudé)
 * Use object_initialize_child for correct reference counting
   (Philippe Mathieu-Daudé)
 -----BEGIN PGP SIGNATURE-----
 
 iQIcBAABCAAGBQJc6DhqAAoJECgHk2+YTcWmzhkP/A89XUshKY+FUGrSaMJy+ibQ
 RqTvkQU3ShzejIfmtZKgsq0wxeSAIbXY0OYlDMXusMxcb1JQN4lhQQn2GYty0pVq
 sTUPFWrFuFaxrFJBcjD93TEUUuvKqQJbuAqCCDyhFhGr784mtrZ4gplmXLu1MMLH
 4UtLx8Wg8f3UFzj6IjznFDjs4eXiHS8AwhTLfy2fS7cHT5Hh+JqMorPSkOLNmON1
 L0PTgHix14nX0VwHg/EHmLHwUWXoYUg0VShoFpRHZQTo8MjeHpk3xBm8UnRKN03P
 bKqGipemx2pawQpH4F55EsWFAWgxzM9P/6j4cET7/exTi3hgGh3iE9V3CSGpvHe9
 yf9rgAUadGDUteTrZi6t+BTzgxDBPLbuZaiIHArA+uFzrRJC8EAr6iojV+PYU3MI
 wlhF6GsagvcHgRHuVlxAY92Pc/5YV4s6ySMh1GB0yZ4LwMT0UwP+08Il+ZCul+jp
 pY92bN83o+fgVIGf7gbabDDURcHhsSa7DRWMb9W0b8fT1gEXtjkOP6wiL7HkSDVi
 quSdBaC3bDLBVrwiaLVPm0hNgk/ZjN7VeCIYdapLXmLFAauYvP6BW+Py+5E/VM1n
 WYCTAF9un0n5NiV3ltDkgzb2Nbrc2tKmRyIXE1UcsSAqlrvt2nAQP3GhmURbuMux
 PpZHe1N+2S5yaSk4rhTo
 =m2EC
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/ehabkost/tags/machine-next-pull-request' into staging

Machine Core queue, 2019-05-24

* Display more helpful message when an object type is missing
  (Philippe Mathieu-Daudé)
* Use object_initialize_child for correct reference counting
  (Philippe Mathieu-Daudé)

# gpg: Signature made Fri 24 May 2019 19:31:06 BST
# gpg:                using RSA key 2807936F984DC5A6
# gpg: Good signature from "Eduardo Habkost <ehabkost@redhat.com>" [full]
# Primary key fingerprint: 5A32 2FD5 ABC4 D3DB ACCF  D1AA 2807 936F 984D C5A6

* remotes/ehabkost/tags/machine-next-pull-request:
  hw/intc/nvic: Use object_initialize_child for correct reference counting
  hw/arm/mps2: Use object_initialize_child for correct reference counting
  hw/microblaze/zynqmp: Use object_initialize_child for correct ref. counting
  hw/microblaze/zynqmp: Use object_initialize_child for correct ref. counting
  hw/microblaze/zynqmp: Let the SoC manage the IPI devices
  hw/microblaze/zynqmp: Move the IPI state into the PMUSoC state
  hw/mips: Use object_initialize_child for correct reference counting
  hw/mips: Use object_initialize() on MIPSCPSState
  hw/arm: Use object_initialize_child for correct reference counting
  hw/arm/aspeed: Use object_initialize_child for correct ref. counting
  hw/arm/bcm2835: Use object_initialize_child for correct ref. counting
  hw/arm/bcm2835: Use object_initialize() on PL011State
  hw/arm/bcm2835: Use TYPE_PL011 instead of hardcoded string
  hw/virtio: Use object_initialize_child for correct reference counting
  hw/misc/macio: Use object_initialize_child for correct ref. counting
  hw/ppc/pnv: Use object_initialize_child for correct reference counting
  qom/object: Display more helpful message when an object type is missing

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2019-05-28 10:50:09 +01:00
Alex Bennée
70ff5b07fc tests/qemu-iotests: re-format output to for make check-block
This attempts to clean-up the output to better match the output of the
rest of the QEMU check system when called with -makecheck. This includes:

  - formatting as "  TEST    iotest-FMT: nnn"
  - only dumping config on failure (when -makecheck enabled)

The non-make check output has been cleaned up as well:

  - line re-displayed (\r) at the end
  - fancy colours for pass/fail/skip
  - timestamps always printed (option removed)

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20190503143904.31211-1-alex.bennee@linaro.org>
Tested-by: Thomas Huth <thuth@redhat.com>
2019-05-28 10:28:51 +01:00
Thomas Huth
d4271ce5b5 tests/qemu-iotests/group: Re-use the "auto" group for tests that can always run
Currently, all tests are in the "auto" group. This is a little bit pointless.
OTOH, we need a group for the tests that we can automatically run during
"make check" each time, too. Tests in this new group are supposed to run
with every possible QEMU configuration, for example they must run with every
QEMU binary (also non-x86), without failing when an optional features is
missing (but reporting "skip" is ok), and be able to run on all kind of host
filesystems and users (i.e. also as "nobody" or "root").
So let's use the "auto" group for this class of tests now. The initial
list has been determined by running the iotests with non-x86 QEMU targets
and with our CI pipelines on Gitlab, Cirrus-CI and Travis (i.e. including
macOS and FreeBSD).

Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20190502084506.8009-7-thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-05-28 10:28:51 +01:00
Alex Bennée
90e671a9b2 Makefile.target: support per-target coverage reports
Add support for generating a single targets coverage report. Execute:

  make coverage-report

In the target build directory. This coverage report only cares about
target specific blobs so only searches the target build subdirectory.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
2019-05-28 10:28:51 +01:00
Alex Bennée
5eda6e42cc Makefile: include per-target build directories in coverage report
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
2019-05-28 10:28:51 +01:00
Alex Bennée
30b3718bb9 Makefile: fix coverage-report reference to BUILD_DIR
Commit 337f2311f actually claimed to do this in the commit log but
didn't actually. Oops.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
2019-05-28 10:28:51 +01:00
Alex Bennée
591ff1ffd8 .travis.yml: enable aarch64-softmmu and alpha-softmmu tcg tests
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
2019-05-28 10:28:51 +01:00
Richard Henderson
8d8a8ab544 tests/tcg/alpha: add system boot.S
This provides the bootstrap and low level helper functions for an
alpha kernel.  We use direct access to the DP264 serial port for
test output, and hard machine halt to exit the emulation.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20190501184306.15208-1-richard.henderson@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2019-05-28 10:28:51 +01:00
Alex Bennée
7ac283e963 tests/tcg/multiarch: expand system memory test to cover more
Expand the memory test to cover move of the softmmu code. Specifically
we:

  - improve commentary
  - add some helpers (for later BE support)
  - reduce boiler plate into helpers
  - add signed reads at various sizes/offsets
  - required -DCHECK_UNALIGNED

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
2019-05-28 10:28:51 +01:00
Alex Bennée
936647d3fc tests/tcg/minilib: support %c format char
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
2019-05-28 10:28:51 +01:00
Alex Bennée
67f3da9d74 tests/tcg/multiarch: move the system memory test
There is nothing inherently architecture specific about the memory
test although we may have to manage different restrictions of
unaligned access across architectures.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
2019-05-28 10:28:51 +01:00
Alex Bennée
f6482872f3 tests/tcg/aarch64: add system boot.S
This provides the bootstrap and low level helper functions for an
aarch64 kernel. We use semihosting to handle test output and exiting
the emulation. semihosting's parameter passing is a little funky so we
end up using the stack and pointing to that as the parameter block.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
2019-05-28 10:28:51 +01:00
Alex Bennée
9699e5fb5a editorconfig: add settings for .s/.S files
We are starting to add assembler foe tests/tcg so lets make sure we
get the mode right.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
2019-05-28 10:28:51 +01:00
Alex Bennée
49d755d02e tests/tcg/multiarch: add hello world system test
This is not really i386 only, we can have the same test for all
architectures supporting system tests.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
2019-05-28 10:28:51 +01:00
Alex Bennée
8041650aef tests/tcg/multiarch: add support for multiarch system tests
We can certainly support some common tests for system emulation that
make use of our minimal defined boot.S support. It will still be up to
individual architectures to ensure they build so we provide a
MULTIARCH_TESTS variable that they can tack onto TESTS themselves.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
2019-05-28 10:28:51 +01:00
Philippe Mathieu-Daudé
8ea5962f28 tests/docker: Test more components on the Fedora default image
Install optional dependencies of QEMU to get better coverage.

The following components are now enabled:

  $ ./configure
  ...
  Multipath support yes
  VNC SASL support  yes
  RDMA support      yes
  PVRDMA support    yes
  libiscsi support  yes
  seccomp support   yes
  libpmem support   yes
  libudev           yes

Note: The udev-devel package is provided by systemd-devel.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20190504055440.20406-1-philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Acked-by: Marcel Apfelbaum<marcel.apfelbaum@gmail.com>
2019-05-28 10:28:51 +01:00
Gerd Hoffmann
e8e875b1a3 tests/docker: add ubuntu 18.04
Based on the ubuntu.docker file.
Used to reproduce the build failure Peter was seeing.
Others might find this useful too ;)

Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Message-Id: <20190503070241.24786-1-kraxel@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-05-28 10:28:51 +01:00
Alex Bennée
b052fec8d9 MAINTAINERS: update for semihostings new home
Seeing as I touched it I should at least keep an eye on it.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
2019-05-28 10:28:51 +01:00
Alex Bennée
54eb6cda9d target/mips: convert UHI_plog to use common semihosting code
Rather than printing directly to stdout lets use our common
semihosting code. There is one minor difference in that the output
currently defaults to stderr instead of stdout however this can be
controlled by connecting semihosting to a chardev.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Aleksandar Markovic <amarkovic@wavecomp.com>
2019-05-28 10:28:51 +01:00
Alex Bennée
82ba42666c target/mips: only build mips-semi for softmmu
The is_uhi gates all semihosting calls and always returns false for
CONFIG_USER_ONLY builds. There is no reason to build and link
mips-semi for these builds so lets fix that.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Aleksandar Markovic <amarkovic@wavecomp.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
2019-05-28 10:28:51 +01:00
Alex Bennée
629a0b065a target/arm: correct return values for WRITE/READ in arm-semi
The documentation says the write should return the number of bytes not
written on an error (0 means everything was written). Read provides a
buffer length and the return value should be the buffer length - bytes
actually read. Remove the incorrect FIXME's and return the correct
values.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2019-05-28 10:28:51 +01:00
Alex Bennée
a131795f4c target/arm: add LOG_UNIMP messages to arm-semi
Clean-up our unimplemented bits with a proper message.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
2019-05-28 10:28:51 +01:00
Alex Bennée
0dc077212f target/arm: use the common interface for WRITE0/WRITEC in arm-semi
Now we have a common semihosting console interface use that for our
string output. However ARM is currently unique in also supporting
semihosting for linux-user so we need to replicate the API in
linux-user. If other architectures gain this support we can move the
file later.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
2019-05-28 10:28:50 +01:00