Commit Graph

38398 Commits

Author SHA1 Message Date
Peter Maydell
06feaacfb4 - miscellaneous cleanups for TCG (Emilio) and NBD (Bogdan)
- next part in the thread-safe address_space_* saga: atomic access
   to the bounce buffer and the map_clients list, from Fam
 - optional support for linking with tcmalloc, also from Fam
 - reapplying Peter Crosthwaite's "Respect as_translate_internal
   length clamp" after fixing the SPARC fallout.
 - build system fix from Wei Liu
 - small acpi-build and ioport cleanup by myself
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v2
 
 iQEcBAABCAAGBQJVQJd4AAoJEL/70l94x66DYFYH/3ifhqWZsd4dfJri0CGAHI4i
 SpPmNeouc8W+F/3lwf6Inrh5NnTgd5QzoUBMQaWVkQKwUiWls8g2mXkT3jo0iDqT
 /B40YXnZjNm20MixNaZmk9AsOF6OqPM8EMufau874k5zTlx3tCGAW1QD+I1N7WK7
 DfsFsIUD1svo2prn55fSoitMG1TIVPnpcklb4YGJRbAacQYUDhr5KAIhT1quDR2R
 93BvToyQmPqRQ4YKqnJLp8HAkL4FaJumfFZVvyh2cZvyaYGN/RVdi2Dw985dJDPX
 /z4enE4GCAs4RDw3lZ1RDbiZDqpT2ibFgASg/arX3SxzqHirOGvMdkOjO99r9j4=
 =aLjh
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into staging

- miscellaneous cleanups for TCG (Emilio) and NBD (Bogdan)
- next part in the thread-safe address_space_* saga: atomic access
  to the bounce buffer and the map_clients list, from Fam
- optional support for linking with tcmalloc, also from Fam
- reapplying Peter Crosthwaite's "Respect as_translate_internal
  length clamp" after fixing the SPARC fallout.
- build system fix from Wei Liu
- small acpi-build and ioport cleanup by myself

# gpg: Signature made Wed Apr 29 09:34:00 2015 BST using RSA key ID 78C7AE83
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>"
# gpg:                 aka "Paolo Bonzini <pbonzini@redhat.com>"
# gpg: WARNING: This key is not certified with sufficiently trusted signatures!
# gpg:          It is not certain that the signature belongs to the owner.
# Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4  E2F7 7E15 100C CD36 69B1
#      Subkey fingerprint: F133 3857 4B66 2389 866C  7682 BFFB D25F 78C7 AE83

* remotes/bonzini/tags/for-upstream: (22 commits)
  nbd/trivial: fix type cast for ioctl
  translate-all: use bitmap helpers for PageDesc's bitmap
  target-i386: disable LINT0 after reset
  Makefile.target: prepend $libs_softmmu to $LIBS
  milkymist: do not modify libs-softmmu
  configure: Add support for tcmalloc
  exec: Respect as_translate_internal length clamp
  ioport: reserve the whole range of an I/O port in the AddressSpace
  ioport: loosen assertions on emulation of 16-bit ports
  ioport: remove wrong comment
  ide: there is only one data port
  gus: clean up MemoryRegionPortio
  sb16: remove useless mixer_write_indexw
  sun4m: fix slavio sysctrl and led register sizes
  acpi-build: remove dependency from ram_addr.h
  memory: add memory_region_ram_resize
  dma-helpers: Fix race condition of continue_after_map_failure and dma_aio_cancel
  exec: Notify cpu_register_map_client caller if the bounce buffer is available
  exec: Protect map_client_list with mutex
  linux-user, bsd-user: Remove two calls to cpu_exec_init_all
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2015-04-30 12:04:11 +01:00
Peter Maydell
a1fe58f6ad -----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
 
 iQIcBAABAgAGBQJVQBHQAAoJEH3vgQaq/DkOPd4QAIqIcGHd6Frr9ZFnETQ3WN0S
 TzQ834zgNV/zdrnSGoSF3Abcurq1XYEqqh3HUIDpOwqO0ZeeLXjrA1WG7HxAEb/y
 Cdcqg9isfD4am8dREgPKkKRVoKosxu1CJcwXcnu67USsX8MpyiZc6TD/Rmyjq5qO
 HtauRj82NffMPz1QtydwOvoRvlqmuyiGkuCThnaBgMDcBXtcIWmT2YjtGn93nEY2
 i92m/cJyeRc/RZMByo2FOe3fGzt46tj3GNcDM2MrhrEOKaBGfzyvQa9hqBhTY5KE
 a1HwGtXYUPB0cXUmbfEMVUoQ1JUa4hFG8RVOJElt78GYpJhn5S6OF/PQO6JRLGJO
 WxcjDUDlIAJ+Vv0OD1xKbhaeM/mFF5beL9WbB3ow8pwRw9Mj/tn3IYr1WPUTb1dO
 +xd9MAJByEi2bAeiQ/WZB2tvRdrqJZDlFs5N/OB+VLTEmgbhXBZ++M98IR9O6dP3
 chdctCfXSh/XrwskJ66La2SORpi+D6VRTkiiPe0OBaR+VKHI+UxLA5ie8LUxpVAj
 OStt3KXSnLlL76KZW88eWw0416a3BxmHWHKyG3azFWTBCPiEwfSexjjlX3o0uRAW
 Vj9/SpEKi29XIIWKwgxv8NHjONtCQp+6TYSY+g06Lqy87taO363z4JBtDWp8Gq6B
 UlNzwmHqp13c3Tn4jgQk
 =Kpae
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/jnsnow/tags/ide-pull-request' into staging

# gpg: Signature made Wed Apr 29 00:03:44 2015 BST using RSA key ID AAFC390E
# gpg: Good signature from "John Snow (John Huston) <jsnow@redhat.com>"
# gpg: WARNING: This key is not certified with sufficiently trusted signatures!
# gpg:          It is not certain that the signature belongs to the owner.
# Primary key fingerprint: FAEB 9711 A12C F475 812F  18F2 88A9 064D 1835 61EB
#      Subkey fingerprint: F9B7 ABDB BCAC DF95 BE76  CBD0 7DEF 8106 AAFC 390E

* remotes/jnsnow/tags/ide-pull-request:
  qtest: Add assertion that required environment variable is set
  qtest/ahci: add flush retry test
  libqos: add blkdebug_prepare_script
  libqtest: add qmp_async
  libqtest: add qmp_eventwait
  qtest/ahci: Allow override of default CLI options
  qtest/ahci: Add simple flush test
  qtest/ahci: test different disk sectors
  qtest/ahci: add qcow2 support to ahci-test
  fdc: remove sparc sun4m mutations

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2015-04-30 10:10:31 +01:00
Bogdan Purcareata
d064d9f381 nbd/trivial: fix type cast for ioctl
This fixes ioctl behavior on powerpc e6500 platforms with 64bit kernel and 32bit
userspace. The current type cast has no effect there and the value passed to the
kernel is still 0. Probably an issue related to the compiler, since I'm assuming
the same configuration works on a similar setup on x86.

Also ensure consistency with previous type cast in TRACE message.

Signed-off-by: Bogdan Purcareata <bogdan.purcareata@freescale.com>
Message-Id: <1428058914-32050-1-git-send-email-bogdan.purcareata@freescale.com>
Cc: qemu-stable@nongnu.org
[Fix parens as noticed by Michael. - Paolo]
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2015-04-28 22:14:15 +02:00
Emilio G. Cota
510a647fa2 translate-all: use bitmap helpers for PageDesc's bitmap
Here we have an open-coded byte-based bitmap implementation.
Get rid of it since there's a ulong-based implementation to be
used by all code.

Signed-off-by: Emilio G. Cota <cota@braap.org>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2015-04-28 22:14:14 +02:00
Nadav Amit
b8eb5512fd target-i386: disable LINT0 after reset
Due to old Seabios bug, QEMU reenable LINT0 after reset. This bug is long gone
and therefore this hack is no longer needed.  Since it violates the
specifications, it is removed.

Signed-off-by: Nadav Amit <namit@cs.technion.ac.il>
Message-Id: <1428881529-29459-2-git-send-email-namit@cs.technion.ac.il>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2015-04-28 22:14:14 +02:00
Wei Liu
7398dfc779 Makefile.target: prepend $libs_softmmu to $LIBS
I discovered a problem when trying to build QEMU statically with gcc.
libm is an element of LIBS while libpixman-1 is an element in
libs_softmmu. Libpixman references functions in libm, so the original
ordering makes linking fail.

This fix is to reorder $libs_softmmu and $LIBS to make -lm appear after
-lpixman-1. However I'm not quite sure if this is the right fix, hence
the RFC tag.

Normally QEMU is built with c++ compiler which happens to link in libm
(at least this is the case with g++), so building QEMU statically
normally just works and nobody notices this issue.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
Message-Id: <1425912873-21215-1-git-send-email-wei.liu2@citrix.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2015-04-28 22:14:14 +02:00
Paolo Bonzini
738e4171de milkymist: do not modify libs-softmmu
This is better and prepares for the next patch.  When we copy
libs_softmmu's value into LIBS with a := assignment, we cannot
anymore modify libs_softmmu in the Makefiles.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2015-04-28 22:14:14 +02:00
Fam Zheng
2847b46958 configure: Add support for tcmalloc
This adds "--enable-tcmalloc" and "--disable-tcmalloc" to allow linking
to libtcmalloc from gperftools.

tcmalloc is a malloc implementation that works well with threads and is
fast, so it is good for performance.

It is disabled by default, because the MALLOC_PERTURB_ flag we use in
tests doesn't work with tcmalloc. However we can enable tcmalloc
specific heap checker and profilers later.

An IOPS gain can be observed with virtio-blk-dataplane, other parts of
QEMU will directly benefit from it as well:

==========================================================
                       glibc malloc
----------------------------------------------------------
rw         bs         iodepth    bw     iops       latency
read       4k         1          150    38511      24
----------------------------------------------------------

==========================================================
                         tcmalloc
----------------------------------------------------------
rw         bs         iodepth    bw     iops       latency
read       4k         1          156    39969      23
----------------------------------------------------------

Signed-off-by: Fam Zheng <famz@redhat.com>
Message-Id: <1427338992-27057-1-git-send-email-famz@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2015-04-28 22:14:11 +02:00
Ed Maste
c836867498 qtest: Add assertion that required environment variable is set
Signed-off-by: Ed Maste <emaste@freebsd.org>
Reviewed-by: John Snow <jsnow@redhat.com>
Message-id: 1427911244-22565-1-git-send-email-emaste@freebsd.org
Signed-off-by: John Snow <jsnow@redhat.com>
2015-04-28 15:27:51 -04:00
John Snow
cf5aa89e9d qtest/ahci: add flush retry test
Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 1426018503-821-7-git-send-email-jsnow@redhat.com
2015-04-28 15:27:51 -04:00
John Snow
72c85e949f libqos: add blkdebug_prepare_script
Pull this helper out of ide-test and into libqos,
to be shared with ahci-test.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 1426018503-821-6-git-send-email-jsnow@redhat.com
2015-04-28 15:27:51 -04:00
John Snow
ba4ed39346 libqtest: add qmp_async
Add qmp_async, which lets us send QMP commands asynchronously.
This is useful when we want to send commands that will trigger
event responses, but we don't know in what order to expect them.

Sometimes the event responses may arrive even before the command
confirmation will show up, so it is convenient to leave the responses
in the stream.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 1426018503-821-5-git-send-email-jsnow@redhat.com
2015-04-28 15:27:51 -04:00
John Snow
8fe941f749 libqtest: add qmp_eventwait
Allow the user to poll until a desired interrupt occurs.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 1426018503-821-4-git-send-email-jsnow@redhat.com
2015-04-28 15:27:51 -04:00
John Snow
debaaa114a qtest/ahci: Allow override of default CLI options
Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 1426018503-821-3-git-send-email-jsnow@redhat.com
2015-04-28 15:27:51 -04:00
John Snow
4e217074ca qtest/ahci: Add simple flush test
Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 1426018503-821-2-git-send-email-jsnow@redhat.com
2015-04-28 15:27:51 -04:00
John Snow
727be1a755 qtest/ahci: test different disk sectors
Test sector offset 0, 1, and the last sector(s)
in LBA28 and LBA48 modes.

Signed-off-by: John Snow <jsnow@redhat.com>
Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 1426274523-22661-3-git-send-email-jsnow@redhat.com
2015-04-28 15:27:51 -04:00
John Snow
122fdf2d88 qtest/ahci: add qcow2 support to ahci-test
This will enable the testing of high offsets without
wasting a lot of disk space, and does not impact the
previous tests.

mkimg and mkqcow2 are added to libqos for other tests.

Signed-off-by: John Snow <jsnow@redhat.com>
Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 1426274523-22661-2-git-send-email-jsnow@redhat.com
2015-04-28 15:27:51 -04:00
Hervé Poussineau
24a5c62cfe fdc: remove sparc sun4m mutations
They were introduced in 6f7e9aec5e and
82407d1a40 and lots of bug fixes were done after that.

This fixes (at least) the detection of the floppy controller on Debian 4.0r9/SPARC,
and SS-5's OBP initialization routine still works.

Signed-off-by: Hervé Poussineau <hpoussin@reactos.org>
Tested-by: Mark Cave-Ayland <mark.cave-ayland@ilande.co.uk>
Message-id: 1426351846-6497-1-git-send-email-hpoussin@reactos.org
Signed-off-by: John Snow <jsnow@redhat.com>
2015-04-28 15:27:51 -04:00
Peter Maydell
52b7aba62f VFIO updates
- Correction to BAR overflow
  - Fix error sign
  - Reset workaround for AMD Bonaire & Hawaii GPUs
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v1
 
 iQIcBAABAgAGBQJVP8LTAAoJECObm247sIsiKiQQAKVLMuxjSSUSDlu5L2tXWUIA
 EjP3S7Ir8gGgCTNQ97wYGHb/yuIe36z4TP6aTaXB+F6i7PxZIvObnMm35C08BZIg
 9WM5ts6duvrPHr2gvhtm9P0WJhtzUbIhGpJ2oI6dLXOHcqdSe6T5OKp23kERJ0Q1
 7Il0jk59cs5ts82mkFauQp7jmXBUwjChxDPVXUCqu4SF6MnBVGTXUv7kp/dOv9NW
 9BJ+SnHPiVfAqSr+ASZ0jzeR4uOALQiJs8qwiSGSahAxbC2qLoknYl60OTq8zCWL
 GWIkFUqeVcdX6+0pheWll4apK94Leu8esSs89fjJe9qyBMLCzKoemG8qVMuRkB9H
 fPjeyePeuTvw190eKYdrvYY6FAAGLGc4TAY0peU+5FZRXJIaEDehmq3/6H7gSQ5l
 a7CugBh90uwZ99qCtTaWgPQU2rELxZKomA6+V6An7rDh0peSevtLNcpHsMln81RW
 k70LTSKk0JDfUWiZntjC4oTBzBZfetU2Git5VCwrRKmtGs86xIuZvA0w8a3bDNMR
 mFPvgw0CZF2hExcxaEFo655YJWpWNCKfR4WJ7chMsPLry4QJwaKmQYUekvsLaxQl
 RCmG80fjR+l3uJz/MULonYLICx1sknSkCjrKwS7obsOeRS5fHvRApaw3e7EkoIra
 r3tVXR847/o95vVlEf+n
 =yNld
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/awilliam/tags/vfio-update-20150428.0' into staging

VFIO updates
 - Correction to BAR overflow
 - Fix error sign
 - Reset workaround for AMD Bonaire & Hawaii GPUs

# gpg: Signature made Tue Apr 28 18:26:43 2015 BST using RSA key ID 3BB08B22
# gpg: Good signature from "Alex Williamson <alex.williamson@redhat.com>"
# gpg:                 aka "Alex Williamson <alex@shazbot.org>"
# gpg:                 aka "Alex Williamson <alwillia@redhat.com>"
# gpg:                 aka "Alex Williamson <alex.l.williamson@gmail.com>"

* remotes/awilliam/tags/vfio-update-20150428.0:
  vfio-pci: Reset workaround for AMD Bonaire and Hawaii GPUs
  vfio-pci: Fix error path sign
  vfio-pci: Further fix BAR size overflow

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2015-04-28 18:58:15 +01:00
Alex Williamson
5655f931ab vfio-pci: Reset workaround for AMD Bonaire and Hawaii GPUs
Somehow these GPUs manage not to respond to a PCI bus reset, removing
our primary mechanism for resetting graphics cards.  The result is
that these devices typically work well for a single VM boot.  If the
VM is rebooted or restarted, the guest driver is not able to init the
card from the dirty state, resulting in a blue screen for Windows
guests.

The workaround is to use a device specific reset.  This is not 100%
reliable though since it depends on the incoming state of the device,
but it substantially improves the usability of these devices in a VM.

Credit to Alex Deucher <alexander.deucher@amd.com> for his guidance.

Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
2015-04-28 11:14:02 -06:00
Alex Williamson
c6d231e2fd vfio-pci: Fix error path sign
This is an impossible error path due to the fact that we're reading a
kernel provided, rather than user provided link, which will certainly
always fit in PATH_MAX.  Currently it returns a fixed 26 char path
plus %d group number, which typically maxes out at double digits.
However, the caller of the initfn certainly expects a less-than zero
return value on error, not just a non-zero value.  Therefore we
should correct the sign here.

Reported-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
2015-04-28 11:14:02 -06:00
Alex Williamson
07ceaf9880 vfio-pci: Further fix BAR size overflow
In an analysis by Laszlo, the resulting type of our calculation for
the end of the MSI-X table, and thus the start of memory after the
table, is uint32_t.  We're therefore not correctly preventing the
corner case overflow that we intended to fix here where a BAR >=4G
could place the MSI-X table to end exactly at the 4G boundary.  The
MSI-X table offset is defined by the hardware spec to 32bits, so we
simply use a cast rather than changing data structure types.  This
scenario is purely theoretically, typically the MSI-X table is located
at the front of the BAR.

Reported-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
2015-04-28 11:14:02 -06:00
Peter Maydell
a9392bc93c Block patches
-----BEGIN PGP SIGNATURE-----
 Version: GnuPG v2.0.22 (GNU/Linux)
 
 iQIcBAABAgAGBQJVP5qZAAoJEH8JsnLIjy/WpUoP/2T7iyDQgcqvbLbnkoOz+Kq6
 /bll6VhLDzkRmgi3talyNOhEu0jAzvHq4BmA76WSxKme7/ykSx1skYsVmbRzLqcO
 4pun8Y+Ym074z5A65JgkuwOsa8ZhVwVtd4JkAjmeG5wZZsdJkK1YXt805eDYlAaf
 zA0J0oSVm4NqOAfqRI4NfWWYHQ16L6oPpgQ5e45ljcbh2nfNaZCr8qk1xZW5x3/K
 62DnOdX5cmBClemxeM8uy2u6sfFMy2+DKyuY4hNoRhDKbulYgrRTLZRTNCnZw44Y
 ZjaqudxV4DR4tEalhEkLsy1LGt+I/saP1M/jtZzwg2tRbQi6vQGNYTrlLJjMXDkW
 MyACmdtIc5zmsrPDiKLjEg3h3pRn5vj6TheOtIN4ILZ/elSuJbKgV7M5f3isrBZw
 czavCjKkVCKe4L+beMZ1yNRpy1mSbpj1Zlxxpawok6isOxkyqFtqKidiq5g2F6r+
 zGEk2ZBRlYQHyc979ZdbKtOEFFjzli0WjN2ul7ObhhkPKT8bRPuU8fPgwBQuoahH
 5NM/pncB6AusY9b50GiPKEmtHnf86KO/1MWOp/2eLq4oyHt0pgPpwAiIeeu24Z5a
 pBvsz2+a4ysvl+g8JYIgrg+ME2ym4UTr+BX+kYEU54X2NZE7A7iCmmYd8yai/hJ8
 Nu783KGxQV0iScJwqnVK
 =rl85
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block patches

# gpg: Signature made Tue Apr 28 15:35:05 2015 BST using RSA key ID C88F2FD6
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>"

* remotes/kevin/tags/for-upstream: (76 commits)
  block: move I/O request processing to block/io.c
  block: extract bdrv_setup_io_funcs()
  block: add bdrv_set_dirty()/bdrv_reset_dirty() to block_int.h
  block: replace bdrv_states iteration with bdrv_next()
  vmdk: Widen before shifting 32 bit header field
  block/dmg: make it modular
  block/mirror: Always call block_job_sleep_ns()
  iotests: add incremental backup granularity tests
  iotests: add incremental backup failure recovery test
  iotests: add simple incremental backup case
  iotests: add QMP event waiting queue
  iotests: add invalid input incremental backup tests
  hbitmap: truncate tests
  block: Resize bitmaps on bdrv_truncate
  block: Ensure consistent bitmap function prototypes
  block: add BdrvDirtyBitmap documentation
  qmp: Add dirty bitmap status field in query-block
  qmp: add block-dirty-bitmap-clear
  qmp: Add support of "dirty-bitmap" sync mode for drive-backup
  block: Add bitmap successors
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2015-04-28 16:55:03 +01:00
Stefan Hajnoczi
61007b316c block: move I/O request processing to block/io.c
The block.c file has grown to over 6000 lines.  It is time to split this
file so there are fewer conflicts and the code is easier to maintain.

Extract I/O request processing code:
 * Read
 * Write
 * Zero writes and making the image empty
 * Flush
 * Discard
 * ioctl
 * Tracked requests and queuing
 * Throttling and copy-on-read
 * Block status and allocated functions
 * Refreshing block limits
 * Reading/writing vmstate
 * qemu_blockalign() and friends

The patch simply moves code from block.c into block/io.c.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2015-04-28 15:36:17 +02:00
Stefan Hajnoczi
0eb7217e49 block: extract bdrv_setup_io_funcs()
Move the code to install coroutine and aio emulation function pointers
in a BlockDriver to its own function.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2015-04-28 15:36:17 +02:00
Stefan Hajnoczi
e0c47b6cb1 block: add bdrv_set_dirty()/bdrv_reset_dirty() to block_int.h
The dirty bitmap functions are called from the block I/O processing
code.  Make them visible to block_int.h users so they can be used
outside block.c.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2015-04-28 15:36:17 +02:00
Stefan Hajnoczi
4f5472cb2d block: replace bdrv_states iteration with bdrv_next()
The bdrv_states list is a static variable in block.c.

bdrv_drain_all() and bdrv_flush_all() use this variable to iterate over
all drives.

The next patch will move bdrv_drain_all() and bdrv_flush_all() out of
block.c so it's necessary to switch to the public bdrv_next() interface.

Reviewed-by: Alberto Garcia <berto@igalia.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2015-04-28 15:36:17 +02:00
Fam Zheng
7237aecd7e vmdk: Widen before shifting 32 bit header field
Coverity spotted this.

The field is 32 bits, but if it's possible to overflow in 32 bit
left shift.

Signed-off-by: Fam Zheng <famz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2015-04-28 15:36:11 +02:00
Michael Tokarev
5505e8b76f block/dmg: make it modular
dmg can optionally utilize libbz2, make it modular

Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2015-04-28 15:36:11 +02:00
Max Reitz
001c95b740 block/mirror: Always call block_job_sleep_ns()
The mirror block job is trying to take a clever shortcut if delay_ns is
0 and skips block_job_sleep_ns() in that case. But that function must be
called in every block job iteration, because otherwise it is for example
impossible to pause the job.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2015-04-28 15:36:11 +02:00
John Snow
59fc5d844f iotests: add incremental backup granularity tests
Test what happens if you fiddle with the granularity.

Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 1429314609-29776-22-git-send-email-jsnow@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2015-04-28 15:36:11 +02:00
John Snow
24618f5381 iotests: add incremental backup failure recovery test
Test the failure case for incremental backups.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 1429314609-29776-21-git-send-email-jsnow@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2015-04-28 15:36:11 +02:00
John Snow
a3d715958c iotests: add simple incremental backup case
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 1429314609-29776-20-git-send-email-jsnow@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2015-04-28 15:36:11 +02:00
John Snow
7898f74e78 iotests: add QMP event waiting queue
A filter is added to allow callers to request very specific
events to be pulled from the event queue, while leaving undesired
events still in the stream.

This allows us to poll for completion data for multiple asynchronous
events in any arbitrary order.

A new timeout context is added to the qmp pull_event method's
wait parameter to allow tests to fail if they do not complete
within some expected period of time.

Also fixed is a bug in qmp.pull_event where we try to retrieve an event
from an empty list if we attempt to retrieve an event with wait=False
but no events have occurred.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 1429314609-29776-19-git-send-email-jsnow@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2015-04-28 15:36:11 +02:00
John Snow
9f7264f57c iotests: add invalid input incremental backup tests
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 1429314609-29776-18-git-send-email-jsnow@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2015-04-28 15:36:11 +02:00
John Snow
a94e87c08c hbitmap: truncate tests
The general approach is to set bits close to the boundaries of
where we are truncating and ensure that everything appears to
have gone OK.

We test growing and shrinking by different amounts:
- Less than the granularity
- Less than the granularity, but across a boundary
- Less than sizeof(unsigned long)
- Less than sizeof(unsigned long), but across a ulong boundary
- More than sizeof(unsigned long)

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 1429314609-29776-17-git-send-email-jsnow@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2015-04-28 15:36:11 +02:00
John Snow
ce1ffea8cd block: Resize bitmaps on bdrv_truncate
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 1429314609-29776-16-git-send-email-jsnow@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2015-04-28 15:36:10 +02:00
John Snow
20dca81075 block: Ensure consistent bitmap function prototypes
We often don't need the BlockDriverState for functions
that operate on bitmaps. Remove it.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 1429314609-29776-15-git-send-email-jsnow@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2015-04-28 15:36:10 +02:00
John Snow
aa0c7ca506 block: add BdrvDirtyBitmap documentation
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 1429314609-29776-14-git-send-email-jsnow@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2015-04-28 15:36:10 +02:00
John Snow
a113534ffb qmp: Add dirty bitmap status field in query-block
Add the "frozen" status booleans, to inform clients
when a bitmap is occupied doing a task.

Signed-off-by: Fam Zheng <famz@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1429314609-29776-13-git-send-email-jsnow@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2015-04-28 15:36:10 +02:00
John Snow
e74e6b78e6 qmp: add block-dirty-bitmap-clear
Add bdrv_clear_dirty_bitmap and a matching QMP command,
qmp_block_dirty_bitmap_clear that enables a user to reset
the bitmap attached to a drive.

This allows us to reset a bitmap in the event of a full
drive backup.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1429314609-29776-12-git-send-email-jsnow@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2015-04-28 15:36:10 +02:00
John Snow
d58d845397 qmp: Add support of "dirty-bitmap" sync mode for drive-backup
For "dirty-bitmap" sync mode, the block job will iterate through the
given dirty bitmap to decide if a sector needs backup (backup all the
dirty clusters and skip clean ones), just as allocation conditions of
"top" sync mode.

Signed-off-by: Fam Zheng <famz@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 1429314609-29776-11-git-send-email-jsnow@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2015-04-28 15:36:10 +02:00
John Snow
9bd2b08f27 block: Add bitmap successors
A bitmap successor is an anonymous BdrvDirtyBitmap that is intended to
be created just prior to a sensitive operation (e.g. Incremental Backup)
that can either succeed or fail, but during the course of which we still
want a bitmap tracking writes.

On creating a successor, we "freeze" the parent bitmap which prevents
its deletion, enabling, anonymization, or creating a bitmap with the
same name.

On success, the parent bitmap can "abdicate" responsibility to the
successor, which will inherit its name. The successor will have been
tracking writes during the course of the backup operation. The parent
will be safely deleted.

On failure, we can "reclaim" the successor from the parent, unifying
them such that the resulting bitmap describes all writes occurring since
the last successful backup, for instance. Reclamation will thaw the
parent, but not explicitly re-enable it.

BdrvDirtyBitmap operations that target a single bitmap are protected
by assertions that the bitmap is not frozen and/or disabled.

BdrvDirtyBitmap operations that target a group of bitmaps, such as
bdrv_{set,reset}_dirty will ignore frozen/disabled drives with a
conditional instead.

Internal functions that enable/disable dirty bitmaps have assertions
added to them to prevent modifying frozen bitmaps.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1429314609-29776-10-git-send-email-jsnow@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2015-04-28 15:36:10 +02:00
John Snow
b8e6fb752e block: Add bitmap disabled status
Add a status indicating the enabled/disabled state of the bitmap.
A bitmap is by default enabled, but you can lock the bitmap into
a read-only state by setting disabled = true.

A previous version of this patch added a QMP interface for changing
the state of the bitmap, but it has since been removed for now until
a use case emerges where this state must be revealed to the user.

The disabled state WILL be used internally for bitmap migration and
bitmap persistence.

Signed-off-by: Fam Zheng <famz@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 1429314609-29776-9-git-send-email-jsnow@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2015-04-28 15:36:10 +02:00
John Snow
be58721dbf hbitmap: add hbitmap_merge
We add a bitmap merge operation to assist in error cases
where we wish to combine two bitmaps together.

This is algorithmically O(bits) provided HBITMAP_LEVELS remains
constant. For a full bitmap on a 64bit machine:
sum(bits/64^k, k, 0, HBITMAP_LEVELS) ~= 1.01587 * bits

We may be able to improve running speed for particularly sparse
bitmaps by using iterators, but the running time for dense maps
will be worse.

We present the simpler solution first, and we can refine it later
if needed.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 1429314609-29776-8-git-send-email-jsnow@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2015-04-28 15:36:10 +02:00
John Snow
8515efbef1 hbitmap: cache array lengths
As a convenience: between incremental backups, bitmap migrations
and bitmap persistence we seem to need to recalculate these a lot.

Because the lengths are a little bit-twiddly, let's just solidly
cache them and be done with it.

Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 1429314609-29776-7-git-send-email-jsnow@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2015-04-28 15:36:10 +02:00
John Snow
592fdd02ae block: Introduce bdrv_dirty_bitmap_granularity()
This returns the granularity (in bytes) of dirty bitmap,
which matches the QMP interface and the existing query
interface.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 1429314609-29776-6-git-send-email-jsnow@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2015-04-28 15:36:10 +02:00
John Snow
341ebc2f81 qmp: Add block-dirty-bitmap-add and block-dirty-bitmap-remove
The new command pair is added to manage a user created dirty bitmap. The
dirty bitmap's name is mandatory and must be unique for the same device,
but different devices can have bitmaps with the same names.

The granularity is an optional field. If it is not specified, we will
choose a default granularity based on the cluster size if available,
clamped to between 4K and 64K to mirror how the 'mirror' code was
already choosing granularity. If we do not have cluster size info
available, we choose 64K. This code has been factored out into a helper
shared with block/mirror.

This patch also introduces the 'block_dirty_bitmap_lookup' helper,
which takes a device name and a dirty bitmap name and validates the
lookup, returning NULL and setting errp if there is a problem with
either field. This helper will be re-used in future patches in this
series.

The types added to block-core.json will be re-used in future patches
in this series, see:
'qapi: Add transaction support to block-dirty-bitmap-{add, enable, disable}'

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1429314609-29776-5-git-send-email-jsnow@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2015-04-28 15:36:10 +02:00
John Snow
5fba6c0e50 qmp: Ensure consistent granularity type
We treat this field with a variety of different types everywhere
in the code. Now it's just uint32_t.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 1429314609-29776-4-git-send-email-jsnow@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2015-04-28 15:36:10 +02:00
Fam Zheng
0db6e54a8a qapi: Add optional field "name" to block dirty bitmap
This field will be set for user created dirty bitmap. Also pass in an
error pointer to bdrv_create_dirty_bitmap, so when a name is already
taken on this BDS, it can report an error message. This is not global
check, two BDSes can have dirty bitmap with a common name.

Implemented bdrv_find_dirty_bitmap to find a dirty bitmap by name, will
be used later when other QMP commands want to reference dirty bitmap by
name.

Add bdrv_dirty_bitmap_make_anon. This unsets the name of dirty bitmap.

Signed-off-by: Fam Zheng <famz@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-id: 1429314609-29776-3-git-send-email-jsnow@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2015-04-28 15:36:10 +02:00