Commit Graph

2115 Commits

Author SHA1 Message Date
Peter Maydell
79cecb3520 pc, pci, virtio: new features, cleanups, fixes
This includes some infrastructure for ipmi smbios tables.
 Beginning of acpi hotplug rework by Igor for supporting >255 CPUs.
 Misc cleanups and fixes.
 
 Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v1
 
 iQEcBAABAgAGBQJXVsQ6AAoJECgfDbjSjVRp/7MH/i39kIjyUY0jrt/UDrNgXLoi
 bi92ZdyyaeULg3rBFHlnweMs2VHChUATTc0DmXpr2hJbXm5MlZHQWHsv3UVpZ93h
 ZVY99b5AF/2Im1MIfDmxImFU9YfHYQuAqW7ZNx+RtXpzuAvdd89K29y80iwlJ251
 B1zhl9Tp8eePE3fZhbRElaTY70ruWonl+HOV9am7tfMDCTugYDPfLqdYT8fnaY98
 GMbSkmRnOaRYeo23dsg2pX7DK+H3I4DO8qvis6Va4pRiwCCf9L0N2GAIrljpMdbk
 yOCvvW3ujvCwkwwTwL1fPZTk1PTF3xhbgFZVvX2zOAWljhYkcesg4L8oERVBjwQ=
 =sqxd
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/mst/tags/for_upstream' into staging

pc, pci, virtio: new features, cleanups, fixes

This includes some infrastructure for ipmi smbios tables.
Beginning of acpi hotplug rework by Igor for supporting >255 CPUs.
Misc cleanups and fixes.

Signed-off-by: Michael S. Tsirkin <mst@redhat.com>

# gpg: Signature made Tue 07 Jun 2016 13:55:22 BST
# gpg:                using RSA key 0x281F0DB8D28D5469
# gpg: Good signature from "Michael S. Tsirkin <mst@kernel.org>"
# gpg:                 aka "Michael S. Tsirkin <mst@redhat.com>"

* remotes/mst/tags/for_upstream: (25 commits)
  virtio: move bi-endian target support to a single location
  pc-dimm: introduce realize callback
  pc-dimm: get memory region from ->get_memory_region()
  acpi: make bios_linker_loader_add_checksum() API offset based
  acpi: make bios_linker_loader_add_pointer() API offset based
  tpm: apci: cleanup TCPA table initialization
  acpi: cleanup bios_linker_loader_cleanup()
  acpi: simplify bios_linker API by removing redundant 'table' argument
  acpi: convert linker from GArray to BIOSLinker structure
  pc: use AcpiDeviceIfClass.send_event to issue GPE events
  acpi: extend ACPI interface to provide send_event hook
  pc: Postpone SMBIOS table installation to post machine init
  ipmi: rework the fwinfo to be fetched from the interface
  tests: acpi: update tables with consolidated legacy cpu-hotplug AML
  pc: acpi: cpuhp-legacy: switch ProcessorID to possible_cpus idx
  pc: acpi: simplify build_legacy_cpu_hotplug_aml() signature
  pc: acpi: consolidate legacy CPU hotplug in one file
  pc: acpi: mark current CPU hotplug functions as legacy
  pc: acpi: cpu-hotplug: make AML CPU_foo defines local to cpu_hotplug_acpi_table.c
  pc: acpi: consolidate \GPE._E02 with the rest of CPU hotplug AML
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2016-06-07 15:30:25 +01:00
Eduardo Habkost
d5aebef884 docker: Don't use eval trick on Makefile
The eval trick for defining DOCKER_SRC_COPY doesn't do anything
useful, as DOCKER_SRC_COPY is immediately expanded just after it
is defined, and CUR_TIME is already defined using ":=". Simply
define it using ":=" so it is evaluated only once.

The eval trick was also triggering an weird error on Travis builds:
  qemu/tests/docker/Makefile.include:34: *** unterminated variable reference.  Stop.

The issue is not easily reproducible (maybe it's a bug in some
versions of Make), but it is avoided if removing the eval trick.

Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2016-06-07 15:00:02 +01:00
Igor Mammedov
53c400a6ac tests: acpi: update tables with consolidated legacy cpu-hotplug AML
Signed-off-by: Igor Mammedov <imammedo@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
2016-06-07 15:36:54 +03:00
Igor Mammedov
d19587db9e tests: acpi: report names of expected files in verbose mode
print expected file name if it doesn't exists if
verbose mode is enabled*. It helps to avoid running
bios-tables-test under debugger to figure out missing
file name.

*)
verbose mode is enabled if "V" env. variable is set

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Reviewed-by: Marcel Apfelbaum <marcel@redhat.com>
Reviewed-by: Marcel Apfelbaum <marcel@redhat.com>
2016-06-07 15:36:54 +03:00
Peter Maydell
2c107d7684 -----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
 
 iQEcBAABAgAGBQJXT9DWAAoJEO8Ells5jWIRgFAH/1ZDXm8V523AMDOEvBAWgqur
 Dj8ZaIwFkqJp7xtLdhS0yKF3xW+vtgx9k+Qftk0S8qEiFKPbThR8iB5VNuesErwd
 AZhWo4bnVhKwtWyMw3BDRDK1N4huAWPMZEva1xovR/Cc9v5IG5mx57/K3Zz5C8ec
 Jsn4DsLKN0q7W0D0dlnbEOkSjl6iKJchvfPCR6UfvrU7BxfXaCZ9Z7Sfh8ec6tfr
 iMgcV9u3A3Zs72gTM9/jdKx8vOrWtdKJufJ8s2Bctc7CyfBNWwnV8PjndhEe3Xvs
 vlYeJopdpDPsdMkMtYD6cevtEgvD5yhOBndJ7et807jjuCvUf837tMhodKkFk9M=
 =SjIZ
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/jasowang/tags/net-pull-request' into staging

# gpg: Signature made Thu 02 Jun 2016 07:23:18 BST using RSA key ID 398D6211
# gpg: Good signature from "Jason Wang (Jason Wang on RedHat) <jasowang@redhat.com>"
# gpg: WARNING: This key is not certified with sufficiently trusted signatures!
# gpg:          It is not certain that the signature belongs to the owner.
# Primary key fingerprint: 215D 46F4 8246 689E C77F  3562 EF04 965B 398D 6211

* remotes/jasowang/tags/net-pull-request: (31 commits)
  Add ENET device to i.MX6 SOC.
  Add ENET/Gbps Ethernet support to FEC device
  i.MX: move FEC device to a register array structure.
  i.MX: Rename i.MX FEC defines to ENET_XXX
  i.MX: reset TX/RX descriptors when FEC is disabled.
  i.MX: Fix FEC code for ECR register reset value.
  i.MX: Fix FEC code for MDIO address selection
  i.MX: Fix FEC code for MDIO operation selection
  net: handle optional VLAN header in checksum computation.
  net: improve UDP/TCP checksum computation.
  e1000e: Introduce qtest for e1000e device
  net: Introduce e1000e device emulation
  e1000: Move out code that will be reused in e1000e
  e1000_regs: Add definitions for Intel 82574-specific bits
  vmxnet3: Use pci_dma_* API instead of cpu_physical_memory_*
  net_pkt: Extend packet abstraction as required by e1000e functionality
  rtl8139: Move more TCP definitions to common header
  net_pkt: Name vmxnet3 packet abstractions more generic
  vmxnet3: Use common MAC address tracing macros
  net: Add macros for MAC address tracing
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2016-06-02 14:26:57 +01:00
Dmitry Fleytman
7c375e2294 e1000e: Introduce qtest for e1000e device
Signed-off-by: Dmitry Fleytman <dmitry.fleytman@ravellosystems.com>
Signed-off-by: Leonid Bloch <leonid.bloch@ravellosystems.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Jason Wang <jasowang@redhat.com>
2016-06-02 10:42:30 +08:00
Dmitry Fleytman
605d52e62f net_pkt: Name vmxnet3 packet abstractions more generic
This patch drops "vmx" prefix from packet abstractions names
to emphasize the fact they are generic and not tied to any
specific network device.

These abstractions will be reused by e1000e emulation implementation
introduced by following patches so their names need generalization.

This patch (except renamed files, adjusted comments and changes in MAINTAINTERS)
was produced by:

git grep -lz 'vmxnet_tx_pkt' | xargs -0 perl -i'' -pE "s/vmxnet_tx_pkt/net_tx_pkt/g"
git grep -lz 'vmxnet_rx_pkt' | xargs -0 perl -i'' -pE "s/vmxnet_rx_pkt/net_rx_pkt/g"
git grep -lz 'VmxnetTxPkt' | xargs -0 perl -i'' -pE "s/VmxnetTxPkt/NetTxPkt/g"
git grep -lz 'VMXNET_TX_PKT' | xargs -0 perl -i'' -pE "s/VMXNET_TX_PKT/NET_TX_PKT/g"
git grep -lz 'VmxnetRxPkt' | xargs -0 perl -i'' -pE "s/VmxnetRxPkt/NetRxPkt/g"
git grep -lz 'VMXNET_RX_PKT' | xargs -0 perl -i'' -pE "s/VMXNET_RX_PKT/NET_RX_PKT/g"
sed -ie 's/VMXNET_/NET_/g' hw/net/vmxnet_rx_pkt.c
sed -ie 's/VMXNET_/NET_/g' hw/net/vmxnet_tx_pkt.c

Signed-off-by: Dmitry Fleytman <dmitry.fleytman@ravellosystems.com>
Signed-off-by: Leonid Bloch <leonid.bloch@ravellosystems.com>
Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Jason Wang <jasowang@redhat.com>
2016-06-02 10:42:27 +08:00
Fam Zheng
35e0f959b5 docker: Add EXTRA_CONFIGURE_OPTS
Whatever passed in this variable will be appended to all
configure commands.

Signed-off-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-id: 1464755128-32490-15-git-send-email-famz@redhat.com
2016-06-01 17:27:35 +08:00
Fam Zheng
d5bd789198 docker: Add travis tool
The script is not prefixed with test- so it won't run with "make docker-test",
because it can take too long.

Run it with "make docker-travis@ubuntu".

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Fam Zheng <famz@redhat.com>
Message-id: 1464755128-32490-13-git-send-email-famz@redhat.com
2016-06-01 17:27:35 +08:00
Fam Zheng
c4f0eed1f3 docker: Add mingw test
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Fam Zheng <famz@redhat.com>
Message-id: 1464755128-32490-12-git-send-email-famz@redhat.com
2016-06-01 17:27:35 +08:00
Fam Zheng
c8908570dc docker: Add clang test
The (currently partially commented out) configure options are suggested
by John Snow <jsnow@redhat.com>.

Signed-off-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-id: 1464755128-32490-11-git-send-email-famz@redhat.com
2016-06-01 17:27:35 +08:00
Fam Zheng
d710ac871c docker: Add full test
This builds all available targets.

Signed-off-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-id: 1464755128-32490-10-git-send-email-famz@redhat.com
2016-06-01 17:27:35 +08:00
Fam Zheng
b7899d63c8 docker: Add quick test
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Fam Zheng <famz@redhat.com>
Message-id: 1464755128-32490-9-git-send-email-famz@redhat.com
2016-06-01 17:27:35 +08:00
Fam Zheng
3568f98ca5 docker: Add common.rc
"requires" checks the "FEATURE" environment for specified prerequisits,
and skip the execution of test if not found.

"build_qemu" is the central routine to compile QEMU for tests to call.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Fam Zheng <famz@redhat.com>
Message-id: 1464755128-32490-8-git-send-email-famz@redhat.com
2016-06-01 17:27:35 +08:00
Fam Zheng
b344aa9132 docker: Add test runner
It's better to have a launcher for all tests, to make it easier to
initialize and manage the environment.

If "DEBUG=1"  a shell prompt will show up before the test runs.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Fam Zheng <famz@redhat.com>
Message-id: 1464755128-32490-7-git-send-email-famz@redhat.com
2016-06-01 17:27:35 +08:00
Fam Zheng
ca853f0c76 docker: Add images
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Fam Zheng <famz@redhat.com>
Message-id: 1464755128-32490-6-git-send-email-famz@redhat.com
2016-06-01 17:27:35 +08:00
Fam Zheng
324027c24c Makefile: Rules for docker testing
This adds a group of make targets to run docker tests, all are available
in source tree without running ./configure.

The usage is shown with "make docker".

Besides the fixed ones, dynamic targets for building each image and
running each test in each image are generated automatically by make,
scanning $(SRC_PATH)/tests/docker/ files with specific patterns.

Alternative to manually list particular targets (docker-TEST@IMAGE)
set, you can control which tests/images to run by filtering variables,
TESTS= and IMAGES=, which are expressed in Makefile pattern syntax,
"foo% %bar ...". For example:

    $ make docker-test IMAGES="ubuntu fedora"

Unfortunately, it's impossible to propagate "-j $JOBS" into make in
containers, however since each combination is made a first class target
in the top Makefile, "make -j$N docker-test" still parallels the tests
coarsely.

Still, $J is made a magic variable to let all make invocations in
containers to use -j$J.

Instead of providing a live version of the source tree to the docker
container we snapshot it with git-archive. This ensures the tree is in a
pristine state for whatever operations the container is going to run on
them.

Uncommitted changes known to files known by the git index will be
included in the snapshot if there are any.

Signed-off-by: Fam Zheng <famz@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-id: 1464755128-32490-5-git-send-email-famz@redhat.com
2016-06-01 17:27:34 +08:00
Fam Zheng
4485b04be9 tests: Add utilities for docker testing
docker.py is added with a number of useful subcommands to manager docker
images and instances for QEMU docker testing. Subcommands are:

run: A wrapper of "docker run" (or "sudo -n docker run" if necessary),
which takes care of killing and removing the running container at
SIGINT.

clean: Tear down all the containers including inactive ones that are
started by docker_run.

build: Compare an image from given dockerfile and rebuild it if they're
different.

Signed-off-by: Fam Zheng <famz@redhat.com>
Message-id: 1464755128-32490-2-git-send-email-famz@redhat.com
2016-06-01 17:25:50 +08:00
Peter Maydell
aef11b8d33 migration: add TLS support to the migration data channel
This is a big refactoring of the migration backend code - moving away from
 QEMUFile to the new QIOChannel framework introduced here.  This brings a
 good level of abstraction and reduction of many lines of code.
 
 This series also adds the ability for many backends (all except RDMA) to
 use TLS for encrypting the migration data between the endpoints.
 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v2
 
 iQIcBAABCAAGBQJXRpKMAAoJEOsLTfxlfvZw2FMQAJmrp8ijvJNtdBa51bNY+xVx
 zvrHDpWco/HxxqyBIGxG7g8Iq+wpNsdgoRxoQkfgIz9RkZiNrzb1kGRiqNFFBKFX
 ziK1QQQ12ETUXwQ6VguBuwLDvCCenyUti0HfKkceG+Zu5263fyp+VzL+PuEtteT3
 M0pZRrifj/TQqCBXR8yhBAo2dCiFETLVoruE+iNg2ipI3JDizxy8bdOU2gfnTayf
 na7lE53pI+Wy8KE+qrhtsEgjHFp48uJ0HwQIIumvVndXFpIhRzCcN/aeVCjNYRjo
 GeI18OJxcimRDwsnfuOwuZKhRcjWfa8WEIKsi8LdRTZFpFL6y9R57XNTBIFfbjOF
 0lkmFTqJTBi3OTPjj0hMjpjOfXhyKUnwdqCAYlAxeuWHhqPDDhtEcnNtGdmQzx4Z
 KvYzc3t31o1gPin024UUfA528PNREszaXhTM90/Dj0dhVSMoG1VQsQjxzkPXxdM1
 wemfic+77Bk4oUrSplhdvvk4nySDWeseEjfdyVU2ixqldy8Ib1+6H+PCjWNotpQ0
 YiDOHBy3rrUh6NhIqb0C2PWvd/9Aqs0nHQHJ8QKYK574MDbVo8mKTACFdoSYoZ1u
 wuif7NL6qkyS55szf0dm8zPBCJ5nIR5SQE98E7+ptXNa8AipfFsTkZrr3aOjcdey
 98AWF9KaZOWRfwgIm3Ft
 =AYEK
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/amit-migration/tags/migration-2.7-2' into staging

migration: add TLS support to the migration data channel

This is a big refactoring of the migration backend code - moving away from
QEMUFile to the new QIOChannel framework introduced here.  This brings a
good level of abstraction and reduction of many lines of code.

This series also adds the ability for many backends (all except RDMA) to
use TLS for encrypting the migration data between the endpoints.

# gpg: Signature made Thu 26 May 2016 07:07:08 BST using RSA key ID 657EF670
# gpg: Good signature from "Amit Shah <amit@amitshah.net>"
# gpg:                 aka "Amit Shah <amit@kernel.org>"
# gpg:                 aka "Amit Shah <amitshah@gmx.net>"

* remotes/amit-migration/tags/migration-2.7-2: (28 commits)
  migration: remove qemu_get_fd method from QEMUFile
  migration: remove support for non-iovec based write handlers
  migration: add support for encrypting data with TLS
  migration: define 'tls-creds' and 'tls-hostname' migration parameters
  migration: don't use an array for storing migrate parameters
  migration: move definition of struct QEMUFile back into qemu-file.c
  migration: delete QEMUFile stdio implementation
  migration: delete QEMUFile sockets implementation
  migration: delete QEMUSizedBuffer struct
  migration: delete QEMUFile buffer implementation
  migration: convert savevm to use QIOChannel for writing to files
  migration: convert RDMA to use QIOChannel interface
  migration: convert exec socket protocol to use QIOChannel
  migration: convert fd socket protocol to use QIOChannel
  migration: convert tcp socket protocol to use QIOChannel
  migration: rename unix.c to socket.c
  migration: convert unix socket protocol to use QIOChannel
  migration: convert post-copy to use QIOChannelBuffer
  migration: add reporting of errors for outgoing migration
  migration: add helpers for creating QEMUFile from a QIOChannel
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2016-05-26 16:09:27 +01:00
Peter Maydell
2c56d06baf Block layer patches
-----BEGIN PGP SIGNATURE-----
 Version: GnuPG v2.0.22 (GNU/Linux)
 
 iQIcBAABAgAGBQJXReG4AAoJEH8JsnLIjy/WKv0P/0Bevm3ahvJuStRdl+u0MgZ7
 9jM69CN3wxH8SVwCJ3zBndWVOrsAbXB+SFTu/0Dr+tq+dw901AN8lspzf1TFiYnL
 5M8B1syjOXUxuOcwl18ytuDu9aQhxbbCIC9pzWqxizm8N2nXP9r/VlNiztm4T2nn
 RV0DyD5XVoV2v3rpFPerv1EgOtO0lmfgSABwk31a5ftULdy1QzDOliLm/l7W7Oke
 xMFJ93OOBd/p0jmLrULM3V/tipRpxcnvijfVoLJb3895/0G7mJ1PCRYl3uokd+cy
 DxmWw9Fil/kfpwmQhpvpWeOeSkh/NXVNDH5fLfl7V1dPyYcs2xZOJz3mNtuMDU2N
 kBxkWtOQivr/q9cIK6sEacDRtTjouRi/DoccR6S8IXs9nY1fJoLBvXufo2mQrwsy
 5heUhrvZ+AD/eZVjQXQyIc9jCLltU12lW1Z+KrVO7HICcT/Z+UGqeROrRrhzjRRl
 iuO2iHXI7I4KSB2Xgrx5WMYZu8GbxHJmfe20O2bupJ5PHRHLF0ySZ0MW3Gbs1W5E
 5e5tU4D4x5JYJ1KYsMg2FsuejqMBh25coqywT7I3XOEOnN8opOPVKoHb86gZOjXD
 wckrSGHkX9rfvRgy+vG/d3RqQOvxtaH1kbWCHikVjHpnVPzz6uPPD1H8/T5s2IGB
 49PdfL0SIOQsl9U/3WPk
 =g40O
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches

# gpg: Signature made Wed 25 May 2016 18:32:40 BST using RSA key ID C88F2FD6
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>"

* remotes/kevin/tags/for-upstream: (31 commits)
  blockjob: Remove BlockJob.bs
  commit: Use BlockBackend for I/O
  backup: Use BlockBackend for I/O
  backup: Remove bs parameter from backup_do_cow()
  backup: Pack Notifier within BackupBlockJob
  backup: Don't leak BackupBlockJob in error path
  mirror: Use BlockBackend for I/O
  mirror: Allow target that already has a BlockBackend
  stream: Use BlockBackend for I/O
  block: Make blk_co_preadv/pwritev() public
  block: Convert block job core to BlockBackend
  block: Default to enabled write cache in blk_new()
  block: Cancel jobs first in bdrv_close_all()
  block: keep a list of block jobs
  block: Rename blk_write_zeroes()
  dma-helpers: change BlockBackend to opaque value in DMAIOFunc
  dma-helpers: change interface to byte-based
  block: Propagate .drained_begin/end callbacks
  block: Fix reconfiguring graph with drained nodes
  block: Make bdrv_drain() use bdrv_drained_begin/end()
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2016-05-26 14:29:30 +01:00
Andreas Färber
a62c89117f qdev: Start disentangling bus from device
Move bus type and related APIs to a separate file bus.c.
This is a first step in breaking up qdev.c into more manageable chunks.

Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
[AF: Rebased onto osdep.h]
Signed-off-by: Andreas Färber <afaerber@suse.de>
[PMM: added bus.o to link line for test-qdev-global-props]
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2016-05-26 14:06:41 +01:00
Daniel P. Berrange
8925839f00 migration: convert savevm to use QIOChannel for writing to files
Convert the exec savevm code to use QIOChannel and QEMUFileChannel,
instead of the stdio APIs.

Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Message-Id: <1461751518-12128-19-git-send-email-berrange@redhat.com>
Signed-off-by: Amit Shah <amit.shah@redhat.com>
2016-05-26 11:31:53 +05:30
Daniel P. Berrange
a8ec4437cd migration: remove use of qemu_bufopen from vmstate tests
Some of the test-vmstate.c test cases use a temporary file
while others use a memory buffer. To facilitate the future
removal of the qemu_bufopen() function, convert all the tests
to use a temporary file.

Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Message-Id: <1461751518-12128-4-git-send-email-berrange@redhat.com>
Signed-off-by: Amit Shah <amit.shah@redhat.com>
2016-05-26 11:31:11 +05:30
Kevin Wolf
b75536c9fa blockjob: Remove BlockJob.bs
There is a single remaining user in qemu-img, and another one in a test
case, both of which can be trivially converted to using BlockJob.blk
instead.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2016-05-25 19:04:21 +02:00
Kevin Wolf
b880481579 mirror: Allow target that already has a BlockBackend
We had to forbid mirroring to a target BDS that already had a BB
attached because the node swapping at job completion would add a second
BB and we didn't support multiple BBs on a single BDS at the time. Now
we do, so we can lift the restriction.

As we allow additional BlockBackends for the target, we must expect
other users to be sending requests. There may no requests be in flight
during the graph modification, so we have to drain those users now.

The core part of this patch is a revert of commit 40365552.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
2016-05-25 19:04:21 +02:00
Max Reitz
109525ad6a block: Drop errp parameter from blk_new()
blk_new() cannot fail so its Error ** parameter has become superfluous.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2016-05-25 19:04:10 +02:00
Max Reitz
21a699afc8 tests: Drop BDS from test-throttle.c
Now that throttling has been moved to the BlockBackend level, we do not
need to create a BDS along with the BB in the I/O throttling test.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2016-05-25 19:04:10 +02:00
Markus Armbruster
17b74b9867 migration: Move qjson.[ch] to migration/
Type QJSON lets you build JSON text.  Its interface mirrors (a subset
of) abstract JSON syntax.

QAPI output visitors also produce JSON text.  They assert their
preconditions and invariants, and therefore abort on incorrect use.

Contrastingly, QJSON does *not* detect incorrect use.  It happily
produces invalid JSON then.  This is what migration wants.

QJSON was designed for migration, and migration is its only user.
Move it to migration/ for proper coverage by MAINTAINERS, and to deter
accidental use outside migration.

[Pointed out by Eric: QJSON was added in commits 0457d07..b174257
 -- Amit]

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <1462380558-2030-2-git-send-email-armbru@redhat.com>
Signed-off-by: Amit Shah <amit.shah@redhat.com>
2016-05-23 14:16:09 +05:30
Peter Maydell
6bd8ab6889 Block layer patches
-----BEGIN PGP SIGNATURE-----
 Version: GnuPG v2.0.22 (GNU/Linux)
 
 iQIcBAABAgAGBQJXPdcnAAoJEH8JsnLIjy/WPEoQAK5vlRYqvQrrevMJviT4ZPUX
 cGGbabOcmfTBHGAgGwRLg+vQ043Sgu14JjtNbrsoSsBwAl9eAhAVGOimiieaY3vR
 35OOUxECswArJzK8I4XRx4KhI871Yq+8kHILPoXpF8L7YU38Zqa1D5z2dcOKYrL8
 Oy5IEfd1+Qfpxg/txKIioP5BzKVpz3V9/8GRNo0iAl7c806NoYFpnM0TXsed9Fjr
 YvUn1AdGHUF0/pV6vU46Qxz4yy1Q+cuoh923z6+YvXTcwok7PbjhAQWWA0qvSTuG
 otnPKMPBhYa6g7XOPD9Mra986vs6vBEGiPS5uqXoM5FqxF4Hc9LIeHEr+3hb+m53
 NLOmGqfct0USY9r6rXsOhZQb7nZCDuhaedv33ZfgE0T0cYxIilHs5PhgFAWfthhP
 aNJYlzbJUhqhTi7CJrJcFoGbNQDxux5qtlFo43M4vz/WYYDrwu8P7O3YO+sH0jU1
 EXJnbtztQvwfsiIEbIzvBRQl3XD9QmCfYO3lRbOwdCnd3ZLy47E2bze4gV3DwzK7
 CsBr+sa49xI8LMswPxTms+A+Inndn8O0mGI32Zi4nBKapjpy5Fb4YG6z8+WPfTKp
 Il1PsSgG84wm4YxGWty/UI4DoPY+hqlIIz1CNuRRNQtZTybLgNCK8ZKYbVlRppmf
 pGPpQ8pmqkeFLmx8hecm
 =ntKz
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches

# gpg: Signature made Thu 19 May 2016 16:09:27 BST using RSA key ID C88F2FD6
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>"

* remotes/kevin/tags/for-upstream: (31 commits)
  qemu-iotests: Fix regression in 136 on aio_read invalid
  qemu-iotests: Simplify 109 with unaligned qemu-img compare
  qemu-io: Fix recent UI updates
  block: clarify error message for qmp-eject
  qemu-iotests: Some more write_zeroes tests
  qcow2: Fix write_zeroes with partially allocated backing file cluster
  qcow2: fix condition in is_zero_cluster
  block: Propagate AioContext change to all children
  block: Remove BlockDriverState.blk
  block: Don't return throttling info in query-named-block-nodes
  block: Avoid bs->blk in bdrv_next()
  block: Add bdrv_has_blk()
  block: Remove bdrv_aio_multiwrite()
  blockjob: Don't touch BDS iostatus
  blockjob: Don't set iostatus of target
  block: User BdrvChild callback for device name
  block: Use BdrvChild callbacks for change_media/resize
  block: Don't check throttled reqs in bdrv_requests_pending()
  Revert "block: Forbid I/O throttling on nodes with multiple parents for 2.6"
  block: Remove bdrv_move_feature_fields()
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2016-05-19 16:54:12 +01:00
Eric Blake
37546ff28f qemu-iotests: Fix regression in 136 on aio_read invalid
Commit 093ea232 removed the ability for aio_read and aio_write
to artificially inflate the invalid statistics counters for
block devices, since it no longer flags unaligned offset or
length.  Add 'aio_read -i' and 'aio_write -i' to restore
the ability, and update test 136 to use it.

Reported-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-id: 1463416983-28318-4-git-send-email-eblake@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2016-05-19 16:56:58 +02:00
Eric Blake
9e28bb26c2 qemu-iotests: Simplify 109 with unaligned qemu-img compare
For some time now, qemu-img compare has been able to compare
unaligned images.  So we no longer need test 109's hack of
resizing to sector boundaries before invoking compare.

Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 1463416983-28318-3-git-send-email-eblake@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2016-05-19 16:56:58 +02:00
Kevin Wolf
1ef7d01021 qemu-iotests: Some more write_zeroes tests
This covers some more write_zeroes cases which are relevant for the
recent qcow2 optimisations that check the allocation status of the
backing file for partial cluster write_zeroes requests.

This needs to be separate from 034 because we can only support qcow2 in
this test case for multiple reasons: We check the allocation status
after write_zeroes with 'qemu-img map' and the optimised behaviour that
produces zero clusters is only implemented in qcow2; second, the map
command returns offsets that are qcow2 specific; and finally, we also
use 512 byte clusters which aren't supported for formats like qed.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2016-05-19 16:45:31 +02:00
Kevin Wolf
79c719b755 block: Don't return throttling info in query-named-block-nodes
query-named-block-nodes should not return information that is related
to the attached BlockBackend rather than the node itself, so throttling
information needs to be removed from it.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
2016-05-19 16:45:31 +02:00
Kevin Wolf
91c6e4b7bb block: Remove bdrv_aio_multiwrite()
Since virtio-blk implements request merging itself these days, the only
remaining users are test cases for the function. That doesn't make the
function exactly useful any more.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2016-05-19 16:45:31 +02:00
Kevin Wolf
97148076e8 block: Move I/O throttling configuration functions to BlockBackend
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
2016-05-19 16:45:30 +02:00
Kevin Wolf
27ccdd5259 block: Move throttling fields from BDS to BB
This patch changes where the throttling state is stored (used to be the
BlockDriverState, now it is the BlockBackend), but it doesn't actually
make it a BB level feature yet. For example, throttling is still
disabled when the BDS is detached from the BB.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
2016-05-19 16:45:30 +02:00
Kevin Wolf
49d2165d7d block: Convert throttle_group_get_name() to BlockBackend
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
2016-05-19 16:45:29 +02:00
Kevin Wolf
31dce3ccca block: throttle-groups: Use BlockBackend pointers internally
As a first step towards moving I/O throttling to the BlockBackend level,
this patch changes all pointers in struct ThrottleGroup from referencing
a BlockDriverState to referencing a BlockBackend.

This change is valid because we made sure that throttling can only be
enabled on BDSes which have a BB attached.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
2016-05-19 16:45:29 +02:00
Kevin Wolf
a5614993d7 block: Make sure throttled BDSes always have a BB
It was already true in principle that a throttled BDS always has a BB
attached, except that the order of operations while attaching or
detaching a BDS to/from a BB wasn't careful enough.

This commit breaks graph manipulations while I/O throttling is enabled.
It would have been possible to keep things working with some temporary
hacks, but quite cumbersome, so it's not worth the hassle. We'll fix
things again in a minute.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
2016-05-19 16:45:29 +02:00
Paolo Bonzini
87776ab72b qemu-common: stop including qemu/host-utils.h from qemu-common.h
Move it to the actual users.  There are some inclusions of
qemu/host-utils.h in headers, but they are all necessary.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2016-05-19 16:42:28 +02:00
Paolo Bonzini
58369e22cf qemu-common: stop including qemu/bswap.h from qemu-common.h
Move it to the actual users.  There are still a few includes of
qemu/bswap.h in headers; removing them is left for future work.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2016-05-19 16:42:28 +02:00
Stefan Weil
cb8d4c8f54 Fix some typos found by codespell
Signed-off-by: Stefan Weil <sw@weilnetz.de>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>
2016-05-18 15:04:27 +03:00
Peter Maydell
f68419eee9 Block layer patches
-----BEGIN PGP SIGNATURE-----
 Version: GnuPG v2.0.22 (GNU/Linux)
 
 iQIcBAABAgAGBQJXNIcBAAoJEH8JsnLIjy/WiUQP/Rzfo8pe7TWA2InxdcDOPsx4
 2/tHHJdVkffnNX5rdBvc0mOUZNJxej0NtJu2e63BB+ydYju//xw8gruKU7TR+Nd3
 nPNSsqk80prK3RNgWu7qymBvIkHDDcDQhlp48HKq+dxrfConXtHmoXapGsqc0S47
 xu03oC6WzSIyLf7TLytcjUmEprQSaCGOwsb/XaHAWL750fFAGcdy/K5PWBpUv6DN
 T0jZ3u4UneE1jeabRmqAwjgDJXC9l6riH9fP/ZtYhgNlNj84zlMXajUHSULhGknP
 cTGjwwg9tOvhcjTdhdRmWlvG1m0T77ZX3icfZLhcTdb/Uz68NXVqs8P25IGV9McD
 DPrb3T/M8JUoqLXJxIpxUm2Levof5v0dUF1PHmN5bT7pshcqv/1J7v8Fdtf9l9mp
 zI0+FK1TZ102C0H2F7AWYZSlo2EfNUSd02QQx6MbfDokDIlIxY+EgP1/Es5XlkqC
 wc7HrJvq+uix2zXw9bn9Vg9p/nDuxlRx+ppRRarNNRonaqTrx/1qAaas4bsqc9Gz
 H6gxw7BHybm0TZFdHqAdIonpesecYw6yWUXT/mQehbfphsmQmu/d2HvF2C9uUm4X
 O0JduBlKOTm2hMcg5qL6Gko8WaQIctdCJH/1Onts92cZnm8Vr/9zcmMgwGoCd7sE
 +t6Yg0jqpTUJwhZhIuCw
 =NbjJ
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches

# gpg: Signature made Thu 12 May 2016 14:37:05 BST using RSA key ID C88F2FD6
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>"

* remotes/kevin/tags/for-upstream: (69 commits)
  qemu-iotests: iotests: fail hard if not run via "check"
  block: enable testing of LUKS driver with block I/O tests
  block: add support for encryption secrets in block I/O tests
  block: add support for --image-opts in block I/O tests
  qemu-io: Add 'write -z -u' to test MAY_UNMAP flag
  qemu-io: Add 'write -f' to test FUA flag
  qemu-io: Allow unaligned access by default
  qemu-io: Use bool for command line flags
  qemu-io: Make 'open' subcommand more like command line
  qemu-io: Add missing option documentation
  qmp: add monitor command to add/remove a child
  quorum: implement bdrv_add_child() and bdrv_del_child()
  Add new block driver interface to add/delete a BDS's child
  qemu-img: check block status of backing file when converting.
  iotests: fix the redirection order in 083
  block: Inactivate all children
  block: Drop superfluous invalidating bs->file from drivers
  block: Invalidate all children
  nbd: Simplify client FUA handling
  block: Honor BDRV_REQ_FUA during write_zeroes
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2016-05-12 16:33:40 +01:00
Sascha Silbe
5a8fabf333 qemu-iotests: iotests: fail hard if not run via "check"
Running an iotests-based Python test directly might appear to work,
but may fail in subtle ways and is insecure:

- It creates files with predictable file names in a world-writable
  location (/var/tmp).

- Tests expect the environment to be set up by check. E.g. 041 and 055
  may take the wrong code paths if QEMU_DEFAULT_MACHINE is not
  set. This can lead to false negatives.

Instead fail hard and tell the user we want to be run via "check".

The actual environment expected by the tests is currently only defined
by the implementation of "check". We use two of the environment
variables set by "check" as indication of whether we're being run via
"check". Anyone writing their own test runner (replacing "check") will
need to replicate the full environment (in a broader sense, not just
environment variables) provided by "check" anyway, including setting
the two environment variables we check. Whereas a regular developer
just trying to invoke the tests usually won't have both of these
defined in their environment so we can catch their mistake and give
out useful advice.

Signed-off-by: Sascha Silbe <silbe@linux.vnet.ibm.com>
Reviewed-by: Bo Tu <tubo@linux.vnet.ibm.com>
Message-id: 1461094442-16014-1-git-send-email-silbe@linux.vnet.ibm.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2016-05-12 15:33:24 +02:00
Daniel P. Berrange
4e9b25fb05 block: enable testing of LUKS driver with block I/O tests
This adds support for testing the LUKS driver with the block
I/O test framework.

   cd tests/qemu-io-tests
   ./check -luks

A handful of test cases are modified to work with luks

 - 004 - whitelist luks format
 - 012 - use TEST_IMG_FILE instead of TEST_IMG for file ops
 - 048 - use TEST_IMG_FILE instead of TEST_IMG for file ops.
         don't assume extended image contents is all zeros,
         explicitly initialize with zeros
         Make file size smaller to avoid having to decrypt
         1 GB of data.
 - 052 - don't assume initial image contents is all zeros,
         explicitly initialize with zeros
 - 100 - don't assume initial image contents is all zeros,
         explicitly initialize with zeros

With this patch applied, the results are as follows:

  Passed: 001 002 003 004 005 008 009 010 011 012 021 032 043
          047 048 049 052 087 100 134 143
  Failed: 033 120 140 145
 Skipped: 007 013 014 015 017 018 019 020 022 023 024 025 026
          027 028 029 030 031 034 035 036 037 038 039 040 041
          042 043 044 045 046 047 049 050 051 053 054 055 056
          057 058 059 060 061 062 063 064 065 066 067 068 069
          070 071 072 073 074 075 076 077 078 079 080 081 082
          083 084 085 086 087 088 089 090 091 092 093 094 095
          096 097 098 099 101 102 103 104 105 107 108 109 110
          111 112 113 114 115 116 117 118 119 121 122 123 124
          128 129 130 131 132 133 134 135 136 137 138 139 141
          142 144 146 148 150 152

The reasons for the failed tests are:

 - 033 - needs adapting to use image opts syntax with blkdebug
         and test image in order to correctly set align property
 - 120 - needs adapting to use correct -drive syntax for luks
 - 140 - needs adapting to use correct -drive syntax for luks
 - 145 - needs adapting to use correct -drive syntax for luks

The vast majority of skipped tests are exercising code that is
qcow2 specific, though a couple could probably be usefully
enabled for luks too.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Message-id: 1462896689-18450-4-git-send-email-berrange@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2016-05-12 15:33:24 +02:00
Daniel P. Berrange
b7e875b2f9 block: add support for encryption secrets in block I/O tests
The LUKS block driver tests will require the ability to specify
encryption secrets with block devices. This requires using the
--object argument to qemu-img/qemu-io to create a 'secret'
object.

When the IMGKEYSECRET env variable is set, it provides the
password to be associated with a secret called 'keysec0'

The _qemu_img_wrapper function isn't modified as that needs
to cope with differing syntax for subcommands, so can't be
made to use the image opts syntax unconditionally.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Message-id: 1462896689-18450-3-git-send-email-berrange@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2016-05-12 15:33:24 +02:00
Daniel P. Berrange
076003f526 block: add support for --image-opts in block I/O tests
Currently all block tests use the traditional syntax for images
just specifying a filename. To support the LUKS driver without
resorting to JSON, the tests need to be able to use the new
--image-opts argument to qemu-img and qemu-io.

This introduces a new env variable IMGOPTSSYNTAX. If this is
set to 'true', then qemu-img/qemu-io should use --image-opts.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Message-id: 1462896689-18450-2-git-send-email-berrange@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
2016-05-12 15:33:24 +02:00
Eric Blake
093ea232b0 qemu-io: Allow unaligned access by default
There's no reason to require the user to specify a flag just so
they can pass in unaligned numbers.  Keep 'read -p' and 'write -p'
as no-ops so that I don't have to hunt down and update all users
of qemu-io, but otherwise make their behavior default as 'read' and
'write'.  Also fix 'write -z', 'readv', 'writev', 'writev',
'aio_read', 'aio_write', and 'aio_write -z'.  For now, 'read -b',
'write -b', and 'write -c' still require alignment (and 'multiwrite',
but that's slated to die soon).

qemu-iotest 23 is updated to match, as the only test that was
previously explicitly expecting an error on an unaligned request.

Signed-off-by: Eric Blake <eblake@redhat.com>
Message-id: 1462677405-4752-5-git-send-email-eblake@redhat.com
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2016-05-12 15:33:24 +02:00
Wei Jiangang
9036e87c74 iotests: fix the redirection order in 083
It should redirect stdout to /dev/null first,
then redirect stderr to whatever stdout currently points at.

Signed-off-by: Wei Jiangang <weijg.fnst@cn.fujitsu.com>
Message-id: 1461665601-14908-1-git-send-email-weijg.fnst@cn.fujitsu.com
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
2016-05-12 15:33:23 +02:00
Janne Karhunen
f249924e96 Allow users to specify the vmdk virtual hardware version.
Vmdk images have metadata to indicate the vmware virtual
hardware version image was created/tested to run with.
Allow users to specify that version via new 'hwversion'
option.

[ kwolf: Adjust qemu-iotests common.filter ]

Signed-off-by: Janne Karhunen <Janne.Karhunen@gmail.com>
Reviewed-by: Fam Zheng <famz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2016-05-12 15:22:08 +02:00