Commit Graph

21 Commits

Author SHA1 Message Date
Thomas Huth
b482fb43de hw/misc/sga: Remove the deprecated "sga" device
It's been deprecated since QEMU v6.2, so it should be OK to
finally remove this now.

Message-Id: <20230209161540.1054669-1-thuth@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2023-02-14 09:02:42 +01:00
Murilo Opsfelder Araujo
8763196c2c tests/migration: add support for ppc64le for guestperf.py
Add support for ppc64le for guestperf.py. On ppc, console is usually
hvc0 and serial device for pseries machine is spapr-vty.

Signed-off-by: Murilo Opsfelder Araujo <muriloo@linux.ibm.com>
Reviewed-by: Daniel Henrique Barboza <danielhb413@gmail.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Message-Id: <20220809002451.91541-3-muriloo@linux.ibm.com>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
2023-02-05 06:40:28 -03:00
dinglimin
770beadb98 tests/migration: remove the unused local variable
Remove the unused local variable "records".

Signed-off-by: dinglimin <dinglimin@cmss.chinamobile.com>
Reviewed-by: Ján Tomko <jtomko@redhat.com>
Message-Id: <20220928080555.2263-1-dinglimin@cmss.chinamobile.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2022-10-11 12:37:12 +02:00
Hyman
927fae0eb9 tests/migration: fix unix socket migration
The test aborts and error message as the following be throwed:
"No such file or directory: '/var/tmp/qemu-migrate-{pid}.migrate",
when the unix socket migration test nearly done. The reason is
qemu removes the unix socket file after migration before
guestperf.py script do it. So pre-check if the socket file exists
when removing it to prevent the guestperf program from aborting.

See also commit f9cc00346d ("tests/migration: fix unix socket batch
migration").

Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Signed-off-by: Hyman <huangy81@chinatelecom.cn>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2021-07-14 14:15:52 +01:00
Hyman Huang(黄勇)
fa264f4266 tests/migration: fix "downtime_limit" type when "migrate-set-parameters"
migrate-set-parameters parse "downtime_limit" as integer type when
execute "migrate-set-parameters" before migration, and, the unit
dowtime_limit is milliseconds, fix this two so that test can go
smoothly.

Signed-off-by: Hyman Huang(黄勇) <huangy81@chinatelecom.cn>
Message-Id: <31d82df24cc0c468dbe4d2d86730158ebf248071.1622729934.git.huangy81@chinatelecom.cn>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2021-07-05 10:51:26 +01:00
Hyman Huang(黄勇)
c99fb3a50d tests/migration: parse the thread-id key of CpuInfoFast
thread_id in CpuInfoFast is deprecated, parse thread-id instead
after execute qmp query-cpus-fast. fix this so that test can
go smoothly.

Signed-off-by: Hyman Huang(黄勇) <huangy81@chinatelecom.cn>
Message-Id: <584578c0a0dd781cee45f72ddf517f6e6a41c504.1622729934.git.huangy81@chinatelecom.cn>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2021-07-05 10:51:26 +01:00
Hyman
1c3baa1ac4 tests/migration: introduce multifd into guestperf
Guestperf tool does not cover the multifd-enabled migration
currently, it is worth supporting so that developers can
analysis the migration performance with all kinds of
migration.

To request that multifd is enabled, with 4 channels:
$ ./tests/migration/guestperf.py \
    --multifd --multifd-channels 4 --output output.json

To run the entire standardized set of multifd-enabled
comparisons, with unix migration:
$ ./tests/migration/guestperf-batch.py \
    --dst-host localhost --transport unix \
    --filter compr-multifd* --output outputdir

Signed-off-by: Hyman Huang(黄勇) <huangy81@chinatelecom.cn>
Message-Id: <cfeeb04d17ad932c42a9871294058b77429ad1b7.1616171924.git.huangy81@chinatelecom.cn>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2021-05-13 18:21:14 +01:00
Hyman Huang(黄勇)
e999fa47b2 tests/migration: fix parameter of auto-converge migration
when execute the following test command:
$ ./guestperf-batch.py --auto-converge \
    --auto-converge-step {percent} ...
test aborts and error message be throwed as the following:
"Parameter 'x-cpu-throttle-increment' is unexpected"

The reason is that 'x-cpu-throttle-increment' has been
deprecated and 'cpu-throttle-increment' was introduced
Since v2.7. Use the new parameter instead.

Signed-off-by: Hyman Huang(黄勇) <huangy81@chinatelecom.cn>
Message-Id: <0195d34a317ce3cc417b3efd275e30cad35a7618.1616513998.git.huangy81@chinatelecom.cn>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2021-04-07 18:37:56 +01:00
Peter Maydell
1b507e55f8 Remove many old deprecated features
The following features have been deprecated for well over the 2
 release cycle we promise
 
   ``-drive file=json:{...{'driver':'file'}}`` (since 3.0)
   ``-vnc acl`` (since 4.0.0)
   ``-mon ...,control=readline,pretty=on|off`` (since 4.1)
   ``migrate_set_downtime`` and ``migrate_set_speed`` (since 2.8.0)
   ``query-named-block-nodes`` result ``encryption_key_missing`` (since 2.10.0)
   ``query-block`` result ``inserted.encryption_key_missing`` (since 2.10.0)
   ``migrate-set-cache-size`` and ``query-migrate-cache-size`` (since 2.11.0)
   ``query-named-block-nodes`` and ``query-block`` result dirty-bitmaps[i].status (since 4.0)
   ``query-cpus`` (since 2.12.0)
   ``query-cpus-fast`` ``arch`` output member (since 3.0.0)
   ``query-events`` (since 4.0)
   chardev client socket with ``wait`` option (since 4.0)
   ``acl_show``, ``acl_reset``, ``acl_policy``, ``acl_add``, ``acl_remove`` (since 4.0.0)
   ``ide-drive`` (since 4.2)
   ``scsi-disk`` (since 4.2)
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEE2vOm/bJrYpEtDo4/vobrtBUQT98FAmBTHBsACgkQvobrtBUQ
 T9/6cA//QVlWFvNJjmYuXMFOkYdokqde9W66iMO2vt+p3LSqlcxVa59+ZOcb5XWc
 7xYcZlDHEHRMe8DWrTwUyjftJ7Ild+/HY5Nz8NqDFvX08umIhC6qmzBSG64O3fU5
 KtSLWe+KfbUH2usmczQO1zGG8mSIAMnOdugpLln9W13v7X9NO4yC+/O36Mm8l4VK
 VEF+o9w7+s2l7QloLdhVe7RVOrqz6+9TFHx1+q1pPhMbzlv5vbfp9vIreiCgmqiO
 rPLXB4bjklktWSkJBe/2Jt7ixNQZrT0DfnCmshHJmmBxTCI1+b798Xy+7j9Dz8ER
 LzKHhlsilCv/jb8QKlIZSxvoDE02vAgwyVXlqxc7FdyaLmg/Emwr5LGUFHKCyhDR
 3hmJafNokdWPcHwBYwRTdbG1OlgKA+yHSBzQ4OnjsTsKrrzlBrm9B9DiDVABXadS
 5NnhvjZfaRmzjXZinf1th7fdMtLs4k+9XKn6szkrp2QhtlX9gyhkXuOhdwRVcyJB
 V/8dLZDdOB3Kpbu5DGlmmIzdHQx5bo/tQqZlvJJNmcOWe+TKwHg56WRUYQ3PFQZK
 M2A4XTd5kil6+PT7AnDHLoYAFdfScq6R32V38AI7S34kPrejftbSbDz8EVRXnrzj
 zzXGNWtSQpIDZGgNEg3Ta6aIf28TxLCA7nRhuPYIDcswlUsHn4c=
 =cFRH
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/berrange-gitlab/tags/dep-many-pull-request' into staging

Remove many old deprecated features

The following features have been deprecated for well over the 2
release cycle we promise

  ``-drive file=json:{...{'driver':'file'}}`` (since 3.0)
  ``-vnc acl`` (since 4.0.0)
  ``-mon ...,control=readline,pretty=on|off`` (since 4.1)
  ``migrate_set_downtime`` and ``migrate_set_speed`` (since 2.8.0)
  ``query-named-block-nodes`` result ``encryption_key_missing`` (since 2.10.0)
  ``query-block`` result ``inserted.encryption_key_missing`` (since 2.10.0)
  ``migrate-set-cache-size`` and ``query-migrate-cache-size`` (since 2.11.0)
  ``query-named-block-nodes`` and ``query-block`` result dirty-bitmaps[i].status (since 4.0)
  ``query-cpus`` (since 2.12.0)
  ``query-cpus-fast`` ``arch`` output member (since 3.0.0)
  ``query-events`` (since 4.0)
  chardev client socket with ``wait`` option (since 4.0)
  ``acl_show``, ``acl_reset``, ``acl_policy``, ``acl_add``, ``acl_remove`` (since 4.0.0)
  ``ide-drive`` (since 4.2)
  ``scsi-disk`` (since 4.2)

# gpg: Signature made Thu 18 Mar 2021 09:23:39 GMT
# gpg:                using RSA key DAF3A6FDB26B62912D0E8E3FBE86EBB415104FDF
# gpg: Good signature from "Daniel P. Berrange <dan@berrange.com>" [full]
# gpg:                 aka "Daniel P. Berrange <berrange@redhat.com>" [full]
# Primary key fingerprint: DAF3 A6FD B26B 6291 2D0E  8E3F BE86 EBB4 1510 4FDF

* remotes/berrange-gitlab/tags/dep-many-pull-request:
  block: remove support for using "file" driver with block/char devices
  block: remove 'dirty-bitmaps' field from 'BlockInfo' struct
  block: remove dirty bitmaps 'status' field
  block: remove 'encryption_key_missing' flag from QAPI
  hw/scsi: remove 'scsi-disk' device
  hw/ide: remove 'ide-drive' device
  chardev: reject use of 'wait' flag for socket client chardevs
  machine: remove 'arch' field from 'query-cpus-fast' QMP command
  machine: remove 'query-cpus' QMP command
  migrate: remove QMP/HMP commands for speed, downtime and cache size
  monitor: remove 'query-events' QMP command
  monitor: raise error when 'pretty' option is used with HMP
  ui, monitor: remove deprecated VNC ACL option and HMP commands

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-03-18 19:00:50 +00:00
Daniel P. Berrangé
8af54b9172 machine: remove 'query-cpus' QMP command
The newer 'query-cpus-fast' command avoids side effects on the guest
execution. Note that some of the field names are different in the
'query-cpus-fast' command.

Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Tested-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2021-03-18 09:22:55 +00:00
Daniel P. Berrangé
cbde7be900 migrate: remove QMP/HMP commands for speed, downtime and cache size
The generic 'migrate_set_parameters' command handle all types of param.

Only the QMP commands were documented in the deprecations page, but the
rationale for deprecating applies equally to HMP, and the replacements
exist. Furthermore the HMP commands are just shims to the QMP commands,
so removing the latter breaks the former unless they get re-implemented.

Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2021-03-18 09:22:55 +00:00
Hyman
f9cc00346d tests/migration: fix unix socket batch migration
when execute the following test command:
"guestperf-batch.py --dst-host localhost --transport unix ..."
test aborts and error message as the following be throwed:
"launching VM Failed: [Errno 98] Address already in use".

The reason is that batch script use the same monitor socket
in all test cases and do not remove the socket file. The second
migration test will launch vm use the same socket file as
the first, so we get the error message. To fix it, just remove
the socket file each time we have done the migration test.

Signed-off-by: Hyman <huangy81@chinatelecom.cn>
Message-Id: <c3fc438993b87a6ab0bea3d07f6ca0260d29936e.1615397103.git.huangy81@chinatelecom.cn>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Cleber Rosa <crosa@redhat.com>
2021-03-16 21:08:37 -04:00
Thomas Huth
c8c9dc42b7 Remove the deprecated -realtime option
It has been marked as deprecated since QEMU v4.2, replaced by
the -overcommit option. Time to remove it now.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20201210155808.233895-4-thuth@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-12-15 12:52:02 -05:00
Gan Qixin
3a645d364c tests/migration: Fix LGPL information in the file headers
There never was a "Lesser GPL version 2.0", It is either "GPL version 2.0"
or "Lesser GPL version 2.1". This patch replaces all "Lesser GPL version 2.0"
with "Lesser GPL version 2.1" in the tests/migration folder.

Signed-off-by: Gan Qixin <ganqixin@huawei.com>
Message-Id: <20201110184223.549499-2-ganqixin@huawei.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-11-15 17:04:40 +01:00
Paolo Bonzini
423edd9a31 drop "from __future__ import print_function"
This is only needed for Python 2, which we do not support anymore.

Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Acked-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20200204160604.19883-1-pbonzini@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:15:16 +01:00
Thomas Huth
976e8c5414 Replace '-machine accel=xyz' with '-accel xyz'
We've got a separate option to configure the accelerator nowadays, which
is shorter to type and the preferred way of specifying an accelerator.
Use it in the source and examples to show that it is the favored option.
(However, do not touch the places yet which also specify other machine
options or multiple accelerators - these are currently still better
handled with one single "-machine" statement instead)

Signed-off-by: Thomas Huth <thuth@redhat.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20190904052739.22123-1-thuth@redhat.com>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
2019-09-19 12:01:48 +02:00
John Snow
abf0bf998d python/qemu: split QEMUMachine out from underneath __init__.py
It's not obvious that something named __init__.py actually houses
important code that isn't relevant to python packaging glue. Move the
QEMUMachine and related error classes out into their own module.

Adjust users to the new import location.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20190627212816.27298-2-jsnow@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2019-07-01 19:02:10 -03:00
Cleber Rosa
8f8fd9edba Introduce a Python module structure
This is a simple move of Python code that wraps common QEMU
functionality, and are used by a number of different tests
and scripts.

By treating that code as a real Python module, we can more easily:
 * reuse code
 * have a proper place for the module's own unittests
 * apply a more consistent style
 * generate documentation

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Caio Carrara <ccarrara@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20190206162901.19082-2-crosa@redhat.com>
Signed-off-by: Cleber Rosa <crosa@redhat.com>
2019-02-22 14:07:01 -05:00
Eduardo Habkost
f03868bd56 python: futurize -f libfuturize.fixes.fix_print_with_import
Change all Python code to use print as a function.

This is necessary for Python 3 compatibility.

Done using:

  $ py=$( (g grep -l -E '^#!.*python';find -name '*.py' -printf '%P\n';) | \
    sort -u | grep -v README.sh4)
  $ futurize -w -f libfuturize.fixes.fix_print_with_import $py

Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Acked-by: Fam Zheng <famz@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Message-Id: <20180608122952.2009-2-ehabkost@redhat.com>
[ehabkost: fixup tests/docker/docker.py]
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2018-06-08 14:39:24 -03:00
Eduardo Habkost
1a6d375710 scripts: Remove debug parameter from QEMUMachine
All scripts that use the QEMUMachine and QEMUQtestMachine classes
(device-crash-test, tests/migration/*, iotests.py, basevm.py)
already configure logging.

The basicConfig() call inside QEMUMachine.__init__() is being
kept just to make sure a script would still work if it didn't
configure logging.

Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
Message-Id: <20171005172013.3098-4-ehabkost@redhat.com>
Reviewed-by: Lukáš Doktor <ldoktor@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2017-10-11 15:15:17 -03:00
Daniel P. Berrange
409437e16d tests: introduce a framework for testing migration performance
This introduces a moderately general purpose framework for
testing performance of migration.

The initial guest workload is provided by the included 'stress'
program, which is configured to spawn one thread per guest CPU
and run a maximally memory intensive workload. It will loop
over GB of memory, xor'ing each byte with data from a 4k array
of random bytes. This ensures heavy read and write load across
all of guest memory to stress the migration performance. While
running the 'stress' program will record how long it takes to
xor each GB of memory and print this data for later reporting.

The test engine will spawn a pair of QEMU processes, either on
the same host, or with the target on a remote host via ssh,
using the host kernel and a custom initrd built with 'stress'
as the /init binary. Kernel command line args are set to ensure
a fast kernel boot time (< 1 second) between launching QEMU and
the stress program starting execution.

None the less, the test engine will initially wait N seconds for
the guest workload to stablize, before starting the migration
operation. When migration is running, the engine will use pause,
post-copy, autoconverge, xbzrle compression and multithread
compression features, as well as downtime & bandwidth tuning
to encourage completion. If migration completes, the test engine
will wait N seconds again for the guest workooad to stablize on
the target host. If migration does not complete after a preset
number of iterations, it will be aborted.

While the QEMU process is running on the source host, the test
engine will sample the host CPU usage of QEMU as a whole, and
each vCPU thread. While migration is running, it will record
all the stats reported by 'query-migration'. Finally, it will
capture the output of the stress program running in the guest.

All the data produced from a single test execution is recorded
in a structured JSON file. A separate program is then able to
create interactive charts using the "plotly" python + javascript
libraries, showing the characteristics of the migration.

The data output provides visualization of the effect on guest
vCPU workloads from the migration process, the corresponding
vCPU utilization on the host, and the overall CPU hit from
QEMU on the host. This is correlated from statistics from the
migration process, such as downtime, vCPU throttling and iteration
number.

While the tests can be run individually with arbitrary parameters,
there is also a facility for producing batch reports for a number
of pre-defined scenarios / comparisons, in order to be able to
get standardized results across different hardware configurations
(eg TCP vs RDMA, or comparing different VCPU counts / memory
sizes, etc).

To use this, first you must build the initrd image

 $ make tests/migration/initrd-stress.img

To run a a one-shot test with all default parameters

 $ ./tests/migration/guestperf.py > result.json

This has many command line args for varying its behaviour.
For example, to increase the RAM size and CPU count and
bind it to specific host NUMA nodes

 $ ./tests/migration/guestperf.py \
       --mem 4 --cpus 2 \
       --src-mem-bind 0 --src-cpu-bind 0,1 \
       --dst-mem-bind 1 --dst-cpu-bind 2,3 \
       > result.json

Using mem + cpu binding is strongly recommended on NUMA
machines, otherwise the guest performance results will
vary wildly between runs of the test due to lucky/unlucky
NUMA placement, making sensible data analysis impossible.

To make it run across separate hosts:

 $ ./tests/migration/guestperf.py \
       --dst-host somehostname > result.json

To request that post-copy is enabled, with switchover
after 5 iterations

 $ ./tests/migration/guestperf.py \
       --post-copy --post-copy-iters 5 > result.json

Once a result.json file is created, a graph of the data
can be generated, showing guest workload performance per
thread and the migration iteration points:

 $ ./tests/migration/guestperf-plot.py --output result.html \
        --migration-iters --split-guest-cpu result.json

To further include host vCPU utilization and overall QEMU
utilization

 $ ./tests/migration/guestperf-plot.py --output result.html \
        --migration-iters --split-guest-cpu \
	--qemu-cpu --vcpu-cpu result.json

NB, the 'guestperf-plot.py' command requires that you have
the plotly python library installed. eg you must do

 $ pip install --user  plotly

Viewing the result.html file requires that you have the
plotly.min.js file in the same directory as the HTML
output. This js file is installed as part of the plotly
python library, so can be found in

  $HOME/.local/lib/python2.7/site-packages/plotly/offline/plotly.min.js

The guestperf-plot.py program can accept multiple json files
to plot, enabling results from different configurations to
be compared.

Finally, to run the entire standardized set of comparisons

  $ ./tests/migration/guestperf-batch.py \
       --dst-host somehost \
       --mem 4 --cpus 2 \
       --src-mem-bind 0 --src-cpu-bind 0,1 \
       --dst-mem-bind 1 --dst-cpu-bind 2,3
       --output tcp-somehost-4gb-2cpu

will store JSON files from all scenarios in the directory
named tcp-somehost-4gb-2cpu

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Message-Id: <1469020993-29426-7-git-send-email-berrange@redhat.com>
Signed-off-by: Amit Shah <amit.shah@redhat.com>
2016-07-22 13:23:39 +05:30