Commit Graph

61071 Commits

Author SHA1 Message Date
Philippe Mathieu-Daudé
9d5e546af0 docker: Fix trivial typo
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20180508144358.13530-1-f4bug@amsat.org>
Reviewed-by: Laurent Vivier <laurent@vivier.eu>
Signed-off-by: Fam Zheng <famz@redhat.com>
2018-05-18 16:52:53 +08:00
Peter Maydell
a4207e3b00 -----BEGIN PGP SIGNATURE-----
iQIcBAABAgAGBQJa/GyHAAoJEL2+eyfA3jBX6QMP/0mgTLa12N0wAsTAe+QcLBkM
 wh7tJF/vlGBDigxzrGb43unq8wQT7HoCx83XpjpSwirLD+n5vQ++xeoKB0CTs/27
 jH29jqY52/jPRv293kTwizybQASWBMDxQcPAtYrXd3shmm8BwzrnP3L1zFuAT0cn
 xCaVD/ZXkA+OFn9QMQASu0bbmfhpfksvW/xIPnQNCjYLY/FpoanxJBe0yuc3VSaA
 nAbw30HU6jD8n4e/YHpoVBn4KFN8B3DEqcobmcWl/byD7OhlQJYof8k5smcuvdfz
 IkUBNMBy3i73qJWdPXKaef4G+Ku9rVe48kT2jaTbw7OGbY95Djc9fuChkf2G1zlJ
 UDJQN3x7AeiOFu/cHvzmyEXdbwBVvrHojERd4YK7OUHkS/EwM3cRm+BAKzHcypF2
 d7OixPF8AO0rQzO2FL3KrUW7wuQZbKTDL4Cw7BB/F01iakvV+ZN+WBOucSL5trNI
 ouSLhoceobBdaCea1nECHmxFk+xcIhNyXn0ylxc8+O8RLcFzm39o4k5nkQyzPiD7
 kqSCs43P/EocSVohH78O2zdvZ2dPpv9nzm8NZ2vbC4aLAX7+b/iqTSuNz6MA8ocI
 KVdsyRbJUX4Tz3tnPM4usWkfkIf2MyIDHXu9zOOtSwZ22/k2cIXxmn3lbI176Ev0
 V7WJ4BGp6FCOD2Q4Aj8h
 =VUBH
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/cody/tags/block-pull-request' into staging

# gpg: Signature made Wed 16 May 2018 18:38:15 BST
# gpg:                using RSA key BDBE7B27C0DE3057
# gpg: Good signature from "Jeffrey Cody <jcody@redhat.com>"
# gpg:                 aka "Jeffrey Cody <jeff@codyprime.org>"
# gpg:                 aka "Jeffrey Cody <codyprime@gmail.com>"
# Primary key fingerprint: 9957 4B4D 3474 90E7 9D98  D624 BDBE 7B27 C0DE 3057

* remotes/cody/tags/block-pull-request:
  nfs: Remove processed options from QDict
  nfs: Fix error path in nfs_options_qdict_to_qapi()
  blockjob: do not cancel timer in resume
  qemu-iotests: reduce chance of races in 185

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-05-17 11:59:50 +01:00
Peter Maydell
eb7514ae10 migration/next for 20180515
-----BEGIN PGP SIGNATURE-----
 
 iQIcBAABCAAGBQJa+1ceAAoJEPSH7xhYctcjdHQP/2To/iZXf7lp2mgxnN0YeDRV
 ivOooHXsCfYx5MtZCFo11wlZjGlnE6b5yEUzU7MapRLUgO3kd+gbDld/90gVdbLg
 umDL2nxTSjvIAlpg6O8aiQLaQLPvMDf1WSQQdyBHcqauy3KLaFmlm0ldK1besatQ
 KFniKXc8fL0R9QWA4aGDe22Sq9m7M9AKfmJU3NO9qNvxDImyMU2ix4w6cTZ0BtXk
 ltLju5zegBkKUxLFCgip7k6iCHh//tnBymQaLx6O7DdUJWIdKS+fHuCbA8WR24v2
 pKsTcTICjI4FLWJMyIiH012paHoYGQIGdIx3Dv0JoQA6fT8x3arpztX3YVYFqrwF
 hsC6CAnHufDioipEhJk7JcEAGJ+82krqvTG1nRQe/ULvDSHGDjpwF1y0BxzGMVw6
 hU+Ds4ohxyWJYJ0cx9msuKyIlCj1EooIuO6l1unmjBdRAptC8/bZXITVTixEHG+e
 X5vnBqloCL7PUATlaXipqVJfLA8T1MMNfwb11bI4OhTfteueFAAdyeJ2zCBqGBLa
 SK1sxYvMYY3/pzBhAO3YcTJIQofDQ2IciUvylr2jzCd2hB5Qj5In9789LVbl5I9B
 55cICuVcsiXS7krJysSxtdCbjOB44Iv6M09vmA4ap8dmi9AWL/4LQIAJHVc1+koK
 6y76K1BBFdB6aMkDzBeg
 =1a+b
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/juanquintela/tags/migration/20180515' into staging

migration/next for 20180515

# gpg: Signature made Tue 15 May 2018 22:54:38 BST
# gpg:                using RSA key F487EF185872D723
# gpg: Good signature from "Juan Quintela <quintela@redhat.com>"
# gpg:                 aka "Juan Quintela <quintela@trasno.org>"
# Primary key fingerprint: 1899 FF8E DEBF 58CC EE03  4B82 F487 EF18 5872 D723

* remotes/juanquintela/tags/migration/20180515: (40 commits)
  Migration+TLS: Fix crash due to double cleanup
  migration: Textual fixups for blocktime
  migration: update index field when delete or qsort RDMALocalBlock
  migration: update docs
  migration/hmp: add migrate_pause command
  migration/qmp: add command migrate-pause
  migration: introduce lock for to_dst_file
  hmp/migration: add migrate_recover command
  qmp/migration: new command migrate-recover
  migration: init dst in migration_object_init too
  migration: final handshake for the resume
  migration: setup ramstate for resume
  migration: synchronize dirty bitmap for resume
  migration: introduce SaveVMHandlers.resume_prepare
  migration: new message MIG_RP_MSG_RESUME_ACK
  migration: new cmd MIG_CMD_POSTCOPY_RESUME
  migration: new message MIG_RP_MSG_RECV_BITMAP
  migration: new cmd MIG_CMD_RECV_BITMAP
  migration: wakeup dst ram-load-thread for recover
  migration: new state "postcopy-recover"
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-05-17 11:10:12 +01:00
Peter Maydell
61126a8b4b x86 queue, 2018-05-15
* KnightsMill CPU model
 * CLDEMOTE(Demote Cache Line) cpu feature
 * pc-i440fx-2.13 and pc-q35-2.13 machine-types
 * Add model-specific cache information to EPYC CPU model
 -----BEGIN PGP SIGNATURE-----
 
 iQIcBAABCAAGBQJa+1bIAAoJECgHk2+YTcWmebgP/2337b3L2z5K6YRBOjaH/Kmw
 xuBmns8B2ta4/KFX1IjX+7TRvl8qJT39TadLs6OGaPKDHi/OIK7vhXZZdgqm3o2H
 WKpsPYp6MPYx/Ff8eOesHGUyxDTqP49rMYTTbP1GdgoulqjUq0XqgmdF4+uKesei
 1v4NG7M6H2VxlAdkF1PeZUyuuEEZT6F1T2O43FDTQHOQRWmb1XAHiSgiqC4431J9
 9SVe4E0v8i96zrMVeaezvQrMVCCGIo3zC3JZBfbX205Yehl/eCK5WktO03zIn9yS
 IK7Gqwh4RfWmRg2wef+5qEQ1fl11XlH895/F0wMcZ8sRrSm04jNcBs9O4kc59Zz2
 kyvaGkCG14aTC2Y35H0BsNd1AszKaWfknIJKNMnPkeXYLYAbPm1bwgBmXVXAa0Ux
 mI6dk4ArwHALNl2srNOnJRvtmgohm+1HCZtlstSzHVWzNFS9nfGWK3kpgDqXft8w
 TKe2r928uQtympCK7s43wIstT1KRwaVn7ivNpcotJgojCuUg0tK4PQvP82Z/ZXUh
 pUxhW03al6Dz2Khh8vpz88rryCh72i5BQ9S7D8kAcFpyQDd6DfbLv6ebu/mWgjLR
 EwMdJkLQjXGs1SNcpA085KVksGW5U1S2avjeakkp3ceSA/fBt4UdftTmwG6oORaS
 pCxNIzozXx4QdDK81aaE
 =uP+F
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/ehabkost/tags/x86-next-pull-request' into staging

x86 queue, 2018-05-15

* KnightsMill CPU model
* CLDEMOTE(Demote Cache Line) cpu feature
* pc-i440fx-2.13 and pc-q35-2.13 machine-types
* Add model-specific cache information to EPYC CPU model

# gpg: Signature made Tue 15 May 2018 22:53:12 BST
# gpg:                using RSA key 2807936F984DC5A6
# gpg: Good signature from "Eduardo Habkost <ehabkost@redhat.com>"
# Primary key fingerprint: 5A32 2FD5 ABC4 D3DB ACCF  D1AA 2807 936F 984D C5A6

* remotes/ehabkost/tags/x86-next-pull-request:
  i386: Add new property to control cache info
  pc: add 2.13 machine types
  i386: Initialize cache information for EPYC family processors
  i386: Add cache information in X86CPUDefinition
  i386: Helpers to encode cache information consistently
  x86/cpu: Enable CLDEMOTE(Demote Cache Line) cpu feature
  i386: add KnightsMill cpu model

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-05-17 09:57:55 +01:00
Kevin Wolf
c82be42cc8 nfs: Remove processed options from QDict
Commit c22a03454 QAPIfied option parsing in the NFS block driver, but
forgot to remove all the options we processed. Therefore, we get an
error in bdrv_open_inherit(), which thinks the remaining options are
invalid. Trying to open an NFS image will result in an error like this:

    Block protocol 'nfs' doesn't support the option 'server.host'

Remove all options from the QDict to make the NFS driver work again.

Cc: qemu-stable@nongnu.org
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-id: 20180516160816.26259-1-kwolf@redhat.com
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Jeff Cody <jcody@redhat.com>
Signed-off-by: Jeff Cody <jcody@redhat.com>
2018-05-16 13:37:47 -04:00
Kevin Wolf
54b7af4369 nfs: Fix error path in nfs_options_qdict_to_qapi()
Don't throw away local_err, but propagate it to errp.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-id: 20180516161034.27440-1-kwolf@redhat.com
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Jeff Cody <jcody@redhat.com>
Signed-off-by: Jeff Cody <jcody@redhat.com>
2018-05-16 13:37:47 -04:00
Stefan Hajnoczi
4c7e813ce9 blockjob: do not cancel timer in resume
Currently the timer is cancelled and the block job is entered by
block_job_resume().  This behavior causes drain to run extra blockjob
iterations when the job was sleeping due to the ratelimit.

This patch leaves the job asleep when block_job_resume() is called.
Jobs can still be forcibly woken up using block_job_enter(), which is
used to cancel jobs.

After this patch drain no longer runs extra blockjob iterations.  This
is the expected behavior that qemu-iotests 185 used to rely on.  We
temporarily changed the 185 test output to make it pass for the QEMU
2.12 release but now it's time to address this issue.

Cc: QingFeng Hao <haoqf@linux.vnet.ibm.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: QingFeng Hao <haoqf@linux.vnet.ibm.com>
Message-id: 20180508135436.30140-3-stefanha@redhat.com
Reviewed-by: Jeff Cody <jcody@redhat.com>
Signed-off-by: Jeff Cody <jcody@redhat.com>
2018-05-16 13:37:33 -04:00
Stefan Hajnoczi
ddf2d98a94 qemu-iotests: reduce chance of races in 185
Commit 8565c3ab53 ("qemu-iotests: fix
185") identified a race condition in a sub-test.

Similar issues also affect the other sub-tests.  If disk I/O completes
quickly, it races with the QMP 'quit' command.  This causes spurious
test failures because QMP events are emitted in an unpredictable order.

This test relies on QEMU internals and there is no QMP API for getting
deterministic behavior needed to make this test 100% reliable.  At the
same time, the test is useful and it would be a shame to remove it.

Add sleep 0.5 to reduce the chance of races.  This is not a real fix but
appears to reduce spurious failures in practice.

Cc: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 20180508135436.30140-2-stefanha@redhat.com
Reviewed-by: Jeff Cody <jcody@redhat.com>
Signed-off-by: Jeff Cody <jcody@redhat.com>
2018-05-16 13:37:17 -04:00
Dr. David Alan Gilbert
8b7bf2bada Migration+TLS: Fix crash due to double cleanup
During a TLS connect we see:
  migration_channel_connect calls
  migration_tls_channel_connect
  (calls after TLS setup)
  migration_channel_connect

My previous error handling fix made migration_channel_connect
call migrate_fd_connect in all cases; unfortunately the above
means it gets called twice and crashes doing double cleanup.

Fixes: 688a3dcba9

Reported-by: Peter Krempa <pkrempa@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20180430185943.35714-1-dgilbert@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
2018-05-15 22:13:08 +02:00
Dr. David Alan Gilbert
5e50cae409 migration: Textual fixups for blocktime
Blank lines and comments as suggested by Eric.

Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20180427111502.9822-1-dgilbert@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
2018-05-15 22:13:08 +02:00
Lidong Chen
71cd73061c migration: update index field when delete or qsort RDMALocalBlock
rdma_delete_block function deletes RDMALocalBlock base on index field,
but not update the index field. So when next time invoke rdma_delete_block,
it will not work correctly.

If start and cancel migration repeatedly, some RDMALocalBlock not invoke
ibv_dereg_mr to decrease kernel mm_struct vmpin. When vmpin is large than
max locked memory limitation, ibv_reg_mr will failed, and migration can not
start successfully again.

Signed-off-by: Lidong Chen <lidongchen@tencent.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Message-Id: <1525618499-1560-1-git-send-email-lidongchen@tencent.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>

Signed-off-by: Lidong Chen <jemmy858585@gmail.com>
2018-05-15 22:13:08 +02:00
Dr. David Alan Gilbert
edd7080692 migration: update docs
Update the migration docs:

Among other changes:
  * Added a general list of advice for device authors
  * Reordered the section on conditional state (subsections etc)
    into the order we prefer.
  * Add a note about firmware

Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Reviewed-by: Balamuruhan S <bala24@linux.vnet.ibm.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Message-Id: <20180503191059.19576-1-dgilbert@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
2018-05-15 22:13:08 +02:00
Peter Xu
d37297dc66 migration/hmp: add migrate_pause command
Wrapper for QMP command "migrate-pause".

Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180502104740.12123-25-peterx@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
2018-05-15 22:13:08 +02:00
Peter Xu
bfbf89c2b5 migration/qmp: add command migrate-pause
It pauses an ongoing migration.  Currently it only supports postcopy.
Note that this command will work on either side of the migration.
Basically when we trigger this on one side, it'll interrupt the other
side as well since the other side will get notified on the disconnect
event.

However, it's still possible that the other side is not notified, for
example, when the network is totally broken, or due to some firewall
configuration changes.  In that case, we will also need to run the same
command on the other side so both sides will go into the paused state.

Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180502104740.12123-24-peterx@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>

---
s/2.12/2.13/
2018-05-15 22:12:57 +02:00
Peter Xu
62df066fff migration: introduce lock for to_dst_file
Let's introduce a lock for that QEMUFile since we are going to operate
on it in multiple threads.

Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180502104740.12123-23-peterx@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
2018-05-15 22:12:41 +02:00
Peter Xu
3b563c4be0 hmp/migration: add migrate_recover command
Sister command to migrate-recover in QMP.

Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180502104740.12123-22-peterx@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
2018-05-15 22:12:41 +02:00
Peter Xu
02affd41b1 qmp/migration: new command migrate-recover
The first allow-oob=true command.  It's used on destination side when
the postcopy migration is paused and ready for a recovery.  After
execution, a new migration channel will be established for postcopy to
continue.

Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180502104740.12123-21-peterx@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
---
s/2.12/2.13/
2018-05-15 22:11:45 +02:00
Peter Xu
e1b1b1bc36 migration: init dst in migration_object_init too
Though we may not need it, now we init both the src/dst migration
objects in migration_object_init() so that even incoming migration
object would be thread safe (it was not).

Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180502104740.12123-20-peterx@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
2018-05-15 20:57:01 +02:00
Peter Xu
9419069695 migration: final handshake for the resume
Finish the last step to do the final handshake for the recovery.

First source sends one MIG_CMD_RESUME to dst, telling that source is
ready to resume.

Then, dest replies with MIG_RP_MSG_RESUME_ACK to source, telling that
dest is ready to resume (after switch to postcopy-active state).

When source received the RESUME_ACK, it switches its state to
postcopy-active, and finally the recovery is completed.

Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180502104740.12123-19-peterx@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
2018-05-15 20:57:00 +02:00
Peter Xu
08614f3497 migration: setup ramstate for resume
After we updated the dirty bitmaps of ramblocks, we also need to update
the critical fields in RAMState to make sure it is ready for a resume.

Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180502104740.12123-18-peterx@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
2018-05-15 20:56:59 +02:00
Peter Xu
edd090c728 migration: synchronize dirty bitmap for resume
This patch implements the first part of core RAM resume logic for
postcopy. ram_resume_prepare() is provided for the work.

When the migration is interrupted by network failure, the dirty bitmap
on the source side will be meaningless, because even the dirty bit is
cleared, it is still possible that the sent page was lost along the way
to destination. Here instead of continue the migration with the old
dirty bitmap on source, we ask the destination side to send back its
received bitmap, then invert it to be our initial dirty bitmap.

The source side send thread will issue the MIG_CMD_RECV_BITMAP requests,
once per ramblock, to ask for the received bitmap. On destination side,
MIG_RP_MSG_RECV_BITMAP will be issued, along with the requested bitmap.
Data will be received on the return-path thread of source, and the main
migration thread will be notified when all the ramblock bitmaps are
synchronized.

Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180502104740.12123-17-peterx@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
2018-05-15 20:56:57 +02:00
Peter Xu
d1b8eadbc4 migration: introduce SaveVMHandlers.resume_prepare
This is hook function to be called when a postcopy migration wants to
resume from a failure. For each module, it should provide its own
recovery logic before we switch to the postcopy-active state.

Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180502104740.12123-16-peterx@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
2018-05-15 20:56:55 +02:00
Peter Xu
13955b89ce migration: new message MIG_RP_MSG_RESUME_ACK
Creating new message to reply for MIG_CMD_POSTCOPY_RESUME. One uint32_t
is used as payload to let the source know whether destination is ready
to continue the migration.

Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180502104740.12123-15-peterx@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
2018-05-15 20:56:53 +02:00
Peter Xu
3f5875eca5 migration: new cmd MIG_CMD_POSTCOPY_RESUME
Introducing this new command to be sent when the source VM is ready to
resume the paused migration.  What the destination does here is
basically release the fault thread to continue service page faults.

Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180502104740.12123-14-peterx@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
2018-05-15 20:56:52 +02:00
Peter Xu
a335debb35 migration: new message MIG_RP_MSG_RECV_BITMAP
Introducing new return path message MIG_RP_MSG_RECV_BITMAP to send
received bitmap of ramblock back to source.

This is the reply message of MIG_CMD_RECV_BITMAP, it contains not only
the header (including the ramblock name), and it was appended with the
whole ramblock received bitmap on the destination side.

When the source receives such a reply message (MIG_RP_MSG_RECV_BITMAP),
it parses it, convert it to the dirty bitmap by inverting the bits.

One thing to mention is that, when we send the recv bitmap, we are doing
these things in extra:

- converting the bitmap to little endian, to support when hosts are
  using different endianess on src/dst.

- do proper alignment for 8 bytes, to support when hosts are using
  different word size (32/64 bits) on src/dst.

Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180502104740.12123-13-peterx@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
2018-05-15 20:56:51 +02:00
Peter Xu
f25d42253c migration: new cmd MIG_CMD_RECV_BITMAP
Add a new vm command MIG_CMD_RECV_BITMAP to request received bitmap for
one ramblock.

Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180502104740.12123-12-peterx@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
2018-05-15 20:56:49 +02:00
Peter Xu
d96c9e8d78 migration: wakeup dst ram-load-thread for recover
On the destination side, we cannot wake up all the threads when we got
reconnected. The first thing to do is to wake up the main load thread,
so that we can continue to receive valid messages from source again and
reply when needed.

At this point, we switch the destination VM state from postcopy-paused
back to postcopy-recover.

Now we are finally ready to do the resume logic.

Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180502104740.12123-11-peterx@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
2018-05-15 20:56:48 +02:00
Peter Xu
135b87b4f0 migration: new state "postcopy-recover"
Introducing new migration state "postcopy-recover". If a migration
procedure is paused and the connection is rebuilt afterward
successfully, we'll switch the source VM state from "postcopy-paused" to
the new state "postcopy-recover", then we'll do the resume logic in the
migration thread (along with the return path thread).

This patch only do the state switch on source side. Another following up
patch will handle the state switching on destination side using the same
status bit.

Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180502104740.12123-10-peterx@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>

---
s/2.11/2.13/
2018-05-15 20:56:30 +02:00
Peter Xu
d3e35b8f62 migration: rebuild channel on source
This patch detects the "resume" flag of migration command, rebuild the
channels only if the flag is set.

Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180502104740.12123-9-peterx@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
2018-05-15 20:55:19 +02:00
Peter Xu
7a4da28b26 qmp: hmp: add migrate "resume" option
It will be used when we want to resume one paused migration.

Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180502104740.12123-8-peterx@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
---
s/2.12/2.13/
2018-05-15 20:54:49 +02:00
Peter Xu
3a7804c306 migration: allow fault thread to pause
Allows the fault thread to stop handling page faults temporarily. When
network failure happened (and if we expect a recovery afterwards), we
should not allow the fault thread to continue sending things to source,
instead, it should halt for a while until the connection is rebuilt.

When the dest main thread noticed the failure, it kicks the fault thread
to switch to pause state.

Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180502104740.12123-7-peterx@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
2018-05-15 20:24:27 +02:00
Peter Xu
14b1742eaa migration: allow src return path to pause
Let the thread pause for network issues.

Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180502104740.12123-6-peterx@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
2018-05-15 20:24:27 +02:00
Peter Xu
b411b844fb migration: allow dst vm pause on postcopy
When there is IO error on the incoming channel (e.g., network down),
instead of bailing out immediately, we allow the dst vm to switch to the
new POSTCOPY_PAUSE state. Currently it is still simple - it waits the
new semaphore, until someone poke it for another attempt.

One note is that here on ram loading thread we cannot detect the
POSTCOPY_ACTIVE state, but we need to detect the more specific
POSTCOPY_INCOMING_RUNNING state, to make sure we have already loaded all
the device states.

Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180502104740.12123-5-peterx@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
2018-05-15 20:24:27 +02:00
Peter Xu
b23c2ade25 migration: implement "postcopy-pause" src logic
Now when network down for postcopy, the source side will not fail the
migration. Instead we convert the status into this new paused state, and
we will try to wait for a rescue in the future.

If a recovery is detected, migration_thread() will reset its local
variables to prepare for that.

Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180502104740.12123-4-peterx@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
2018-05-15 20:24:27 +02:00
Peter Xu
a688d2c1ab migration: new postcopy-pause state
Introducing a new state "postcopy-paused", which can be used when the
postcopy migration is paused. It is targeted for postcopy network
failure recovery.

Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180502104740.12123-3-peterx@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
2018-05-15 20:24:27 +02:00
Peter Xu
e89f5ff2c3 migration: let incoming side use thread context
The old incoming migration is running in main thread and default
gcontext.  With the new qio_channel_add_watch_full() we can now let it
run in the thread's own gcontext (if there is one).

Currently this patch does nothing alone.  But when any of the incoming
migration is run in another iothread (e.g., the upcoming migrate-recover
command), this patch will bind the incoming logic to the iothread
instead of the main thread (which may already get page faulted and
hanged).

RDMA is not considered for now since it's not even using the QIO watch
framework at all.

Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20180502104740.12123-2-peterx@redhat.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
2018-05-15 20:24:27 +02:00
Juan Quintela
8c4598f2b1 migration: Define MultifdRecvParams sooner
Once there, we don't need the struct names anywhere, just the
typedefs.  And now also document all fields.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2018-05-15 20:24:27 +02:00
Juan Quintela
af8b7d2b09 migration: Transmit initial package through the multifd channels
Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>

--

Be network agnostic.
Add error checking for all values.
2018-05-15 20:24:27 +02:00
Juan Quintela
36c2f8be2c migration: Delay start of migration main routines
We need to make sure that we have started all the multifd threads.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2018-05-15 20:24:27 +02:00
Juan Quintela
60df2d4ae5 migration: Create multifd channels
In both sides.  We still don't transmit anything through them.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2018-05-15 20:24:27 +02:00
Juan Quintela
3854956ad7 migration: Export functions to create send channels
Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2018-05-15 20:24:27 +02:00
Juan Quintela
62c1e0ca73 migration: Be sure all recv channels are created
We need them before we start migration.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2018-05-15 20:24:27 +02:00
Juan Quintela
667707078d migration: terminate_* can be called for other threads
Once there, make  count field to always be accessed with atomic
operations.  To make blocking operations, we need to know that the
thread is running, so create a bool to indicate that.

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>

--

Once here, s/terminate_multifd_*-threads/multifd_*_terminate_threads/
This is consistente with every other function
2018-05-15 20:24:27 +02:00
Juan Quintela
71bb07dbfc migration: Introduce multifd_recv_new_channel()
Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
2018-05-15 20:24:27 +02:00
Juan Quintela
7a169d745c migration: Set error state in case of error
Signed-off-by: Juan Quintela <quintela@redhat.com>
2018-05-15 20:24:27 +02:00
Juan Quintela
cdf338152f tests: Migration ppc now inlines its program
No need to write it to a file.  Just need a proper firmware O:-)

Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Laurent Vivier <lvivier@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
2018-05-15 20:24:27 +02:00
Juan Quintela
2884100cc6 tests: Add migration precopy test
Signed-off-by: Juan Quintela <quintela@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
2018-05-15 20:24:00 +02:00
Xiao Guangrong
701b1876c0 migration: fix saving normal page even if it's been compressed
Fix the bug introduced by da3f56cb2e (migration: remove
ram_save_compressed_page()), It should be 'return' rather than
'res'

Sorry for this stupid mistake :(

Signed-off-by: Xiao Guangrong <xiaoguangrong@tencent.com>
Message-Id: <20180428081045.8878-1-xiaoguangrong@tencent.com>
Signed-off-by: Juan Quintela <quintela@redhat.com>
2018-05-15 20:24:00 +02:00
Peter Maydell
c416eecea5 Block layer patches:
- Switch AIO/callback based block drivers to a byte-based interface
 - Block jobs: Expose error string via query-block-jobs
 - Block job cleanups and fixes
 - hmp: Allow using a qdev id in block_set_io_throttle
 -----BEGIN PGP SIGNATURE-----
 
 iQIcBAABAgAGBQJa+v22AAoJEH8JsnLIjy/Wy5kP/21fPSvmyxMsSQ5lF5hlkodr
 qNFIV6EuQ46YXSr4KzK2Dw88YR18nI5SlMd6mzWF9qx7WhjTMeHhARG9G497cQty
 yDb3Y6dwiuhVndWLMzj/590miqk5TnJvFx5ii88oEnsrbcjKmTY78KMkl/q1bHSp
 qL7sBhI3zPol+y28mvILPXgKsqnabvS/cmsQJCISUfSdFsnsxXVABUPI/WKe1ecs
 UE3tl3cDA/0F8TYDerPUX1RZLJcr7yoXc91ieVWrzug4SStY3HZqWT8SShe2igN0
 w5eWshUBhccHKeiqKx8vdXN7MzNP4v5H2lstTdqV/zDQEXO9vqF1X91zaRBJNb+o
 A4M3lZU/U3xideBo7Hvp4euJ5f6ZKNswpeIC3Ppky788Q+HU/d+cQF+eZUxm+t8y
 vVtixTToSt52dIaAPMsssWCtVwkS4IFO9RLXJeRs94XR3ocrsSdJNOQPTodWafjF
 BXIF4wyKlPvVMzisvpj6jsjVR4Oq7J7P+EXq+hAMbio9WlsXssWu0ZedH8oV+mKl
 UkEB5TgRAaz8MpYNCCEhUwdmHLwlc/PaPN+BAlvQCnDWMtfiFTys6Sh9N9vv/c16
 HISDkR8sf7gPm3VLnt2fPoVOQMU3Gjtany+RySXfn1AzCRG3tPzdS/Ay+s1WpoYv
 7F2Pm9QLepP7xRB48yD3
 =JFBS
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches:

- Switch AIO/callback based block drivers to a byte-based interface
- Block jobs: Expose error string via query-block-jobs
- Block job cleanups and fixes
- hmp: Allow using a qdev id in block_set_io_throttle

# gpg: Signature made Tue 15 May 2018 16:33:10 BST
# gpg:                using RSA key 7F09B272C88F2FD6
# gpg: Good signature from "Kevin Wolf <kwolf@redhat.com>"
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74  56FE 7F09 B272 C88F 2FD6

* remotes/kevin/tags/for-upstream: (37 commits)
  iotests: Add test for -U/force-share conflicts
  qemu-img: Use only string options in img_open_opts
  qemu-io: Use purely string blockdev options
  block: Document BDRV_REQ_WRITE_UNCHANGED support
  qemu-img: Check post-truncation size
  iotests: Add test for COR across nodes
  iotests: Copy 197 for COR filter driver
  iotests: Clean up wrap image in 197
  block: Support BDRV_REQ_WRITE_UNCHANGED in filters
  block/quorum: Support BDRV_REQ_WRITE_UNCHANGED
  block: Set BDRV_REQ_WRITE_UNCHANGED for COR writes
  block: Add BDRV_REQ_WRITE_UNCHANGED flag
  block: BLK_PERM_WRITE includes ..._UNCHANGED
  block: Add COR filter driver
  iotests: Skip 181 and 201 without userfaultfd
  iotests: Add failure matching to common.qemu
  docs: Document the new default sizes of the qcow2 caches
  qcow2: Give the refcount cache the minimum possible size by default
  specs/qcow2: Clarify that compressed clusters have the COPIED bit reset
  Fix error message about compressed clusters with OFLAG_COPIED
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2018-05-15 17:02:00 +01:00
Babu Moger
ab8f992e3e i386: Add new property to control cache info
The property legacy-cache will be used to control the cache information.
If user passes "-cpu legacy-cache" then older information will
be displayed even if the hardware supports new information. Otherwise
use the statically loaded cache definitions if available.

Renamed the previous cache structures to legacy_*. If there is any change in
the cache information, then it needs to be initialized in builtin_x86_defs.

Signed-off-by: Babu Moger <babu.moger@amd.com>
Tested-by: Geoffrey McRae <geoff@hostfission.com>
Message-Id: <20180514164156.27034-3-babu.moger@amd.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
2018-05-15 11:33:33 -03:00