Migration Pull request (20231103)
Hi In this PULL: - dirty limit fixes (hyman) - coverity issues (juan) Please apply. -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEEGJn/jt6/WMzuA0uC9IfvGFhy1yMFAmVE4dgACgkQ9IfvGFhy 1yPBgxAAvrsCHwU6/m9y+XGokyHTKKKIVysLip/14jAjDL+viLYgxdVyOGQKQGBa +yV+XHTaEyKdihKG4Z5nWuC0yM+sdZQpWXQAcHJLPaPs5wDGICICpdAFY2LbAWSK jtX9uq7crywIL4mVKiX+HOjRUPCAYUx/2TcqJf2+0+MKDEVC33ikxNbcx8ZELY+Q +hGyOws3mkHSQjyaNUVgnnQtGzikYqcNO2efa+zVPdXYd+TUWW2e9I++Qf48r0Hv OqeZAB7bSAb39PNRuj0I1gt4d3WTHzHt7BSpX1OuFqQnzLw8vS5iDQH943WAyGkY NblZVb8pyzSg1Jy18H/SmrJDXeufRwqFwD+1NHyxGjsF89KOuVUqGrGpRXhMBtmA DSzdgn5jqW5lI1po9FqGdlPTFlhstpMH3DSfPQWurvJh42oM38gmSEHLBNpc4tXo 8udMYI09H/kHUoNMTZNGjnZO9LfarGsag6eOJP1bMMublhRlKCaL9RIyV9oOHycE IeOeQFeBP/BmYFLWbVPeZej7uiqsEc7VPDJK2QXns210UYanaWmggkmpdAr0I0EV pEKHSfVv1qlIlFH4d7MhcJzP2/rY62EC5tYQjT0UaBnCRcDInKrNWa3kbDL0akwr 0aJgpbT5ipknVChtwnMWJlbqpeW/VUF5g0jVpYQ3jbe/Zf+OtmU= =Pv8z -----END PGP SIGNATURE----- Merge tag 'migration-20231103-pull-request' of https://gitlab.com/juan.quintela/qemu into staging Migration Pull request (20231103) Hi In this PULL: - dirty limit fixes (hyman) - coverity issues (juan) Please apply. # -----BEGIN PGP SIGNATURE----- # # iQIzBAABCAAdFiEEGJn/jt6/WMzuA0uC9IfvGFhy1yMFAmVE4dgACgkQ9IfvGFhy # 1yPBgxAAvrsCHwU6/m9y+XGokyHTKKKIVysLip/14jAjDL+viLYgxdVyOGQKQGBa # +yV+XHTaEyKdihKG4Z5nWuC0yM+sdZQpWXQAcHJLPaPs5wDGICICpdAFY2LbAWSK # jtX9uq7crywIL4mVKiX+HOjRUPCAYUx/2TcqJf2+0+MKDEVC33ikxNbcx8ZELY+Q # +hGyOws3mkHSQjyaNUVgnnQtGzikYqcNO2efa+zVPdXYd+TUWW2e9I++Qf48r0Hv # OqeZAB7bSAb39PNRuj0I1gt4d3WTHzHt7BSpX1OuFqQnzLw8vS5iDQH943WAyGkY # NblZVb8pyzSg1Jy18H/SmrJDXeufRwqFwD+1NHyxGjsF89KOuVUqGrGpRXhMBtmA # DSzdgn5jqW5lI1po9FqGdlPTFlhstpMH3DSfPQWurvJh42oM38gmSEHLBNpc4tXo # 8udMYI09H/kHUoNMTZNGjnZO9LfarGsag6eOJP1bMMublhRlKCaL9RIyV9oOHycE # IeOeQFeBP/BmYFLWbVPeZej7uiqsEc7VPDJK2QXns210UYanaWmggkmpdAr0I0EV # pEKHSfVv1qlIlFH4d7MhcJzP2/rY62EC5tYQjT0UaBnCRcDInKrNWa3kbDL0akwr # 0aJgpbT5ipknVChtwnMWJlbqpeW/VUF5g0jVpYQ3jbe/Zf+OtmU= # =Pv8z # -----END PGP SIGNATURE----- # gpg: Signature made Fri 03 Nov 2023 20:04:40 HKT # gpg: using RSA key 1899FF8EDEBF58CCEE034B82F487EF185872D723 # gpg: Good signature from "Juan Quintela <quintela@redhat.com>" [full] # gpg: aka "Juan Quintela <quintela@trasno.org>" [full] # Primary key fingerprint: 1899 FF8E DEBF 58CC EE03 4B82 F487 EF18 5872 D723 * tag 'migration-20231103-pull-request' of https://gitlab.com/juan.quintela/qemu: migration: Unlock mutex in error case docs/migration: Add the dirty limit section tests/migration: Introduce dirty-limit into guestperf tests/migration: Introduce dirty-ring-size option into guestperf tests: Add migration dirty-limit capability test system/dirtylimit: Drop the reduplicative check system/dirtylimit: Fix a race situation Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
This commit is contained in:
commit
f3604191e2
@ -594,6 +594,77 @@ path.
|
||||
Return path - opened by main thread, written by main thread AND postcopy
|
||||
thread (protected by rp_mutex)
|
||||
|
||||
Dirty limit
|
||||
=====================
|
||||
The dirty limit, short for dirty page rate upper limit, is a new capability
|
||||
introduced in the 8.1 QEMU release that uses a new algorithm based on the KVM
|
||||
dirty ring to throttle down the guest during live migration.
|
||||
|
||||
The algorithm framework is as follows:
|
||||
|
||||
::
|
||||
|
||||
------------------------------------------------------------------------------
|
||||
main --------------> throttle thread ------------> PREPARE(1) <--------
|
||||
thread \ | |
|
||||
\ | |
|
||||
\ V |
|
||||
-\ CALCULATE(2) |
|
||||
\ | |
|
||||
\ | |
|
||||
\ V |
|
||||
\ SET PENALTY(3) -----
|
||||
-\ |
|
||||
\ |
|
||||
\ V
|
||||
-> virtual CPU thread -------> ACCEPT PENALTY(4)
|
||||
------------------------------------------------------------------------------
|
||||
|
||||
When the qmp command qmp_set_vcpu_dirty_limit is called for the first time,
|
||||
the QEMU main thread starts the throttle thread. The throttle thread, once
|
||||
launched, executes the loop, which consists of three steps:
|
||||
|
||||
- PREPARE (1)
|
||||
|
||||
The entire work of PREPARE (1) is preparation for the second stage,
|
||||
CALCULATE(2), as the name implies. It involves preparing the dirty
|
||||
page rate value and the corresponding upper limit of the VM:
|
||||
The dirty page rate is calculated via the KVM dirty ring mechanism,
|
||||
which tells QEMU how many dirty pages a virtual CPU has had since the
|
||||
last KVM_EXIT_DIRTY_RING_FULL exception; The dirty page rate upper
|
||||
limit is specified by caller, therefore fetch it directly.
|
||||
|
||||
- CALCULATE (2)
|
||||
|
||||
Calculate a suitable sleep period for each virtual CPU, which will be
|
||||
used to determine the penalty for the target virtual CPU. The
|
||||
computation must be done carefully in order to reduce the dirty page
|
||||
rate progressively down to the upper limit without oscillation. To
|
||||
achieve this, two strategies are provided: the first is to add or
|
||||
subtract sleep time based on the ratio of the current dirty page rate
|
||||
to the limit, which is used when the current dirty page rate is far
|
||||
from the limit; the second is to add or subtract a fixed time when
|
||||
the current dirty page rate is close to the limit.
|
||||
|
||||
- SET PENALTY (3)
|
||||
|
||||
Set the sleep time for each virtual CPU that should be penalized based
|
||||
on the results of the calculation supplied by step CALCULATE (2).
|
||||
|
||||
After completing the three above stages, the throttle thread loops back
|
||||
to step PREPARE (1) until the dirty limit is reached.
|
||||
|
||||
On the other hand, each virtual CPU thread reads the sleep duration and
|
||||
sleeps in the path of the KVM_EXIT_DIRTY_RING_FULL exception handler, that
|
||||
is ACCEPT PENALTY (4). Virtual CPUs tied with writing processes will
|
||||
obviously exit to the path and get penalized, whereas virtual CPUs involved
|
||||
with read processes will not.
|
||||
|
||||
In summary, thanks to the KVM dirty ring technology, the dirty limit
|
||||
algorithm will restrict virtual CPUs as needed to keep their dirty page
|
||||
rate inside the limit. This leads to more steady reading performance during
|
||||
live migration and can aid in improving large guest responsiveness.
|
||||
|
||||
Postcopy
|
||||
========
|
||||
|
||||
|
110
migration/ram.c
110
migration/ram.c
@ -3030,71 +3030,71 @@ static int ram_save_iterate(QEMUFile *f, void *opaque)
|
||||
* MAX_WAIT (if curious, further see commit 4508bd9ed8053ce) below, which
|
||||
* guarantees that we'll at least released it in a regular basis.
|
||||
*/
|
||||
qemu_mutex_lock(&rs->bitmap_mutex);
|
||||
WITH_RCU_READ_LOCK_GUARD() {
|
||||
if (ram_list.version != rs->last_version) {
|
||||
ram_state_reset(rs);
|
||||
}
|
||||
|
||||
/* Read version before ram_list.blocks */
|
||||
smp_rmb();
|
||||
|
||||
ret = rdma_registration_start(f, RAM_CONTROL_ROUND);
|
||||
if (ret < 0) {
|
||||
qemu_file_set_error(f, ret);
|
||||
goto out;
|
||||
}
|
||||
|
||||
t0 = qemu_clock_get_ns(QEMU_CLOCK_REALTIME);
|
||||
i = 0;
|
||||
while ((ret = migration_rate_exceeded(f)) == 0 ||
|
||||
postcopy_has_request(rs)) {
|
||||
int pages;
|
||||
|
||||
if (qemu_file_get_error(f)) {
|
||||
break;
|
||||
WITH_QEMU_LOCK_GUARD(&rs->bitmap_mutex) {
|
||||
WITH_RCU_READ_LOCK_GUARD() {
|
||||
if (ram_list.version != rs->last_version) {
|
||||
ram_state_reset(rs);
|
||||
}
|
||||
|
||||
pages = ram_find_and_save_block(rs);
|
||||
/* no more pages to sent */
|
||||
if (pages == 0) {
|
||||
done = 1;
|
||||
break;
|
||||
/* Read version before ram_list.blocks */
|
||||
smp_rmb();
|
||||
|
||||
ret = rdma_registration_start(f, RAM_CONTROL_ROUND);
|
||||
if (ret < 0) {
|
||||
qemu_file_set_error(f, ret);
|
||||
goto out;
|
||||
}
|
||||
|
||||
if (pages < 0) {
|
||||
qemu_file_set_error(f, pages);
|
||||
break;
|
||||
}
|
||||
t0 = qemu_clock_get_ns(QEMU_CLOCK_REALTIME);
|
||||
i = 0;
|
||||
while ((ret = migration_rate_exceeded(f)) == 0 ||
|
||||
postcopy_has_request(rs)) {
|
||||
int pages;
|
||||
|
||||
rs->target_page_count += pages;
|
||||
|
||||
/*
|
||||
* During postcopy, it is necessary to make sure one whole host
|
||||
* page is sent in one chunk.
|
||||
*/
|
||||
if (migrate_postcopy_ram()) {
|
||||
compress_flush_data();
|
||||
}
|
||||
|
||||
/*
|
||||
* we want to check in the 1st loop, just in case it was the 1st
|
||||
* time and we had to sync the dirty bitmap.
|
||||
* qemu_clock_get_ns() is a bit expensive, so we only check each
|
||||
* some iterations
|
||||
*/
|
||||
if ((i & 63) == 0) {
|
||||
uint64_t t1 = (qemu_clock_get_ns(QEMU_CLOCK_REALTIME) - t0) /
|
||||
1000000;
|
||||
if (t1 > MAX_WAIT) {
|
||||
trace_ram_save_iterate_big_wait(t1, i);
|
||||
if (qemu_file_get_error(f)) {
|
||||
break;
|
||||
}
|
||||
|
||||
pages = ram_find_and_save_block(rs);
|
||||
/* no more pages to sent */
|
||||
if (pages == 0) {
|
||||
done = 1;
|
||||
break;
|
||||
}
|
||||
|
||||
if (pages < 0) {
|
||||
qemu_file_set_error(f, pages);
|
||||
break;
|
||||
}
|
||||
|
||||
rs->target_page_count += pages;
|
||||
|
||||
/*
|
||||
* During postcopy, it is necessary to make sure one whole host
|
||||
* page is sent in one chunk.
|
||||
*/
|
||||
if (migrate_postcopy_ram()) {
|
||||
compress_flush_data();
|
||||
}
|
||||
|
||||
/*
|
||||
* we want to check in the 1st loop, just in case it was the 1st
|
||||
* time and we had to sync the dirty bitmap.
|
||||
* qemu_clock_get_ns() is a bit expensive, so we only check each
|
||||
* some iterations
|
||||
*/
|
||||
if ((i & 63) == 0) {
|
||||
uint64_t t1 = (qemu_clock_get_ns(QEMU_CLOCK_REALTIME) - t0) /
|
||||
1000000;
|
||||
if (t1 > MAX_WAIT) {
|
||||
trace_ram_save_iterate_big_wait(t1, i);
|
||||
break;
|
||||
}
|
||||
}
|
||||
i++;
|
||||
}
|
||||
i++;
|
||||
}
|
||||
}
|
||||
qemu_mutex_unlock(&rs->bitmap_mutex);
|
||||
|
||||
/*
|
||||
* Must occur before EOS (or any QEMUFile operation)
|
||||
|
@ -411,12 +411,20 @@ void dirtylimit_set_all(uint64_t quota,
|
||||
|
||||
void dirtylimit_vcpu_execute(CPUState *cpu)
|
||||
{
|
||||
if (dirtylimit_in_service() &&
|
||||
dirtylimit_vcpu_get_state(cpu->cpu_index)->enabled &&
|
||||
cpu->throttle_us_per_full) {
|
||||
trace_dirtylimit_vcpu_execute(cpu->cpu_index,
|
||||
cpu->throttle_us_per_full);
|
||||
usleep(cpu->throttle_us_per_full);
|
||||
if (cpu->throttle_us_per_full) {
|
||||
dirtylimit_state_lock();
|
||||
|
||||
if (dirtylimit_in_service() &&
|
||||
dirtylimit_vcpu_get_state(cpu->cpu_index)->enabled) {
|
||||
dirtylimit_state_unlock();
|
||||
trace_dirtylimit_vcpu_execute(cpu->cpu_index,
|
||||
cpu->throttle_us_per_full);
|
||||
|
||||
g_usleep(cpu->throttle_us_per_full);
|
||||
return;
|
||||
}
|
||||
|
||||
dirtylimit_state_unlock();
|
||||
}
|
||||
}
|
||||
|
||||
@ -644,10 +652,6 @@ static struct DirtyLimitInfoList *dirtylimit_query_all(void)
|
||||
|
||||
struct DirtyLimitInfoList *qmp_query_vcpu_dirty_limit(Error **errp)
|
||||
{
|
||||
if (!dirtylimit_in_service()) {
|
||||
return NULL;
|
||||
}
|
||||
|
||||
return dirtylimit_query_all();
|
||||
}
|
||||
|
||||
|
@ -135,4 +135,27 @@ COMPARISONS = [
|
||||
Scenario("compr-multifd-channels-64",
|
||||
multifd=True, multifd_channels=64),
|
||||
]),
|
||||
|
||||
# Looking at effect of dirty-limit with
|
||||
# varying x_vcpu_dirty_limit_period
|
||||
Comparison("compr-dirty-limit-period", scenarios = [
|
||||
Scenario("compr-dirty-limit-period-500",
|
||||
dirty_limit=True, x_vcpu_dirty_limit_period=500),
|
||||
Scenario("compr-dirty-limit-period-800",
|
||||
dirty_limit=True, x_vcpu_dirty_limit_period=800),
|
||||
Scenario("compr-dirty-limit-period-1000",
|
||||
dirty_limit=True, x_vcpu_dirty_limit_period=1000),
|
||||
]),
|
||||
|
||||
|
||||
# Looking at effect of dirty-limit with
|
||||
# varying vcpu_dirty_limit
|
||||
Comparison("compr-dirty-limit", scenarios = [
|
||||
Scenario("compr-dirty-limit-10MB",
|
||||
dirty_limit=True, vcpu_dirty_limit=10),
|
||||
Scenario("compr-dirty-limit-20MB",
|
||||
dirty_limit=True, vcpu_dirty_limit=20),
|
||||
Scenario("compr-dirty-limit-50MB",
|
||||
dirty_limit=True, vcpu_dirty_limit=50),
|
||||
]),
|
||||
]
|
||||
|
@ -102,6 +102,8 @@ class Engine(object):
|
||||
info.get("expected-downtime", 0),
|
||||
info.get("setup-time", 0),
|
||||
info.get("cpu-throttle-percentage", 0),
|
||||
info.get("dirty-limit-throttle-time-per-round", 0),
|
||||
info.get("dirty-limit-ring-full-time", 0),
|
||||
)
|
||||
|
||||
def _migrate(self, hardware, scenario, src, dst, connect_uri):
|
||||
@ -203,6 +205,21 @@ class Engine(object):
|
||||
resp = dst.cmd("migrate-set-parameters",
|
||||
multifd_channels=scenario._multifd_channels)
|
||||
|
||||
if scenario._dirty_limit:
|
||||
if not hardware._dirty_ring_size:
|
||||
raise Exception("dirty ring size must be configured when "
|
||||
"testing dirty limit migration")
|
||||
|
||||
resp = src.cmd("migrate-set-capabilities",
|
||||
capabilities = [
|
||||
{ "capability": "dirty-limit",
|
||||
"state": True }
|
||||
])
|
||||
resp = src.cmd("migrate-set-parameters",
|
||||
x_vcpu_dirty_limit_period=scenario._x_vcpu_dirty_limit_period)
|
||||
resp = src.cmd("migrate-set-parameters",
|
||||
vcpu_dirty_limit=scenario._vcpu_dirty_limit)
|
||||
|
||||
resp = src.cmd("migrate", uri=connect_uri)
|
||||
|
||||
post_copy = False
|
||||
@ -325,7 +342,6 @@ class Engine(object):
|
||||
cmdline = "'" + cmdline + "'"
|
||||
|
||||
argv = [
|
||||
"-accel", "kvm",
|
||||
"-cpu", "host",
|
||||
"-kernel", self._kernel,
|
||||
"-initrd", self._initrd,
|
||||
@ -333,6 +349,11 @@ class Engine(object):
|
||||
"-m", str((hardware._mem * 1024) + 512),
|
||||
"-smp", str(hardware._cpus),
|
||||
]
|
||||
if hardware._dirty_ring_size:
|
||||
argv.extend(["-accel", "kvm,dirty-ring-size=%s" %
|
||||
hardware._dirty_ring_size])
|
||||
else:
|
||||
argv.extend(["-accel", "kvm"])
|
||||
|
||||
argv.extend(self._get_qemu_serial_args())
|
||||
|
||||
|
@ -23,7 +23,8 @@ class Hardware(object):
|
||||
src_cpu_bind=None, src_mem_bind=None,
|
||||
dst_cpu_bind=None, dst_mem_bind=None,
|
||||
prealloc_pages = False,
|
||||
huge_pages=False, locked_pages=False):
|
||||
huge_pages=False, locked_pages=False,
|
||||
dirty_ring_size=0):
|
||||
self._cpus = cpus
|
||||
self._mem = mem # GiB
|
||||
self._src_mem_bind = src_mem_bind # List of NUMA nodes
|
||||
@ -33,6 +34,7 @@ class Hardware(object):
|
||||
self._prealloc_pages = prealloc_pages
|
||||
self._huge_pages = huge_pages
|
||||
self._locked_pages = locked_pages
|
||||
self._dirty_ring_size = dirty_ring_size
|
||||
|
||||
|
||||
def serialize(self):
|
||||
@ -46,6 +48,7 @@ class Hardware(object):
|
||||
"prealloc_pages": self._prealloc_pages,
|
||||
"huge_pages": self._huge_pages,
|
||||
"locked_pages": self._locked_pages,
|
||||
"dirty_ring_size": self._dirty_ring_size,
|
||||
}
|
||||
|
||||
@classmethod
|
||||
@ -59,4 +62,5 @@ class Hardware(object):
|
||||
data["dst_mem_bind"],
|
||||
data["prealloc_pages"],
|
||||
data["huge_pages"],
|
||||
data["locked_pages"])
|
||||
data["locked_pages"],
|
||||
data["dirty_ring_size"])
|
||||
|
@ -81,7 +81,9 @@ class Progress(object):
|
||||
downtime,
|
||||
downtime_expected,
|
||||
setup_time,
|
||||
throttle_pcent):
|
||||
throttle_pcent,
|
||||
dirty_limit_throttle_time_per_round,
|
||||
dirty_limit_ring_full_time):
|
||||
|
||||
self._status = status
|
||||
self._ram = ram
|
||||
@ -91,6 +93,10 @@ class Progress(object):
|
||||
self._downtime_expected = downtime_expected
|
||||
self._setup_time = setup_time
|
||||
self._throttle_pcent = throttle_pcent
|
||||
self._dirty_limit_throttle_time_per_round = \
|
||||
dirty_limit_throttle_time_per_round
|
||||
self._dirty_limit_ring_full_time = \
|
||||
dirty_limit_ring_full_time
|
||||
|
||||
def serialize(self):
|
||||
return {
|
||||
@ -102,6 +108,10 @@ class Progress(object):
|
||||
"downtime_expected": self._downtime_expected,
|
||||
"setup_time": self._setup_time,
|
||||
"throttle_pcent": self._throttle_pcent,
|
||||
"dirty_limit_throttle_time_per_round":
|
||||
self._dirty_limit_throttle_time_per_round,
|
||||
"dirty_limit_ring_full_time":
|
||||
self._dirty_limit_ring_full_time,
|
||||
}
|
||||
|
||||
@classmethod
|
||||
@ -114,4 +124,6 @@ class Progress(object):
|
||||
data["downtime"],
|
||||
data["downtime_expected"],
|
||||
data["setup_time"],
|
||||
data["throttle_pcent"])
|
||||
data["throttle_pcent"],
|
||||
data["dirty_limit_throttle_time_per_round"],
|
||||
data["dirty_limit_ring_full_time"])
|
||||
|
@ -30,7 +30,9 @@ class Scenario(object):
|
||||
auto_converge=False, auto_converge_step=10,
|
||||
compression_mt=False, compression_mt_threads=1,
|
||||
compression_xbzrle=False, compression_xbzrle_cache=10,
|
||||
multifd=False, multifd_channels=2):
|
||||
multifd=False, multifd_channels=2,
|
||||
dirty_limit=False, x_vcpu_dirty_limit_period=500,
|
||||
vcpu_dirty_limit=1):
|
||||
|
||||
self._name = name
|
||||
|
||||
@ -60,6 +62,10 @@ class Scenario(object):
|
||||
self._multifd = multifd
|
||||
self._multifd_channels = multifd_channels
|
||||
|
||||
self._dirty_limit = dirty_limit
|
||||
self._x_vcpu_dirty_limit_period = x_vcpu_dirty_limit_period
|
||||
self._vcpu_dirty_limit = vcpu_dirty_limit
|
||||
|
||||
def serialize(self):
|
||||
return {
|
||||
"name": self._name,
|
||||
@ -79,6 +85,9 @@ class Scenario(object):
|
||||
"compression_xbzrle_cache": self._compression_xbzrle_cache,
|
||||
"multifd": self._multifd,
|
||||
"multifd_channels": self._multifd_channels,
|
||||
"dirty_limit": self._dirty_limit,
|
||||
"x_vcpu_dirty_limit_period": self._x_vcpu_dirty_limit_period,
|
||||
"vcpu_dirty_limit": self._vcpu_dirty_limit,
|
||||
}
|
||||
|
||||
@classmethod
|
||||
|
@ -60,6 +60,8 @@ class BaseShell(object):
|
||||
parser.add_argument("--prealloc-pages", dest="prealloc_pages", default=False)
|
||||
parser.add_argument("--huge-pages", dest="huge_pages", default=False)
|
||||
parser.add_argument("--locked-pages", dest="locked_pages", default=False)
|
||||
parser.add_argument("--dirty-ring-size", dest="dirty_ring_size",
|
||||
default=0, type=int)
|
||||
|
||||
self._parser = parser
|
||||
|
||||
@ -89,7 +91,9 @@ class BaseShell(object):
|
||||
|
||||
locked_pages=args.locked_pages,
|
||||
huge_pages=args.huge_pages,
|
||||
prealloc_pages=args.prealloc_pages)
|
||||
prealloc_pages=args.prealloc_pages,
|
||||
|
||||
dirty_ring_size=args.dirty_ring_size)
|
||||
|
||||
|
||||
class Shell(BaseShell):
|
||||
@ -127,6 +131,17 @@ class Shell(BaseShell):
|
||||
parser.add_argument("--multifd-channels", dest="multifd_channels",
|
||||
default=2, type=int)
|
||||
|
||||
parser.add_argument("--dirty-limit", dest="dirty_limit", default=False,
|
||||
action="store_true")
|
||||
|
||||
parser.add_argument("--x-vcpu-dirty-limit-period",
|
||||
dest="x_vcpu_dirty_limit_period",
|
||||
default=500, type=int)
|
||||
|
||||
parser.add_argument("--vcpu-dirty-limit",
|
||||
dest="vcpu_dirty_limit",
|
||||
default=1, type=int)
|
||||
|
||||
def get_scenario(self, args):
|
||||
return Scenario(name="perfreport",
|
||||
downtime=args.downtime,
|
||||
@ -150,7 +165,12 @@ class Shell(BaseShell):
|
||||
compression_xbzrle_cache=args.compression_xbzrle_cache,
|
||||
|
||||
multifd=args.multifd,
|
||||
multifd_channels=args.multifd_channels)
|
||||
multifd_channels=args.multifd_channels,
|
||||
|
||||
dirty_limit=args.dirty_limit,
|
||||
x_vcpu_dirty_limit_period=\
|
||||
args.x_vcpu_dirty_limit_period,
|
||||
vcpu_dirty_limit=args.vcpu_dirty_limit)
|
||||
|
||||
def run(self, argv):
|
||||
args = self._parser.parse_args(argv)
|
||||
|
@ -3091,6 +3091,166 @@ static void test_vcpu_dirty_limit(void)
|
||||
dirtylimit_stop_vm(vm);
|
||||
}
|
||||
|
||||
static void migrate_dirty_limit_wait_showup(QTestState *from,
|
||||
const int64_t period,
|
||||
const int64_t value)
|
||||
{
|
||||
/* Enable dirty limit capability */
|
||||
migrate_set_capability(from, "dirty-limit", true);
|
||||
|
||||
/* Set dirty limit parameters */
|
||||
migrate_set_parameter_int(from, "x-vcpu-dirty-limit-period", period);
|
||||
migrate_set_parameter_int(from, "vcpu-dirty-limit", value);
|
||||
|
||||
/* Make sure migrate can't converge */
|
||||
migrate_ensure_non_converge(from);
|
||||
|
||||
/* To check limit rate after precopy */
|
||||
migrate_set_capability(from, "pause-before-switchover", true);
|
||||
|
||||
/* Wait for the serial output from the source */
|
||||
wait_for_serial("src_serial");
|
||||
}
|
||||
|
||||
/*
|
||||
* This test does:
|
||||
* source destination
|
||||
* start vm
|
||||
* start incoming vm
|
||||
* migrate
|
||||
* wait dirty limit to begin
|
||||
* cancel migrate
|
||||
* cancellation check
|
||||
* restart incoming vm
|
||||
* migrate
|
||||
* wait dirty limit to begin
|
||||
* wait pre-switchover event
|
||||
* convergence condition check
|
||||
*
|
||||
* And see if dirty limit migration works correctly.
|
||||
* This test case involves many passes, so it runs in slow mode only.
|
||||
*/
|
||||
static void test_migrate_dirty_limit(void)
|
||||
{
|
||||
g_autofree char *uri = g_strdup_printf("unix:%s/migsocket", tmpfs);
|
||||
QTestState *from, *to;
|
||||
int64_t remaining;
|
||||
uint64_t throttle_us_per_full;
|
||||
/*
|
||||
* We want the test to be stable and as fast as possible.
|
||||
* E.g., with 1Gb/s bandwith migration may pass without dirty limit,
|
||||
* so we need to decrease a bandwidth.
|
||||
*/
|
||||
const int64_t dirtylimit_period = 1000, dirtylimit_value = 50;
|
||||
const int64_t max_bandwidth = 400000000; /* ~400Mb/s */
|
||||
const int64_t downtime_limit = 250; /* 250ms */
|
||||
/*
|
||||
* We migrate through unix-socket (> 500Mb/s).
|
||||
* Thus, expected migration speed ~= bandwidth limit (< 500Mb/s).
|
||||
* So, we can predict expected_threshold
|
||||
*/
|
||||
const int64_t expected_threshold = max_bandwidth * downtime_limit / 1000;
|
||||
int max_try_count = 10;
|
||||
MigrateCommon args = {
|
||||
.start = {
|
||||
.hide_stderr = true,
|
||||
.use_dirty_ring = true,
|
||||
},
|
||||
.listen_uri = uri,
|
||||
.connect_uri = uri,
|
||||
};
|
||||
|
||||
/* Start src, dst vm */
|
||||
if (test_migrate_start(&from, &to, args.listen_uri, &args.start)) {
|
||||
return;
|
||||
}
|
||||
|
||||
/* Prepare for dirty limit migration and wait src vm show up */
|
||||
migrate_dirty_limit_wait_showup(from, dirtylimit_period, dirtylimit_value);
|
||||
|
||||
/* Start migrate */
|
||||
migrate_qmp(from, uri, "{}");
|
||||
|
||||
/* Wait for dirty limit throttle begin */
|
||||
throttle_us_per_full = 0;
|
||||
while (throttle_us_per_full == 0) {
|
||||
throttle_us_per_full =
|
||||
read_migrate_property_int(from, "dirty-limit-throttle-time-per-round");
|
||||
usleep(100);
|
||||
g_assert_false(got_src_stop);
|
||||
}
|
||||
|
||||
/* Now cancel migrate and wait for dirty limit throttle switch off */
|
||||
migrate_cancel(from);
|
||||
wait_for_migration_status(from, "cancelled", NULL);
|
||||
|
||||
/* Check if dirty limit throttle switched off, set timeout 1ms */
|
||||
do {
|
||||
throttle_us_per_full =
|
||||
read_migrate_property_int(from, "dirty-limit-throttle-time-per-round");
|
||||
usleep(100);
|
||||
g_assert_false(got_src_stop);
|
||||
} while (throttle_us_per_full != 0 && --max_try_count);
|
||||
|
||||
/* Assert dirty limit is not in service */
|
||||
g_assert_cmpint(throttle_us_per_full, ==, 0);
|
||||
|
||||
args = (MigrateCommon) {
|
||||
.start = {
|
||||
.only_target = true,
|
||||
.use_dirty_ring = true,
|
||||
},
|
||||
.listen_uri = uri,
|
||||
.connect_uri = uri,
|
||||
};
|
||||
|
||||
/* Restart dst vm, src vm already show up so we needn't wait anymore */
|
||||
if (test_migrate_start(&from, &to, args.listen_uri, &args.start)) {
|
||||
return;
|
||||
}
|
||||
|
||||
/* Start migrate */
|
||||
migrate_qmp(from, uri, "{}");
|
||||
|
||||
/* Wait for dirty limit throttle begin */
|
||||
throttle_us_per_full = 0;
|
||||
while (throttle_us_per_full == 0) {
|
||||
throttle_us_per_full =
|
||||
read_migrate_property_int(from, "dirty-limit-throttle-time-per-round");
|
||||
usleep(100);
|
||||
g_assert_false(got_src_stop);
|
||||
}
|
||||
|
||||
/*
|
||||
* The dirty limit rate should equals the return value of
|
||||
* query-vcpu-dirty-limit if dirty limit cap set
|
||||
*/
|
||||
g_assert_cmpint(dirtylimit_value, ==, get_limit_rate(from));
|
||||
|
||||
/* Now, we have tested if dirty limit works, let it converge */
|
||||
migrate_set_parameter_int(from, "downtime-limit", downtime_limit);
|
||||
migrate_set_parameter_int(from, "max-bandwidth", max_bandwidth);
|
||||
|
||||
/*
|
||||
* Wait for pre-switchover status to check if migration
|
||||
* satisfy the convergence condition
|
||||
*/
|
||||
wait_for_migration_status(from, "pre-switchover", NULL);
|
||||
|
||||
remaining = read_ram_property_int(from, "remaining");
|
||||
g_assert_cmpint(remaining, <,
|
||||
(expected_threshold + expected_threshold / 100));
|
||||
|
||||
migrate_continue(from, "pre-switchover");
|
||||
|
||||
qtest_qmp_eventwait(to, "RESUME");
|
||||
|
||||
wait_for_serial("dest_serial");
|
||||
wait_for_migration_complete(from);
|
||||
|
||||
test_migrate_end(from, to, true);
|
||||
}
|
||||
|
||||
static bool kvm_dirty_ring_supported(void)
|
||||
{
|
||||
#if defined(__linux__) && defined(HOST_X86_64)
|
||||
@ -3301,6 +3461,10 @@ int main(int argc, char **argv)
|
||||
*/
|
||||
if (g_test_slow()) {
|
||||
qtest_add_func("/migration/auto_converge", test_migrate_auto_converge);
|
||||
if (g_str_equal(arch, "x86_64") &&
|
||||
has_kvm && kvm_dirty_ring_supported()) {
|
||||
qtest_add_func("/migration/dirty_limit", test_migrate_dirty_limit);
|
||||
}
|
||||
}
|
||||
qtest_add_func("/migration/multifd/tcp/plain/none",
|
||||
test_multifd_tcp_none);
|
||||
|
Loading…
Reference in New Issue
Block a user