Commit Graph

1584 Commits

Author SHA1 Message Date
Eugenio Pérez
832fef7cc1 util: Return void on iova_tree_remove
It always returns IOVA_OK so nobody uses it.

Acked-by: Jason Wang <jasowang@redhat.com>
Reviewed-by: Peter Xu <peterx@redhat.com>
Signed-off-by: Eugenio Pérez <eperezma@redhat.com>
Message-Id: <20220427154931.3166388-1-eperezma@redhat.com>
Signed-off-by: Laurent Vivier <laurent@vivier.eu>
2022-06-28 10:56:42 +02:00
Emanuele Giuseppe Esposito
7455ff1aa0 aio_wait_kick: add missing memory barrier
It seems that aio_wait_kick always required a memory barrier
or atomic operation in the caller, but nobody actually
took care of doing it.

Let's put the barrier in the function instead, and pair it
with another one in AIO_WAIT_WHILE. Read aio_wait_kick()
comment for further explanation.

Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Emanuele Giuseppe Esposito <eesposit@redhat.com>
Message-Id: <20220524173054.12651-1-eesposit@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2022-06-24 17:07:06 +02:00
Vladimir Sementsov-Ogievskiy
618af89e55 block: simplify handling of try to merge different sized bitmaps
We have too much logic to simply check that bitmaps are of the same
size. Let's just define that hbitmap_merge() and
bdrv_dirty_bitmap_merge_internal() require their argument bitmaps be of
same size, this simplifies things.

Let's look through the callers:

For backup_init_bcs_bitmap() we already assert that merge can't fail.

In bdrv_reclaim_dirty_bitmap_locked() we gracefully handle the error
that can't happen: successor always has same size as its parent, drop
this logic.

In bdrv_merge_dirty_bitmap() we already has assertion and separate
check. Make the check explicit and improve error message.

Signed-off-by: Vladimir Sementsov-Ogievskiy <v.sementsov-og@mail.ru>
Reviewed-by: Nikita Lapshin <nikita.lapshin@virtuozzo.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20220517111206.23585-4-v.sementsov-og@mail.ru>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2022-06-24 17:07:06 +02:00
Richard Henderson
f200ff158d Speed empty timer list in qemu_clock_deadline_ns_all.
Implement remainder for Power3.1 hosts.
 Optimize ppc host icache flushing.
 Cleanups to tcg_accel_ops_init.
 Fix mmio crash accessing unmapped physical memory.
 -----BEGIN PGP SIGNATURE-----
 
 iQFRBAABCgA7FiEEekgeeIaLTbaoWgXAZN846K9+IV8FAmKyLesdHHJpY2hhcmQu
 aGVuZGVyc29uQGxpbmFyby5vcmcACgkQZN846K9+IV8O1wf5AW6JeeUTs2r3owsK
 UpVaRqjlLpNeuktoOQoG8lbVzm1ulEv7zgXYJTZg4cc/83WQZ2G8WzTj3W+Qr/S9
 ECRd73Kou+fK3jTo8I+wPLQjLjkIV4xSABMGz/onxhoAeyS+xcAI4qGuSGrtIg2r
 sQ61V4fWCwvQJdHMyG756Xsh8Xjf18mrNQZ5PLGkyn/e9UIAc4KH6FsgWJdinGEs
 V/oibY20kCXpLxN0ajNmx3x4/NFs/ymMtn1z9fdhVGjAVPY0N6YsxjsGqd/WP/5U
 ui/x0wAhl/VNK2M2+z3hVGfNlMpkzTVG2A3ndD+tYI3nofwTYb/UiakhID7ZX1cQ
 yKDyAw==
 =3Rhw
 -----END PGP SIGNATURE-----

Merge tag 'pull-tcg-20220621' of https://gitlab.com/rth7680/qemu into staging

Speed empty timer list in qemu_clock_deadline_ns_all.
Implement remainder for Power3.1 hosts.
Optimize ppc host icache flushing.
Cleanups to tcg_accel_ops_init.
Fix mmio crash accessing unmapped physical memory.

# -----BEGIN PGP SIGNATURE-----
#
# iQFRBAABCgA7FiEEekgeeIaLTbaoWgXAZN846K9+IV8FAmKyLesdHHJpY2hhcmQu
# aGVuZGVyc29uQGxpbmFyby5vcmcACgkQZN846K9+IV8O1wf5AW6JeeUTs2r3owsK
# UpVaRqjlLpNeuktoOQoG8lbVzm1ulEv7zgXYJTZg4cc/83WQZ2G8WzTj3W+Qr/S9
# ECRd73Kou+fK3jTo8I+wPLQjLjkIV4xSABMGz/onxhoAeyS+xcAI4qGuSGrtIg2r
# sQ61V4fWCwvQJdHMyG756Xsh8Xjf18mrNQZ5PLGkyn/e9UIAc4KH6FsgWJdinGEs
# V/oibY20kCXpLxN0ajNmx3x4/NFs/ymMtn1z9fdhVGjAVPY0N6YsxjsGqd/WP/5U
# ui/x0wAhl/VNK2M2+z3hVGfNlMpkzTVG2A3ndD+tYI3nofwTYb/UiakhID7ZX1cQ
# yKDyAw==
# =3Rhw
# -----END PGP SIGNATURE-----
# gpg: Signature made Tue 21 Jun 2022 01:45:31 PM PDT
# gpg:                using RSA key 7A481E78868B4DB6A85A05C064DF38E8AF7E215F
# gpg:                issuer "richard.henderson@linaro.org"
# gpg: Good signature from "Richard Henderson <richard.henderson@linaro.org>" [ultimate]

* tag 'pull-tcg-20220621' of https://gitlab.com/rth7680/qemu:
  util/cacheflush: Optimize flushing when ppc host has coherent icache
  util/cacheflush: Merge aarch64 ctr_el0 usage
  util: Merge cacheflush.c and cacheinfo.c
  softmmu: Always initialize xlat in address_space_translate_for_iotlb
  qemu-timer: Skip empty timer lists before locking in qemu_clock_deadline_ns_all
  accel/tcg: Reorganize tcg_accel_ops_init()
  accel/tcg: Init TCG cflags in vCPU thread handler
  target/avr: Drop avr_cpu_memory_rw_debug()
  tcg/ppc: implement rem[u]_i{32,64} with mod[su][wd]

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2022-06-21 13:47:20 -07:00
Nicholas Piggin
c79a8e840c util/cacheflush: Optimize flushing when ppc host has coherent icache
On linux, the AT_HWCAP bit PPC_FEATURE_ICACHE_SNOOP indicates
that we can use a simplified 3 instruction flush sequence.

Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
Message-Id: <20220519141131.29839-1-npiggin@gmail.com>
[rth: update after merging cacheflush.c and cacheinfo.c]
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Message-Id: <20220621014837.189139-4-richard.henderson@linaro.org>
2022-06-21 09:28:41 -07:00
Richard Henderson
bdd50dc7d0 util/cacheflush: Merge aarch64 ctr_el0 usage
Merge init_ctr_el0 into arch_cache_info.  In flush_idcache_range,
use the pre-computed line sizes from the global variables.
Use CONFIG_DARWIN in preference to __APPLE__.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Message-Id: <20220621014837.189139-3-richard.henderson@linaro.org>
2022-06-21 09:28:41 -07:00
Richard Henderson
7971375287 util: Merge cacheflush.c and cacheinfo.c
Combine the two files into cacheflush.c.  There's a couple of bits
that would be helpful to share between the two, and combining them
seems better than exporting the bits.

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Message-Id: <20220621014837.189139-2-richard.henderson@linaro.org>
2022-06-21 09:28:41 -07:00
Idan Horowitz
3f42906c9a qemu-timer: Skip empty timer lists before locking in qemu_clock_deadline_ns_all
This decreases qemu_clock_deadline_ns_all's share from 23.2% to 13% in a
profile of icount-enabled aarch64-softmmu.

Signed-off-by: Idan Horowitz <idan.horowitz@gmail.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220114004358.299534-2-idan.horowitz@gmail.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2022-06-21 09:24:34 -07:00
Lucas Mateus Castro (alqotel)
62c9947fb7 host-utils: Implemented signed 256-by-128 division
Based on already existing QEMU implementation created a signed
256 bit by 128 bit division needed to implement the vector divide
extended signed quadword instruction from PowerISA 3.1

Signed-off-by: Lucas Mateus Castro (alqotel) <lucas.araujo@eldorado.org.br>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220525134954.85056-6-lucas.araujo@eldorado.org.br>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
2022-06-20 08:38:58 -03:00
Lucas Mateus Castro (alqotel)
4724bbd284 host-utils: Implemented unsigned 256-by-128 division
Based on already existing QEMU implementation, created an unsigned 256
bit by 128 bit division needed to implement the vector divide extended
unsigned instruction from PowerISA3.1

Signed-off-by: Lucas Mateus Castro (alqotel) <lucas.araujo@eldorado.org.br>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220525134954.85056-5-lucas.araujo@eldorado.org.br>
Signed-off-by: Daniel Henrique Barboza <danielhb413@gmail.com>
2022-06-20 08:38:58 -03:00
Paolo Bonzini
cfb3448922 cutils: add functions for IEC and SI prefixes
Extract the knowledge of IEC and SI prefixes out of size_to_str and
freq_to_str, so that it can be reused when printing statistics.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-06-14 16:50:30 +02:00
Pavel Dovgalyuk
75bbe5e5ec replay: notify vCPU when BH is scheduled
vCPU execution should be suspended when new BH is scheduled.
This is needed to avoid guest timeouts caused by the long cycles
of the execution. In replay mode execution may hang when
vCPU sleeps and block event comes to the queue.
This patch adds notification which wakes up vCPU or interrupts
execution of guest code.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

--

v2: changed first_cpu to current_cpu (suggested by Richard Henderson)
v4: moved vCPU notification to aio_bh_enqueue (suggested by Paolo Bonzini)
Message-Id: <165364837317.688121.17680519919871405281.stgit@pasha-ThinkPad-X280>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-06-06 09:26:53 +02:00
Marc-André Lureau
49e0128c48 util/win32: simplify qemu_get_local_state_dir()
SHGetFolderPath() is a deprecated API:
https://docs.microsoft.com/en-us/windows/win32/api/shlobj_core/nf-shlobj_core-shgetfolderpatha

It is a wrapper for SHGetKnownFolderPath() and CSIDL_COMMON_PATH is
mapped to FOLDERID_ProgramData:
https://docs.microsoft.com/en-us/windows/win32/shell/csidl

g_get_system_data_dirs() is a suitable replacement, as it will have
FOLDERID_ProgramData in the returned list. However, it follows the XDG
Base Directory Specification, if `XDG_DATA_DIRS` is defined, it will be
returned instead.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Stefan Weil <sw@weilnetz.de>
Message-Id: <20220525144140.591926-3-marcandre.lureau@redhat.com>
2022-05-28 11:42:56 +02:00
Marc-André Lureau
06680b15b4 include: move qemu_*_exec_dir() to cutils
The function is required by get_relocated_path() (already in cutils),
and used by qemu-ga and may be generally useful.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20220525144140.591926-2-marcandre.lureau@redhat.com>
2022-05-28 11:42:56 +02:00
Paolo Bonzini
232e925547 thread-pool: remove stopping variable
Just setting the max threads to 0 is enough to stop all workers.

Message-Id: <20220514065012.1149539-4-pbonzini@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Nicolas Saenz Julienne <nsaenzju@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-05-25 21:26:26 +02:00
Paolo Bonzini
900fa208f5 thread-pool: replace semaphore with condition variable
Since commit f9fc8932b1 ("thread-posix: remove the posix semaphore
support", 2022-04-06) QemuSemaphore has its own mutex and condition
variable; this adds unnecessary overhead on I/O with small block sizes.

Check the QTAILQ directly instead of adding the indirection of a
semaphore's count.  Using a semaphore has not been necessary since
qemu_cond_timedwait was introduced; the new code has to be careful about
spurious wakeups but it is simpler, for example thread_pool_cancel does
not have to worry about synchronizing the semaphore count with the number
of elements of pool->request_list.

Note that the return value of qemu_cond_timedwait (0 for timeout, 1 for
signal or spurious wakeup) is different from that of qemu_sem_timedwait
(-1 for timeout, 0 for success).

Reported-by: Lukáš Doktor <ldoktor@redhat.com>
Suggested-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Nicolas Saenz Julienne <nsaenzju@redhat.com>
Message-Id: <20220514065012.1149539-3-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-05-25 21:26:13 +02:00
Paolo Bonzini
3c7b72ddca thread-pool: optimize scheduling of completion bottom half
The completion bottom half was scheduled within the pool->lock
critical section.  That actually results in worse performance,
because the worker thread can run its own small critical section
and go to sleep before the bottom half starts running.

Note that this simple change does not produce an improvement without
changing the thread pool QemuSemaphore to a condition variable.

Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Nicolas Saenz Julienne <nsaenzju@redhat.com>
Message-Id: <20220514065012.1149539-2-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-05-25 21:25:42 +02:00
Richard Henderson
9de5f2b408 * small cleanups for pc-bios/optionrom Makefiles
* checkpatch: fix g_malloc check
 * fix mremap() and RDMA detection
 * confine igd-passthrough-isa-bridge to Xen-enabled builds
 * cover PCI in arm-virt machine qtests
 * add -M boot and -M mem compound properties
 * bump SLIRP submodule
 * support CFI with system libslirp (>= 4.7)
 * clean up CoQueue wakeup functions
 * fix vhost-vsock regression
 * fix --disable-vnc compilation
 * other minor bugfixes
 -----BEGIN PGP SIGNATURE-----
 
 iQFIBAABCAAyFiEE8TM4V0tmI4mGbHaCv/vSX3jHroMFAmJ8/KMUHHBib256aW5p
 QHJlZGhhdC5jb20ACgkQv/vSX3jHroNTTAf9Et1C8iZn+OlZi99wMEeMy8a4mIE5
 CpkBpFphhkBvt3AH7XNsCyL4Gea4QgsI7nOIEVUwvW7gPf85PiBUX8mjrIVg3x1k
 bmMEwMKSTYPmDieAnYBP9zCqZQXNYP8L8WxVs2jFY2GXZ2ZogODYFbvCY4yEEB72
 UR6uIvQRdpiB6BEj8UZ+5i+sDtb0zxqrjzUz8T/PJC9/2JSNgi+sAWWQoQT3PPU7
 R7z2nmEa1VeVLPP6mUHvJKhBltVXF+LyIjQHvo+Tp9tSqp9JwXfFBNQ5W/MFes2D
 skF47N7PdgKRH9Dp4r0j+MqBwoAq86+ao+MKsbQ1Gb91HhoCWt/MrVrVyg==
 =1E6P
 -----END PGP SIGNATURE-----

Merge tag 'for-upstream' of https://gitlab.com/bonzini/qemu into staging

* small cleanups for pc-bios/optionrom Makefiles
* checkpatch: fix g_malloc check
* fix mremap() and RDMA detection
* confine igd-passthrough-isa-bridge to Xen-enabled builds
* cover PCI in arm-virt machine qtests
* add -M boot and -M mem compound properties
* bump SLIRP submodule
* support CFI with system libslirp (>= 4.7)
* clean up CoQueue wakeup functions
* fix vhost-vsock regression
* fix --disable-vnc compilation
* other minor bugfixes

# -----BEGIN PGP SIGNATURE-----
#
# iQFIBAABCAAyFiEE8TM4V0tmI4mGbHaCv/vSX3jHroMFAmJ8/KMUHHBib256aW5p
# QHJlZGhhdC5jb20ACgkQv/vSX3jHroNTTAf9Et1C8iZn+OlZi99wMEeMy8a4mIE5
# CpkBpFphhkBvt3AH7XNsCyL4Gea4QgsI7nOIEVUwvW7gPf85PiBUX8mjrIVg3x1k
# bmMEwMKSTYPmDieAnYBP9zCqZQXNYP8L8WxVs2jFY2GXZ2ZogODYFbvCY4yEEB72
# UR6uIvQRdpiB6BEj8UZ+5i+sDtb0zxqrjzUz8T/PJC9/2JSNgi+sAWWQoQT3PPU7
# R7z2nmEa1VeVLPP6mUHvJKhBltVXF+LyIjQHvo+Tp9tSqp9JwXfFBNQ5W/MFes2D
# skF47N7PdgKRH9Dp4r0j+MqBwoAq86+ao+MKsbQ1Gb91HhoCWt/MrVrVyg==
# =1E6P
# -----END PGP SIGNATURE-----
# gpg: Signature made Thu 12 May 2022 05:25:07 AM PDT
# gpg:                using RSA key F13338574B662389866C7682BFFBD25F78C7AE83
# gpg:                issuer "pbonzini@redhat.com"
# gpg: Good signature from "Paolo Bonzini <bonzini@gnu.org>" [undefined]
# gpg:                 aka "Paolo Bonzini <pbonzini@redhat.com>" [undefined]
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg:          There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 46F5 9FBD 57D6 12E7 BFD4  E2F7 7E15 100C CD36 69B1
#      Subkey fingerprint: F133 3857 4B66 2389 866C  7682 BFFB D25F 78C7 AE83

* tag 'for-upstream' of https://gitlab.com/bonzini/qemu: (27 commits)
  vmxcap: add tertiary execution controls
  vl: make machine type deprecation a warning
  meson: link libpng independent of vnc
  vhost-backend: do not depend on CONFIG_VHOST_VSOCK
  coroutine-lock: qemu_co_queue_restart_all is a coroutine-only qemu_co_enter_all
  coroutine-lock: introduce qemu_co_queue_enter_all
  coroutine-lock: qemu_co_queue_next is a coroutine-only qemu_co_enter_next
  net: slirp: allow CFI with libslirp >= 4.7
  net: slirp: add support for CFI-friendly timer API
  net: slirp: switch to slirp_new
  net: slirp: introduce a wrapper struct for QemuTimer
  slirp: bump submodule past 4.7 release
  machine: move more memory validation to Machine object
  machine: make memory-backend a link property
  machine: add mem compound property
  machine: add boot compound property
  machine: use QAPI struct for boot configuration
  tests/qtest/libqos: Add generic pci host bridge in arm-virt machine
  tests/qtest/libqos: Skip hotplug tests if pci root bus is not hotpluggable
  tests/qtest/libqos/pci: Introduce pio_limit
  ...

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2022-05-12 10:52:15 -07:00
Paolo Bonzini
f0d43b1ece coroutine-lock: qemu_co_queue_restart_all is a coroutine-only qemu_co_enter_all
qemu_co_queue_restart_all is basically the same as qemu_co_enter_all
but without a QemuLockable argument.  That's perfectly fine, but only as
long as the function is marked coroutine_fn.  If used outside coroutine
context, qemu_co_queue_wait will attempt to take the lock and that
is just broken: if you are calling qemu_co_queue_restart_all outside
coroutine context, the lock is going to be a QemuMutex which cannot be
taken twice by the same thread.

The patch adds the marker to qemu_co_queue_restart_all and to its sole
non-coroutine_fn caller; it then reimplements the function in terms of
qemu_co_enter_all_impl, to remove duplicated code and to clarify that the
latter also works in coroutine context.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20220427130830.150180-4-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-05-12 12:29:44 +02:00
Paolo Bonzini
d6ee15adec coroutine-lock: introduce qemu_co_queue_enter_all
Because qemu_co_queue_restart_all does not release the lock, it should
be used only in coroutine context.  Introduce a new function that,
like qemu_co_enter_next, does release the lock, and use it whenever
qemu_co_queue_restart_all was used outside coroutine context.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20220427130830.150180-3-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-05-12 12:29:44 +02:00
Paolo Bonzini
248af9e80a coroutine-lock: qemu_co_queue_next is a coroutine-only qemu_co_enter_next
qemu_co_queue_next is basically the same as qemu_co_enter_next but
without a QemuLockable argument.  That's perfectly fine, but only
as long as the function is marked coroutine_fn.  If used outside
coroutine context, qemu_co_queue_wait will attempt to take the lock
and that is just broken: if you are calling qemu_co_queue_next outside
coroutine context, the lock is going to be a QemuMutex which cannot be
taken twice by the same thread.

The patch adds the marker and reimplements qemu_co_queue_next in terms of
qemu_co_enter_next_impl, to remove duplicated code and to clarify that the
latter also works in coroutine context.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20220427130830.150180-2-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-05-12 12:29:44 +02:00
Kevin Wolf
9ec7a59b5a coroutine: Revert to constant batch size
Commit 4c41c69e changed the way the coroutine pool is sized because for
virtio-blk devices with a large queue size and heavy I/O, it was just
too small and caused coroutines to be deleted and reallocated soon
afterwards. The change made the size dynamic based on the number of
queues and the queue size of virtio-blk devices.

There are two important numbers here: Slightly simplified, when a
coroutine terminates, it is generally stored in the global release pool
up to a certain pool size, and if the pool is full, it is freed.
Conversely, when allocating a new coroutine, the coroutines in the
release pool are reused if the pool already has reached a certain
minimum size (the batch size), otherwise we allocate new coroutines.

The problem after commit 4c41c69e is that it not only increases the
maximum pool size (which is the intended effect), but also the batch
size for reusing coroutines (which is a bug). It means that in cases
with many devices and/or a large queue size (which defaults to the
number of vcpus for virtio-blk-pci), many thousand coroutines could be
sitting in the release pool without being reused.

This is not only a waste of memory and allocations, but it actually
makes the QEMU process likely to hit the vm.max_map_count limit on Linux
because each coroutine requires two mappings (its stack and the guard
page for the stack), causing it to abort() in qemu_alloc_stack() because
when the limit is hit, mprotect() starts to fail with ENOMEM.

In order to fix the problem, change the batch size back to 64 to avoid
uselessly accumulating coroutines in the release pool, but keep the
dynamic maximum pool size so that coroutines aren't freed too early
in heavy I/O scenarios.

Note that this fix doesn't strictly make it impossible to hit the limit,
but this would only happen if most of the coroutines are actually in use
at the same time, not just sitting in a pool. This is the same behaviour
as we already had before commit 4c41c69e. Fully preventing this would
require allowing qemu_coroutine_create() to return an error, but it
doesn't seem to be a scenario that people hit in practice.

Cc: qemu-stable@nongnu.org
Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=2079938
Fixes: 4c41c69e05
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20220510151020.105528-3-kwolf@redhat.com>
Tested-by: Hiroki Narukawa <hnarukaw@yahoo-corp.jp>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2022-05-12 12:21:30 +02:00
Kevin Wolf
98e3ab3505 coroutine: Rename qemu_coroutine_inc/dec_pool_size()
It's true that these functions currently affect the batch size in which
coroutines are reused (i.e. moved from the global release pool to the
allocation pool of a specific thread), but this is a bug and will be
fixed in a separate patch.

In fact, the comment in the header file already just promises that it
influences the pool size, so reflect this in the name of the functions.
As a nice side effect, the shorter function name makes some line
wrapping unnecessary.

Cc: qemu-stable@nongnu.org
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20220510151020.105528-2-kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2022-05-12 12:20:45 +02:00
Nicolas Saenz Julienne
71ad4713cc util/event-loop-base: Introduce options to set the thread pool size
The thread pool regulates itself: when idle, it kills threads until
empty, when in demand, it creates new threads until full. This behaviour
doesn't play well with latency sensitive workloads where the price of
creating a new thread is too high. For example, when paired with qemu's
'-mlock', or using safety features like SafeStack, creating a new thread
has been measured take multiple milliseconds.

In order to mitigate this let's introduce a new 'EventLoopBase'
property to set the thread pool size. The threads will be created during
the pool's initialization or upon updating the property's value, remain
available during its lifetime regardless of demand, and destroyed upon
freeing it. A properly characterized workload will then be able to
configure the pool to avoid any latency spikes.

Signed-off-by: Nicolas Saenz Julienne <nsaenzju@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Message-id: 20220425075723.20019-4-nsaenzju@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2022-05-09 10:43:23 +01:00
Nicolas Saenz Julienne
70ac26b9e5 util/main-loop: Introduce the main loop into QOM
'event-loop-base' provides basic property handling for all 'AioContext'
based event loops. So let's define a new 'MainLoopClass' that inherits
from it. This will permit tweaking the main loop's properties through
qapi as well as through the command line using the '-object' keyword[1].
Only one instance of 'MainLoopClass' might be created at any time.

'EventLoopBaseClass' learns a new callback, 'can_be_deleted()' so as to
mark 'MainLoop' as non-deletable.

[1] For example:
      -object main-loop,id=main-loop,aio-max-batch=<value>

Signed-off-by: Nicolas Saenz Julienne <nsaenzju@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Message-id: 20220425075723.20019-3-nsaenzju@redhat.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
2022-05-09 10:43:23 +01:00
Stefan Hajnoczi
c1fe694357 coroutine-win32: use QEMU_DEFINE_STATIC_CO_TLS()
Thread-Local Storage variables cannot be used directly from coroutine
code because the compiler may optimize TLS variable accesses across
qemu_coroutine_yield() calls. When the coroutine is re-entered from
another thread the TLS variables from the old thread must no longer be
used.

Use QEMU_DEFINE_STATIC_CO_TLS() for the current and leader variables.

I think coroutine-win32.c could get away with __thread because the
variables are only used in situations where either the stale value is
correct (current) or outside coroutine context (loading leader when
current is NULL). Due to the difficulty of being sure that this is
really safe in all scenarios it seems worth converting it anyway.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20220307153853.602859-4-stefanha@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2022-05-04 15:55:23 +02:00
Stefan Hajnoczi
ac387a08a9 coroutine: use QEMU_DEFINE_STATIC_CO_TLS()
Thread-Local Storage variables cannot be used directly from coroutine
code because the compiler may optimize TLS variable accesses across
qemu_coroutine_yield() calls. When the coroutine is re-entered from
another thread the TLS variables from the old thread must no longer be
used.

Use QEMU_DEFINE_STATIC_CO_TLS() for the current and leader variables.
The alloc_pool QSLIST needs a typedef so the return value of
get_ptr_alloc_pool() can be stored in a local variable.

One example of why this code is necessary: a coroutine that yields
before calling qemu_coroutine_create() to create another coroutine is
affected by the TLS issue.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20220307153853.602859-3-stefanha@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2022-05-04 15:55:23 +02:00
Stefan Hajnoczi
34145a307d coroutine-ucontext: use QEMU_DEFINE_STATIC_CO_TLS()
Thread-Local Storage variables cannot be used directly from coroutine
code because the compiler may optimize TLS variable accesses across
qemu_coroutine_yield() calls. When the coroutine is re-entered from
another thread the TLS variables from the old thread must no longer be
used.

Use QEMU_DEFINE_STATIC_CO_TLS() for the current and leader variables.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20220307153853.602859-2-stefanha@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2022-05-04 15:55:23 +02:00
Marc-André Lureau
ff5927baa7 util: rename qemu_*block() socket functions
The qemu_*block() functions are meant to be be used with sockets (the
win32 implementation expects SOCKET)

Over time, those functions where used with Win32 SOCKET or
file-descriptors interchangeably. But for portability, they must only be
used with socket-like file-descriptors. FDs can use
g_unix_set_fd_nonblocking() instead.

Rename the functions with "socket" in the name to prevent bad usages.

This is effectively reverting commit f9e8cacc55 ("oslib-posix:
rename socket_set_nonblock() to qemu_set_nonblock()").

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
2022-05-03 15:53:20 +04:00
Marc-André Lureau
22e135fca3 Replace fcntl(O_NONBLOCK) with g_unix_set_fd_nonblocking()
Suggested-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
2022-05-03 15:47:38 +04:00
Marc-André Lureau
81badab381 util: replace pipe()+cloexec with g_unix_open_pipe()
Suggested-by: Daniel P. Berrangé <berrange@redhat.com>
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
2022-05-03 15:18:14 +04:00
Marc-André Lureau
a7241974ce Replace qemu_pipe() with g_unix_open_pipe()
GLib g_unix_open_pipe() is essentially like qemu_pipe(), available since
2.30.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
2022-05-03 15:17:56 +04:00
Marc-André Lureau
ad24b679d2 block: move fcntl_setfl()
It is only used by block/file-posix.c, move it there.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
2022-05-03 15:17:53 +04:00
Marc-André Lureau
4d14cb0cd7 Use g_unix_set_fd_nonblocking()
API available since glib 2.30. It also preserves errno.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
2022-05-03 15:17:30 +04:00
Marc-André Lureau
638466f777 Use QEMU_SANITIZE_ADDRESS
Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
2022-05-03 15:16:23 +04:00
Paolo Bonzini
7544060ef3 meson, configure: move libgio test to meson
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-04-28 08:52:22 +02:00
Paolo Bonzini
e960a7ee46 remove -writeconfig
Like -set and -readconfig, it would not really be too hard to
extend -writeconfig to parsing mechanisms other than QemuOpts.
However, the uses of -writeconfig are substantially more
limited, as it is generally easier to write the configuration
by hand in the first place.  In addition, -writeconfig does
not even try to detect cases where it prints incorrect
syntax (for example if values have a quote in them, since
qemu_config_parse does not support any kind of escaping.
Just remove it.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20220414145721.326866-1-pbonzini@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-04-28 08:51:56 +02:00
Haiyue Wang
5c08edd1fa error-report: fix g_date_time_format assertion
The 'g_get_real_time' returns the number of microseconds since January
1, 1970 UTC, but 'g_date_time_new_from_unix_utc' needs the number of
seconds, so it will cause the invalid time input:

(process:279642): GLib-CRITICAL (recursed) **: g_date_time_format: assertion 'datetime != NULL' failed

Call function 'g_date_time_new_now_utc' instead, it has the same result
as 'g_date_time_new_from_unix_utc(g_get_real_time() / G_USEC_PER_SEC)';

Fixes: 73dab893b5 ("error-report: replace deprecated g_get_current_time() with glib >= 2.62")
Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20220424105036.291370-1-haiyue.wang@intel.com>
2022-04-28 08:51:56 +02:00
Daniel P. Berrangé
c6b310b37c softmmu: remove deprecated --enable-fips option
Users requiring FIPS support must build QEMU with either the libgcrypt
or gnutls libraries as the crytography backend.

Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2022-04-26 16:12:25 +01:00
Marc-André Lureau
1fbf2665e6 util: replace qemu_get_local_state_pathname()
Simplify the function to only return the directory path. Callers are
adjusted to use the GLib function to build paths, g_build_filename().

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220420132624.2439741-39-marcandre.lureau@redhat.com>
2022-04-21 17:09:09 +04:00
Marc-André Lureau
1b34d08f0b util: use qemu_create() in qemu_write_pidfile()
qemu_open_old(O_CREATE) should be replaced with qemu_create() which
handles Error reporting.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220420132624.2439741-38-marcandre.lureau@redhat.com>
2022-04-21 17:09:09 +04:00
Marc-André Lureau
96eb9b2b47 util: use qemu_write_full() in qemu_write_pidfile()
Mostly for correctness.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220420132624.2439741-37-marcandre.lureau@redhat.com>
2022-04-21 17:09:09 +04:00
Marc-André Lureau
d0dedf2f4c util: simplify write in signal handler
Use qemu_write_full() instead of open-coding a write loop.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220420132624.2439741-36-marcandre.lureau@redhat.com>
2022-04-21 17:09:09 +04:00
Marc-André Lureau
548fb0da73 qga: move qga_get_host_name()
The function is specific to qemu-ga, no need to share it in QEMU.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Konstantin Kostiuk <kkostiuk@redhat.com>
Message-Id: <20220420132624.2439741-32-marcandre.lureau@redhat.com>
2022-04-21 17:09:09 +04:00
Marc-André Lureau
756a98dd70 Move error_printf_unless_qmp() with monitor unit
Since it depends on monitor code, and error_vprintf_unless_qmp() is
already there.

This will help to move error-report in a common subproject.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220420132624.2439741-31-marcandre.lureau@redhat.com>
2022-04-21 17:09:09 +04:00
Marc-André Lureau
9ca9c893b6 include: add qemu/keyval.h
Do not require the whole option machinery to handle keyval, as it is
used by QAPI alone, without the option API. And match the associated
unit name.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220420132624.2439741-24-marcandre.lureau@redhat.com>
2022-04-21 17:03:51 +04:00
Marc-André Lureau
282468c7c4 include: move qemu_fdatasync() to osdep
Move QEMU-specific code to util/osdep.c, so cutils can become a common
subproject.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220420132624.2439741-22-marcandre.lureau@redhat.com>
2022-04-21 17:03:51 +04:00
Marc-André Lureau
73991a9222 include: move qemu_msync() to osdep
The implementation depends on the OS. (and longer-term goal is to move
cutils to a common subproject)

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220420132624.2439741-21-marcandre.lureau@redhat.com>
2022-04-21 17:03:51 +04:00
Marc-André Lureau
951cc9df88 glib-compat: isolate g_date_time_format_iso8601 version-bypass
The solution was discussed with Markus Armbruster during the review:
https://patchew.org/QEMU/20220323155743.1585078-1-marcandre.lureau@redhat.com/20220323155743.1585078-14-marcandre.lureau@redhat.com/

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20220420132624.2439741-3-marcandre.lureau@redhat.com>
2022-04-21 16:56:55 +04:00
Richard Henderson
4e51069d67 util/log: Support per-thread log files
Add a new log flag, tid, to turn this feature on.
Require the log filename to be set, and to contain %d.

Do not allow tid to be turned off once it is on, nor let
the filename be change thereafter.  This avoids the need
for signalling each thread to re-open on a name change.

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20220417183019.755276-40-richard.henderson@linaro.org>
2022-04-20 10:51:11 -07:00