The "expected failure" tests for decodetree result in the
error messages from decodetree ending up in logs and in
V=1 output:
>>> MALLOC_PERTURB_=226 /mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/x86/pyvenv/bin/python3 /mnt/nvmedisk/linaro/qemu-from-laptop/qemu/scripts/decodetree.py --output-null --test-for-error /mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/x86/../../tests/decode/err_argset1.decode
――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――― ✀ ――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――
/mnt/nvmedisk/linaro/qemu-from-laptop/qemu/build/x86/../../tests/decode/err_argset1.decode:5: error: duplicate argument "a"
―――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――
1/44 qemu:decodetree / err_argset1 OK 0.05s
This then produces false positives when scanning the
logfiles for strings like "error: ".
For the expected-failure tests, make decodetree print
"detected:" instead of "error:".
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20230720131521.1325905-1-peter.maydell@linaro.org
The curses display handles most control-X keys, and translates
them into their corresponding keycode. Here we recognize
a few that are missing, Ctrl-@ (null), Ctrl-\ (backslash),
Ctrl-] (right bracket), Ctrl-^ (caret), Ctrl-_ (underscore).
Signed-off-by: Sean Estabrooks <sean.estabrooks@gmail.com>
Message-id: CAHyVn3Bh9CRgDuOmf7G7Ngwamu8d4cVozAcB2i4ymnnggBXNmg@mail.gmail.com
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
A lot of the code called from helper_exception_bkpt_insn() is written
assuming A-profile, but we will also call this helper on M-profile
CPUs when they execute a BKPT insn. This used to work by accident,
but recent changes mean that we will hit an assert when some of this
code calls down into lower level functions that end up calling
arm_security_space_below_el3(), arm_el_is_aa64(), and other functions
that now explicitly assert that the guest CPU is not M-profile.
Handle M-profile directly to avoid the assertions:
* in arm_debug_target_el(), M-profile debug exceptions always
go to EL1
* in arm_debug_exception_fsr(), M-profile always uses the short
format FSR (compare commit d7fe699be5, though in this case
the code in arm_v7m_cpu_do_interrupt() does not need to
look at the FSR value at all)
Cc: qemu-stable@nongnu.org
Resolves: https://gitlab.com/qemu-project/qemu/-/issues/1775
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-id: 20230721143239.1753066-1-peter.maydell@linaro.org
The POSIX definition of the 'read' utility requires that you
specify the variable name to set; omitting the name and
having it default to 'REPLY' is a bashism. If your system
sh is dash, then it will print an error message during build:
qemu/pc-bios/s390-ccw/../../scripts/git-submodule.sh: 106: read: arg count
Specify the variable name explicitly.
Fixes: fdb8fd8cb9 ("git-submodule: allow partial update of .git-submodule-status")
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-id: 20230720153038.1587196-1-peter.maydell@linaro.org
The implementation of the SMMUv3 has multiple places where it reads a
data structure from the guest and directly operates on it without
doing a guest-to-host endianness conversion. Since all SMMU data
structures are little-endian, this means that the SMMU doesn't work
on a big-endian host. In particular, this causes the Avocado test
machine_aarch64_virt.py:Aarch64VirtMachine.test_alpine_virt_tcg_gic_max
to fail on an s390x host.
Add appropriate byte-swapping on reads and writes of guest in-memory
data structures so that the device works correctly on big-endian
hosts.
As part of this we constrain queue_read() to operate only on Cmd
structs and queue_write() on Evt structs, because in practice these
are the only data structures the two functions are used with, and we
need to know what the data structure is to be able to byte-swap its
parts correctly.
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Tested-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Message-id: 20230717132641.764660-1-peter.maydell@linaro.org
Cc: qemu-stable@nongnu.org
The virtio-gpu test is known to be flaky - that's why we also did
not enable the test_s390x_fedora in the gitlab CI. However, a flaky
test can also be annoying when testing locally, so let's rather skip
this subtest by default and start running the test_s390x_fedora test
in the gitlab CI again (since the other things that are tested here
are quite valuable).
Message-Id: <20230724084851.24251-1-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Cédric Le Goater <clg@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
The test in tests/avocado/machine_loongarch.py is currently failing
on big endian hosts like s390x. By comparing the traces between running
the QEMU_EFI.fd bios on a s390x and on a x86 host, it's quickly obvious
that the CSRRD instruction for the CPUID is behaving differently. And
indeed: The code currently does a long read (i.e. 64 bit) from the
address that points to the CPUState->cpu_index field (with tcg_gen_ld_tl()
in the trans_csrrd() function). But this cpu_index field is only an "int"
(i.e. 32 bit). While this dirty pointer magic works on little endian hosts,
it of course fails on big endian hosts. Fix it by using a proper helper
function instead.
Message-Id: <20230720175307.854460-1-thuth@redhat.com>
Reviewed-by: Song Gao <gaosong@loongson.cn>
Signed-off-by: Thomas Huth <thuth@redhat.com>
The tests from tests/avocado/migration.py do not work at all
on s390x - the bios shuts down immediately when it cannot find
a boot disk, so there is nothing left to migrate here. For doing
a proper migration test, we would need a proper payload, but we
already do such tests in the migration *qtest*, so it is unnecessary
to redo such a test here, thus let's simply remove this test.
Message-Id: <20230721164346.10112-1-thuth@redhat.com>
Reviewed-by: Juan Quintela <quintela@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Add a small test to prevent regressions.
Tested-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
Message-Id: <20230724082032.66864-15-iii@linux.ibm.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Add a small test to prevent regressions.
Tested-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
Message-Id: <20230724082032.66864-14-iii@linux.ibm.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Add a small test to prevent regressions.
Tested-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
Message-Id: <20230724082032.66864-13-iii@linux.ibm.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Add a small test to prevent regressions.
Tested-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
Message-Id: <20230724082032.66864-12-iii@linux.ibm.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Add a small test to prevent regressions.
Tested-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
Message-Id: <20230724082032.66864-11-iii@linux.ibm.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Add a small test to prevent regressions.
Tested-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
Message-Id: <20230724082032.66864-10-iii@linux.ibm.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Add a small test to prevent regressions.
Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
Message-Id: <20230724082032.66864-9-iii@linux.ibm.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Type 13 is reserved, so using it should result in specification
exception. Due to an off-by-1 error the code triggers an assertion at a
later point in time instead.
Cc: qemu-stable@nongnu.org
Fixes: da4807527f ("s390x/tcg: Implement VECTOR FP (MAXIMUM|MINIMUM)")
Reviewed-by: David Hildenbrand <david@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
Message-Id: <20230724082032.66864-8-iii@linux.ibm.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
MC requires bit positions 8-11 (upper 4 bits of class) to be zeros,
otherwise it must raise a specification exception.
Cc: qemu-stable@nongnu.org
Fixes: 20d143e2ca ("s390x/tcg: Implement MONITOR CALL")
Reviewed-by: David Hildenbrand <david@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
Message-Id: <20230724082032.66864-6-iii@linux.ibm.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
When the mask is zero, access exceptions should still be recognized for
1 byte at the second-operand address. CC should be set to 0.
Cc: qemu-stable@nongnu.org
Fixes: e023e832d0 ("s390x: translate engine for s390x CPU")
Reviewed-by: David Hildenbrand <david@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
Message-Id: <20230724082032.66864-5-iii@linux.ibm.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
CONVERT TO LOGICAL/FIXED deviate from IEEE 754 in that they raise an
inexact exception on out-of-range inputs. float_flag_invalid_cvti
aligns nicely with that behavior, so convert it to
S390_IEEE_MASK_INEXACT.
Cc: qemu-stable@nongnu.org
Fixes: defb0e3157 ("s390x: Implement opcode helpers")
Reviewed-by: David Hildenbrand <david@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
Message-Id: <20230724082032.66864-4-iii@linux.ibm.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
When the mask is zero, access exceptions should still be recognized for
1 byte at the second-operand address. CC should be set to 0.
Cc: qemu-stable@nongnu.org
Fixes: defb0e3157 ("s390x: Implement opcode helpers")
Reviewed-by: David Hildenbrand <david@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
Message-Id: <20230724082032.66864-3-iii@linux.ibm.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
R2 designates an even-odd register pair; the instruction should raise
a specification exception when R2 is not even.
Cc: qemu-stable@nongnu.org
Fixes: e023e832d0 ("s390x: translate engine for s390x CPU")
Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
Message-Id: <20230724082032.66864-2-iii@linux.ibm.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: David Hildenbrand <david@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
accel/tcg: Fix type of 'last' for pageflags_{find,next}
accel/tcg: Fix sense of read-only probes in ldst_atomicity
accel/tcg: Take mmap_lock in load_atomic*_or_exit
tcg: Add earlyclobber to op_add2 for x86 and s390x
tcg/ppc: Fix race in goto_tb implementation
-----BEGIN PGP SIGNATURE-----
iQFRBAABCgA7FiEEekgeeIaLTbaoWgXAZN846K9+IV8FAmS+O7cdHHJpY2hhcmQu
aGVuZGVyc29uQGxpbmFyby5vcmcACgkQZN846K9+IV8qrAf/VeAFnMbtantUTfM5
zOcfBlutsDlJrNwA/ajFDrPwUDewP7s5cqxImAYqhXfhqlc2RIB3UiMCgSaQ+q6O
MBOH0bEj/zbeIlwRX07ZBWhUYVdqJVd7Nxb1W19YwgG9yieWUxa+Xo1i2fhyXMv+
20VOFB1dPnxYyUMrzh/bSiHE90JFZktO1WzV10FRD+IpnImY9R+YGdpGTpVzUhor
ReRHTkMKyYilY6EEUG2gFhotrY/bbSSSFyl9BcQjkZh11603nAN0mNKxtSjPJnNB
rXhCVEgmbbBvCufsO6szQ03W/7RZ/KCg/DyKqxyCP1Ril4BIOx3tiucROcapXH/K
0y/ycA==
=hdk/
-----END PGP SIGNATURE-----
Merge tag 'pull-tcg-20230724' of https://gitlab.com/rth7680/qemu into staging
accel/tcg: Zero-pad vaddr in tlb debug output
accel/tcg: Fix type of 'last' for pageflags_{find,next}
accel/tcg: Fix sense of read-only probes in ldst_atomicity
accel/tcg: Take mmap_lock in load_atomic*_or_exit
tcg: Add earlyclobber to op_add2 for x86 and s390x
tcg/ppc: Fix race in goto_tb implementation
# -----BEGIN PGP SIGNATURE-----
#
# iQFRBAABCgA7FiEEekgeeIaLTbaoWgXAZN846K9+IV8FAmS+O7cdHHJpY2hhcmQu
# aGVuZGVyc29uQGxpbmFyby5vcmcACgkQZN846K9+IV8qrAf/VeAFnMbtantUTfM5
# zOcfBlutsDlJrNwA/ajFDrPwUDewP7s5cqxImAYqhXfhqlc2RIB3UiMCgSaQ+q6O
# MBOH0bEj/zbeIlwRX07ZBWhUYVdqJVd7Nxb1W19YwgG9yieWUxa+Xo1i2fhyXMv+
# 20VOFB1dPnxYyUMrzh/bSiHE90JFZktO1WzV10FRD+IpnImY9R+YGdpGTpVzUhor
# ReRHTkMKyYilY6EEUG2gFhotrY/bbSSSFyl9BcQjkZh11603nAN0mNKxtSjPJnNB
# rXhCVEgmbbBvCufsO6szQ03W/7RZ/KCg/DyKqxyCP1Ril4BIOx3tiucROcapXH/K
# 0y/ycA==
# =hdk/
# -----END PGP SIGNATURE-----
# gpg: Signature made Mon 24 Jul 2023 09:52:07 BST
# gpg: using RSA key 7A481E78868B4DB6A85A05C064DF38E8AF7E215F
# gpg: issuer "richard.henderson@linaro.org"
# gpg: Good signature from "Richard Henderson <richard.henderson@linaro.org>" [full]
# Primary key fingerprint: 7A48 1E78 868B 4DB6 A85A 05C0 64DF 38E8 AF7E 215F
* tag 'pull-tcg-20230724' of https://gitlab.com/rth7680/qemu:
accel/tcg: Fix type of 'last' for pageflags_{find,next}
accel/tcg: Zero-pad vaddr in tlb_debug output
tcg/{i386, s390x}: Add earlyclobber to the op_add2's first output
accel/tcg: Take mmap_lock in load_atomic*_or_exit
accel/tcg: Fix sense of read-only probes in ldst_atomicity
include/exec: Add WITH_MMAP_LOCK_GUARD
tcg/ppc: Fix race in goto_tb implementation
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
These should match 'start' as target_ulong, not target_long.
On 32bit targets, the parameter was sign-extended to uint64_t,
so only the first mmap within the upper 2GB memory can succeed.
Signed-off-by: Luca Bonissi <qemu@bonslack.org>
Message-Id: <327460e2-0ebd-9edb-426b-1df80d16c32a@bonslack.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
In replacing target_ulong with vaddr and TARGET_FMT_lx with VADDR_PRIx,
the zero-padding of TARGET_FMT_lx got lost. Readd 16-wide zero-padding
for logging consistency.
Suggested-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Anton Johansson <anjo@rev.ng>
Message-Id: <20230713120746.26897-1-anjo@rev.ng>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
i386 and s390x implementations of op_add2 require an earlyclobber,
which is currently missing. This breaks VCKSM in s390x guests. E.g., on
x86_64 the following op:
add2_i32 tmp2,tmp3,tmp2,tmp3,tmp3,tmp2 dead: 0 2 3 4 5 pref=none,0xffff
is translated to:
addl %ebx, %r12d
adcl %r12d, %ebx
Introduce a new C_N1_O1_I4 constraint, and make sure that earlyclobber
of aliased outputs is honored.
Cc: qemu-stable@nongnu.org
Fixes: 82790a8709 ("tcg: Add markup for output requires new register")
Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20230719221310.1968845-7-iii@linux.ibm.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
For user-only, the probe for page writability may race with another
thread's mprotect. Take the mmap_lock around the operation. This
is still faster than the start/end_exclusive fallback.
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
In the initial commit, cdfac37be0, the sense of the test is incorrect,
as the -1/0 return was confusing. In bef6f008b9, we mechanically
invert all callers while changing to false/true return, preserving the
incorrectness of the test.
Now that the return sense is sane, it's easy to see that if !write,
then the page is not modifiable (i.e. most likely read-only, with
PROT_NONE handled via SIGSEGV).
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Commit 20b6643324 ("tcg/ppc: Reorg goto_tb implementation") modified
goto_tb to ensure only a single instruction was patched to prevent
incorrect behavior if a thread was in the middle of multiple
instructions when they were replaced. However this introduced a race
between loading the jmp target into TCG_REG_TB and patching and
executing the direct branch.
The relevant part of the goto_tb implementation:
ld TCG_REG_TB, TARGET_ADDR_LOCATION(TCG_REG_TB)
patch_location:
mtctr TCG_REG_TB
bctr
tb_target_set_jmp_target() will replace 'patch_location' with a direct
branch if the target is in range. The direct branch now relies on
TCG_REG_TB being set up correctly by the ld. Prior to this commit
multiple instructions were patched in for the direct branch case; these
instructions would initialize TCG_REG_TB to the same value as the branch
target.
Imagine the following sequence:
1) Thread A is executing the goto_tb sequence and loads the jmp
target into TCG_REG_TB.
2) Thread B updates the jmp target address and calls
tb_target_set_jmp_target(). This patches a new direct branch into the
goto_tb sequence.
3) Thread A executes the newly patched direct branch. The value in
TCG_REG_TB still contains the old jmp target.
TCG_REG_TB MUST contain the translation block's tc.ptr. Execution will
eventually crash after performing memory accesses generated from a
faulty value in TCG_REG_TB.
This presents as segfaults or illegal instruction exceptions.
Do not revert commit 20b6643324 as it did fix a different race
condition. Instead remove the direct branch optimization and always use
indirect branches.
The direct branch optimization can be re-added later with a race free
sequence.
Fixes: 20b6643324 ("tcg/ppc: Reorg goto_tb implementation")
Resolves: https://gitlab.com/qemu-project/qemu/-/issues/1726
Reported-by: Anushree Mathur <anushree.mathur@linux.vnet.ibm.com>
Tested-by: Anushree Mathur <anushree.mathur@linux.vnet.ibm.com>
Tested-by: Michael Tokarev <mjt@tls.msk.ru>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Co-developed-by: Benjamin Gray <bgray@linux.ibm.com>
Signed-off-by: Jordan Niethe <jniethe5@gmail.com>
Signed-off-by: Benjamin Gray <bgray@linux.ibm.com>
Message-Id: <20230717093001.13167-1-jniethe5@gmail.com>
Upgrade OpenSBI from v1.3 to v1.3.1 and the pre-built bios images
which fixes the boot failure seen when using QEMU to do a direct
kernel boot with Microchip Icicle Kit board machine.
The v1.3.1 release includes the following commits:
0907de3 lib: sbi: fix comment indent
eb736a5 lib: sbi_pmu: Avoid out of bounds access
7828eeb gpio/desginware: add Synopsys DesignWare APB GPIO support
c6a3573 lib: utils: Fix sbi_hartid_to_scratch() usage in ACLINT drivers
057eb10 lib: utils/gpio: Fix RV32 compile error for designware GPIO driver
Signed-off-by: Bin Meng <bmeng@tinylab.org>
Message-Id: <20230719165817.889465-1-bmeng@tinylab.org>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Tested-by: Conor Dooley <conor.dooley@microchip.com>
Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
This reverts commit 518f32221a.
It is causing similar segfaults at least on aarch64, ppc64el
and s390x. Let's revert this one for now and analyze what's
going on later.
Reopens: https://bugs.debian.org/1040981
Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>
- Denis V. Lunev: fix hang with 'ssh ... "qemu-nbd -c"'
- Eric Blake: preliminary work towards NBD 64-bit extensions
-----BEGIN PGP SIGNATURE-----
iQEzBAABCAAdFiEEccLMIrHEYCkn0vOqp6FrSiUnQ2oFAmS4RwcACgkQp6FrSiUn
Q2pXfQf/clnttPdw9BW2cJltFRKeMeZrgn8mut0S7jhC0DWIy6zanzp07MylryHP
EyJ++dCbLEg8mueThL/n5mKsTS/OECtfZO9Ot11WmZqDZVtLKorfmy7YVI3VwMjI
yQqrUIwiYxzZOkPban/MXofY6vJmuia5aGkEmYUyKiHvsLF3Hk2gHPB/qa2S+U6I
QDmC032/L+/LgVkK5r/1vamwJNP29QI4DNp3RiTtcMK5sEZJfMsAZSxFDDdH2pqi
5gyVqw0zNl3vz6znoVy0XZ/8OUVloPKHswyf7xLlBukY1GL5D+aiXz2ilwBvk9aM
SoZzYvaOOBDyJhSjapOvseTqXTNeqQ==
=TB9t
-----END PGP SIGNATURE-----
Merge tag 'pull-nbd-2023-07-19' of https://repo.or.cz/qemu/ericb into staging
NBD patches through 2023-07-19
- Denis V. Lunev: fix hang with 'ssh ... "qemu-nbd -c"'
- Eric Blake: preliminary work towards NBD 64-bit extensions
# -----BEGIN PGP SIGNATURE-----
#
# iQEzBAABCAAdFiEEccLMIrHEYCkn0vOqp6FrSiUnQ2oFAmS4RwcACgkQp6FrSiUn
# Q2pXfQf/clnttPdw9BW2cJltFRKeMeZrgn8mut0S7jhC0DWIy6zanzp07MylryHP
# EyJ++dCbLEg8mueThL/n5mKsTS/OECtfZO9Ot11WmZqDZVtLKorfmy7YVI3VwMjI
# yQqrUIwiYxzZOkPban/MXofY6vJmuia5aGkEmYUyKiHvsLF3Hk2gHPB/qa2S+U6I
# QDmC032/L+/LgVkK5r/1vamwJNP29QI4DNp3RiTtcMK5sEZJfMsAZSxFDDdH2pqi
# 5gyVqw0zNl3vz6znoVy0XZ/8OUVloPKHswyf7xLlBukY1GL5D+aiXz2ilwBvk9aM
# SoZzYvaOOBDyJhSjapOvseTqXTNeqQ==
# =TB9t
# -----END PGP SIGNATURE-----
# gpg: Signature made Wed 19 Jul 2023 21:26:47 BST
# gpg: using RSA key 71C2CC22B1C4602927D2F3AAA7A16B4A2527436A
# gpg: Good signature from "Eric Blake <eblake@redhat.com>" [full]
# gpg: aka "Eric Blake (Free Software Programmer) <ebb9@byu.net>" [full]
# gpg: aka "[jpeg image of size 6874]" [full]
# Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2 F3AA A7A1 6B4A 2527 436A
* tag 'pull-nbd-2023-07-19' of https://repo.or.cz/qemu/ericb:
nbd: Use enum for various negotiation modes
nbd/client: Add safety check on chunk payload length
nbd/client: Simplify cookie vs. index computation
nbd: s/handle/cookie/ to match NBD spec
nbd/server: Refactor to pass full request around
nbd/server: Prepare for alternate-size headers
nbd: Consistent typedef usage in header
nbd/client: Use smarter assert
qemu-nbd: make verbose bool and local variable in main()
qemu-nbd: handle dup2() error when qemu-nbd finished setup process
qemu-nbd: properly report error on error in dup2() after qemu_daemon()
qemu-nbd: properly report error if qemu_daemon() is failed
qemu-nbd: fix regression with qemu-nbd --fork run over ssh
qemu-nbd: pass structure into nbd_client_thread instead of plain char*
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Commit 86f04735ac ("linux-user: Fix brk() to release pages") introduced
the possibility for userspace applications to reduce memory footprint by
calling brk() with a lower address and as such free up memory, the same
way as the Linux kernel allows on physical machines.
This change introduced some failures for applications with errors like
- accesing bytes above the brk heap address on the same page,
- freeing memory below the initial brk address,
and introduced a behaviour which isn't done by the kernel (e.g. zeroing
memory above brk).
This patch series fixes those issues and has been tested with existing
programs (e.g. upx).
Additionally one patch fixes running static armhf executables (e.g. fstype)
which was broken since qemu-8.0.
Changes in v2:
- dropped patch to revert d28b3c90cf ("linux-user: Make sure initial brk(0)
is page-aligned")
- rephrased some commit messages
- fixed Cc email addresses, added new ones
- added R-b tags
Helge
-----BEGIN PGP SIGNATURE-----
iHUEABYKAB0WIQS86RI+GtKfB8BJu973ErUQojoPXwUCZLgGswAKCRD3ErUQojoP
XwkUAQCKb/lkI3IYxiqO48rVyHtLPtkXd+WttFpeZ076p73LTgD+IEpHZL4WV1Rw
4+eqW9vswjZwp1xm9bItLdnP2hkyUgI=
=K3Va
-----END PGP SIGNATURE-----
Merge tag 'linux-user-brk-fixes-pull-request' of https://github.com/hdeller/qemu-hppa into staging
linux-user: brk() syscall fixes and armhf static binary fix
Commit 86f04735ac ("linux-user: Fix brk() to release pages") introduced
the possibility for userspace applications to reduce memory footprint by
calling brk() with a lower address and as such free up memory, the same
way as the Linux kernel allows on physical machines.
This change introduced some failures for applications with errors like
- accesing bytes above the brk heap address on the same page,
- freeing memory below the initial brk address,
and introduced a behaviour which isn't done by the kernel (e.g. zeroing
memory above brk).
This patch series fixes those issues and has been tested with existing
programs (e.g. upx).
Additionally one patch fixes running static armhf executables (e.g. fstype)
which was broken since qemu-8.0.
Changes in v2:
- dropped patch to revert d28b3c90cf ("linux-user: Make sure initial brk(0)
is page-aligned")
- rephrased some commit messages
- fixed Cc email addresses, added new ones
- added R-b tags
Helge
# -----BEGIN PGP SIGNATURE-----
#
# iHUEABYKAB0WIQS86RI+GtKfB8BJu973ErUQojoPXwUCZLgGswAKCRD3ErUQojoP
# XwkUAQCKb/lkI3IYxiqO48rVyHtLPtkXd+WttFpeZ076p73LTgD+IEpHZL4WV1Rw
# 4+eqW9vswjZwp1xm9bItLdnP2hkyUgI=
# =K3Va
# -----END PGP SIGNATURE-----
# gpg: Signature made Wed 19 Jul 2023 16:52:19 BST
# gpg: using EDDSA key BCE9123E1AD29F07C049BBDEF712B510A23A0F5F
# gpg: Good signature from "Helge Deller <deller@gmx.de>" [unknown]
# gpg: aka "Helge Deller <deller@kernel.org>" [unknown]
# gpg: WARNING: This key is not certified with a trusted signature!
# gpg: There is no indication that the signature belongs to the owner.
# Primary key fingerprint: 4544 8228 2CD9 10DB EF3D 25F8 3E5F 3D04 A7A2 4603
# Subkey fingerprint: BCE9 123E 1AD2 9F07 C049 BBDE F712 B510 A23A 0F5F
* tag 'linux-user-brk-fixes-pull-request' of https://github.com/hdeller/qemu-hppa:
linux-user: Fix qemu-arm to run static armhf binaries
linux-user: Fix strace output for old_mmap
linux-user: Fix signed math overflow in brk() syscall
linux-user: Prohibit brk() to to shrink below initial heap address
linux-user: Fix qemu brk() to not zero bytes on current page
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Deciphering the hard-coded list of integer return values from
nbd_start_negotiate() will only get more confusing when adding support
for 64-bit extended headers. Better is to name things in an enum.
Although the function in question is private to client.c, putting the
enum in a public header and including an enum-to-string conversion
will allow its use in more places in upcoming patches.
The enum is intentionally laid out so that operators like <= can be
used to group multiple modes with similar characteristics, and where
the least powerful mode has value 0, even though this patch does not
exploit that. No semantic change intended.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-ID: <20230608135653.2918540-9-eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Our existing use of structured replies either reads into a qiov capped
at 32M (NBD_CMD_READ) or caps allocation to 1000 bytes (see
NBD_MAX_MALLOC_PAYLOAD in block/nbd.c). But the existing length
checks are rather late; if we encounter a buggy (or malicious) server
that sends a super-large payload length, we should drop the connection
right then rather than assuming the layer on top will be careful.
This becomes more important when we permit 64-bit lengths which are
even more likely to have the potential for attempted denial of service
abuse.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Message-ID: <20230608135653.2918540-8-eblake@redhat.com>
Our code relies on a sentinel cookie value of zero for deciding when a
packet has been handled, as well as relying on array indices between 0
and MAX_NBD_REQUESTS-1 for dereferencing purposes. As long as we can
symmetrically convert between two forms, there is no reason to go with
the odd choice of using XOR with a random pointer, when we can instead
simplify the mappings with a mere offset of 1.
Using ((uint64_t)-1) as the sentinel instead of NULL such that the two
macros could be entirely eliminated might also be possible, but would
require a more careful audit to find places where we currently rely on
zero-initialization to be interpreted as the sentinel value, so I did
not pursue that course.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-ID: <20230608135653.2918540-7-eblake@redhat.com>
[eblake: enhance commit message]
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Externally, libnbd exposed the 64-bit opaque marker for each client
NBD packet as the "cookie", because it was less confusing when
contrasted with 'struct nbd_handle *' holding all libnbd state. It
also avoids confusion between the noun 'handle' as a way to identify a
packet and the verb 'handle' for reacting to things like signals.
Upstream NBD changed their spec to favor the name "cookie" based on
libnbd's recommendations[1], so we can do likewise.
[1] https://github.com/NetworkBlockDevice/nbd/commit/ca4392eb2b
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-ID: <20230608135653.2918540-6-eblake@redhat.com>
[eblake: typo fix]
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Part of NBD's 64-bit headers extension involves passing the client's
requested offset back as part of the reply header (one reason it
stated for this change: converting absolute offsets stored in
NBD_REPLY_TYPE_OFFSET_DATA to relative offsets within the buffer is
easier if the absolute offset of the buffer is also available). This
is a refactoring patch to pass the full request around the reply
stack, rather than just the handle, so that later patches can then
access request->from when extended headers are active. Meanwhile,
this patch enables us to now assert that simple replies are only
attempted when appropriate, and otherwise has no semantic change.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Message-ID: <20230608135653.2918540-5-eblake@redhat.com>
Upstream NBD now documents[1] an extension that supports 64-bit effect
lengths in requests. As part of that extension, the size of the reply
headers will change in order to permit a 64-bit length in the reply
for symmetry[2]. Additionally, where the reply header is currently 16
bytes for simple reply, and 20 bytes for structured reply; with the
extension enabled, there will only be one extended reply header, of 32
bytes, with both structured and extended modes sending identical
payloads for chunked replies.
Since we are already wired up to use iovecs, it is easiest to allow
for this change in header size by splitting each structured reply
across multiple iovecs, one for the header (which will become wider in
a future patch according to client negotiation), and the other(s) for
the chunk payload, and removing the header from the payload struct
definitions. Rename the affected functions with s/structured/chunk/
to make it obvious that the code will be reused in extended mode.
Interestingly, the client side code never utilized the packed types,
so only the server code needs to be updated.
[1] https://github.com/NetworkBlockDevice/nbd/blob/extension-ext-header/doc/proto.md
as of NBD commit e6f3b94a934
[2] Note that on the surface, this is because some future server might
permit a 4G+ NBD_CMD_READ and need to reply with that much data in one
transaction. But even though the extended reply length is widened to
64 bits, for now the NBD spec is clear that servers will not reply
with more than a maximum payload bounded by the 32-bit
NBD_INFO_BLOCK_SIZE field; allowing a client and server to mutually
agree to transactions larger than 4G would require yet another
extension.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-ID: <20230608135653.2918540-4-eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
We had a mix of struct declarations followed by typedefs, and direct
struct definitions as part of a typedef. Pick a single style. Also
float forward declarations of opaque types to the top of the file,
rather than interspersed with function declarations, which will help a
future patch that wants to expose yet another opaque type that will be
referenced in NBDRequest. No semantic impact.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-ID: <20230608135653.2918540-3-eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
[eblake: alter patch per mailing list feedback]
Signed-off-by: Eric Blake <eblake@redhat.com>
Assigning strlen() to a uint32_t and then asserting that it isn't too
large doesn't catch the case of an input string 4G in length.
Thankfully, the incoming strings can never be that large: if the
export name or query is reflecting a string the client got from the
server, we already guarantee that we dropped the NBD connection if the
server sent more than 32M in a single reply to our NBD_OPT_* request;
if the export name is coming from qemu, nbd_receive_negotiate()
asserted that strlen(info->name) <= NBD_MAX_STRING_SIZE; and
similarly, a query string via x->dirty_bitmap coming from the user was
bounds-checked in either qemu-nbd or by the limitations of QMP.
Still, it doesn't hurt to be more explicit in how we write our
assertions to not have to analyze whether inadvertent wraparound is
possible.
Fixes: 93676c88 ("nbd: Don't send oversize strings", v4.2.0)
Reported-by: Dr. David Alan Gilbert <dave@treblig.org>
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Message-ID: <20230608135653.2918540-2-eblake@redhat.com>
Pass 'verbose' to nbd_client_thread() inside NbdClientOpts which looks
a little bit cleaner and make it bool as it is used as bool actually.
Signed-off-by: Denis V. Lunev <den@openvz.org>
CC: Eric Blake <eblake@redhat.com>
CC: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Message-ID: <20230717202520.236999-1-den@openvz.org>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
Fail on error, we are in trouble.
Signed-off-by: Denis V. Lunev <den@openvz.org>
CC: Eric Blake <eblake@redhat.com>
CC: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Message-ID: <20230717145544.194786-6-den@openvz.org>
Reviewed-by: Eric Blake <eblake@redhat.com>
[eblake: avoid intermediate variable]
Signed-off-by: Eric Blake <eblake@redhat.com>
We are trying to temporarily redirect stderr of daemonized process to
a pipe to report a error and get failed. In that case we could not
use error_report() helper, but should write the message directly into
the problematic pipe.
Signed-off-by: Denis V. Lunev <den@openvz.org>
CC: Eric Blake <eblake@redhat.com>
CC: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Message-ID: <20230717145544.194786-4-den@openvz.org>
Reviewed-by: Eric Blake <eblake@redhat.com>
[eblake: rearrange patch series, fix typo]
Signed-off-by: Eric Blake <eblake@redhat.com>
errno has been overwritten by dup2() just below qemu_daemon() and thus
improperly returned to the caller. Fix accordingly.
Signed-off-by: Denis V. Lunev <den@openvz.org>
CC: Eric Blake <eblake@redhat.com>
CC: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Message-ID: <20230717145544.194786-5-den@openvz.org>
Reviewed-by: Eric Blake <eblake@redhat.com>
[eblake: reorder patch series]
Signed-off-by: Eric Blake <eblake@redhat.com>
Commit e6df58a557
Author: Hanna Reitz <hreitz@redhat.com>
Date: Wed May 8 23:18:18 2019 +0200
qemu-nbd: Do not close stderr
has introduced an interesting regression. Original behavior of
ssh somehost qemu-nbd /home/den/tmp/file -f raw --fork
was the following:
* qemu-nbd was started as a daemon
* the command execution is done and ssh exited with success
The patch has changed this behavior and 'ssh' command now hangs forever.
According to the normal specification of the daemon() call, we should
endup with STDERR pointing to /dev/null. That should be done at the
very end of the successful startup sequence when the pipe to the
bootstrap process (used for diagnostics) is no longer needed.
This could be achived in the same way as done for 'qemu-nbd -c' case.
That was commit 0eaf453e, also fixing up e6df58a5. STDOUT copying to
STDERR does the trick.
This also leads to proper 'ssh' connection closing which fixes my
original problem.
Signed-off-by: Denis V. Lunev <den@openvz.org>
CC: Eric Blake <eblake@redhat.com>
CC: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
CC: Hanna Reitz <hreitz@redhat.com>
CC: <qemu-stable@nongnu.org>
Message-ID: <20230717145544.194786-3-den@openvz.org>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
