In preparation for compiling tcg/ only once, eliminate
the all_helpers array. Instantiate the info structs for
the generic helpers in accel/tcg/, and the structs for
the target-specific helpers in each translate.c.
Since we don't see all of the info structs at startup,
initialize at first use, using g_once_init_* to make
sure we don't race while doing so.
Reviewed-by: Anton Johansson <anjo@rev.ng>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Create both atomic16_read_ro and atomic16_read_rw.
Previously we pretended that we had atomic16_read in system mode,
because we "know" that all ram is always writable to the host.
Now, expose read-only and read-write versions all of the time.
For aarch64, do not fall back to __atomic_read_16 even if
supported by the compiler, to work around a clang bug.
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Eliminate the CONFIG_USER_ONLY specialization.
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: David Hildenbrand <david@redhat.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Use cpu_ld16_mmu and cpu_st16_mmu to eliminate the special case,
and change all of the *_data_ra functions to match.
Note that we check the alignment of both compare and store
pointers at the top of the function, so MO_ALIGN* may be
safely removed from the individual memory operations.
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: David Hildenbrand <david@redhat.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
No need to roll our own, as this is now provided by tcg.
This was the last use of retxl, so remove that too.
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: David Hildenbrand <david@redhat.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Fix a problem similar to the one fixed by commit 703d03a4aa
("target/s390x: Fix EXECUTE of relative long instructions"), but now
for relative branches.
Reported-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20230426235813.198183-2-iii@linux.ibm.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
It's RRE, not RXE.
Found by running valgrind's none/tests/s390x/bfp-2.
Fixes: 86b59624c4 ("s390x/tcg: Implement LOAD LENGTHENED short HFP to long HFP")
Reviewed-by: David Hildenbrand <david@redhat.com>
Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
Message-Id: <20230511134726.469651-1-iii@linux.ibm.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Convert away from the old interface with the implicit
MemOp argument.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: David Hildenbrand <david@redhat.com>
Reviewed-by: Ilya Leoshkevich <iii@linux.ibm.com>
Message-Id: <20230502135741.1158035-7-richard.henderson@linaro.org>
Both cpu_check_watchpoint() and cpu_watchpoint_address_matches()
are specific to TCG system emulation. Declare them in "tcg-cpu-ops.h"
to be sure accessing them from non-TCG code is a compilation error.
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Message-Id: <20230328173117.15226-2-philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
The "assert(!nonfault)" statement can be triggered by running the
"mvpg" s390x kvm-unit-test with TCG. According to Richard: "... the
assert looks backward. We should only arrive there if nonfault was
true for the probe (otherwise the probe would have raised the
exception directly). I would think we could just remove the assert."
Fixes: 4049431478 ("target/s390x: Fix s390_probe_access for user-only")
Suggested-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20230317135737.597570-1-thuth@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: David Hildenbrand <david@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Relative long instructions now depend on do_unaligned_access() too.
Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20230316164428.275147-12-iii@linux.ibm.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Generate a specification exception in the helper before trying to fetch
the instruction.
Reported-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20230316164428.275147-3-iii@linux.ibm.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
The code uses the wrong base for relative addressing: it should use the
target instruction address and not the EXECUTE's address.
Fix by storing the target instruction address in the new CPUS390XState
member and loading it from the code generated by gen_ri2().
Reported-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: David Hildenbrand <david@redhat.com>
Message-Id: <20230316210751.302423-2-iii@linux.ibm.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
RXSBG usage in the "filetests" test from the wasmtime testsuite makes
tcg_reg_alloc_op() attempt to temp_load() a TEMP_VAL_DEAD temporary,
causing an assertion failure:
0x01000a70: ec14 b040 3057 rxsbg %r1, %r4, 0xb0, 0x40, 0x30
OP after optimization and liveness analysis:
---- 0000000001000a70 0000000000000004 0000000000000006
rotl_i64 tmp2,r4,$0x30 dead: 1 2 pref=0xffff
and_i64 tmp2,tmp2,$0x800000000000ffff dead: 1 pref=0xffff
[xor_i64 tmp3,tmp3,tmp2 dead: 1 2 pref=0xffff]
and_i64 cc_dst,tmp3,$0x800000000000ffff sync: 0 dead: 0 1 2 pref=0xffff
mov_i64 psw_addr,$0x1000a76 sync: 0 dead: 0 1 pref=0xffff
mov_i32 cc_op,$0x6 sync: 0 dead: 0 1 pref=0xffff
call lookup_tb_ptr,$0x6,$1,tmp8,env dead: 1 pref=none
goto_ptr tmp8 dead: 0
set_label $L0
exit_tb $0x7fffe809d183
../tcg/tcg.c:3865: tcg fatal error
The reason is that tmp3 does not have an initial value, which confuses
the register allocator. This also affects the correctness of the
results.
Fix by assigning R1 to it.
Exposed by commit e2e641fa3d ("tcg: Change default temp lifetime to
TEMP_TB").
Fixes: d6c6372e18 ("target-s390: Implement R[NOX]SBG")
Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
Reviewed-by: David Hildenbrand <david@redhat.com>
Message-Id: <20230316172205.281369-2-iii@linux.ibm.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Generate a specification exception if a reserved bit is set in the PSW
mask or if the PSW address is out of bounds dictated by the addressing
mode.
Reported-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
Reviewed-by: David Hildenbrand <david@redhat.com>
Message-Id: <20230315020408.384766-3-iii@linux.ibm.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Currently LPSW does not invert the mask bit 12 and incorrectly copies
the BA bit into the address.
Fix by generating code similar to what s390_cpu_load_normal() does.
Reported-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
Co-developed-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
Signed-off-by: Ilya Leoshkevich <iii@linux.ibm.com>
Reviewed-by: David Hildenbrand <david@redhat.com>
Message-Id: <20230315020408.384766-2-iii@linux.ibm.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
All uses are strictly read-only.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
These fields are no longer read, so remove them and the writes.
Acked-by: David Hildenbrand <david@redhat.com>
Reviewed-by: Ilya Leoshkevich <iii@linux.ibm.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
These were trying to determine if o->in2 was available for
use as a temporary. It's better to just allocate a new one.
Acked-by: David Hildenbrand <david@redhat.com>
Reviewed-by: Ilya Leoshkevich <iii@linux.ibm.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Translators are no longer required to free tcg temporaries.
Acked-by: David Hildenbrand <david@redhat.com>
Reviewed-by: Ilya Leoshkevich <iii@linux.ibm.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Translators are no longer required to free tcg temporaries.
Acked-by: David Hildenbrand <david@redhat.com>
Reviewed-by: Ilya Leoshkevich <iii@linux.ibm.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Translators are no longer required to free tcg temporaries.
Remove the g1 and g2 members of DisasCompare, as they were
used to track which temps needed to be freed.
Acked-by: David Hildenbrand <david@redhat.com>
Reviewed-by: Ilya Leoshkevich <iii@linux.ibm.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
The second operand of COMPARE HALFWORD RELATIVE LONG is a signed
halfword, it does not have the same size as the first operand.
Fixes: a7e836d5eb ("target-s390: Convert COMPARE, COMPARE LOGICAL")
Signed-off-by: Nina Schoetterl-Glausch <nsg@linux.ibm.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: David Hildenbrand <david@redhat.com>
Message-Id: <20230310114157.3024170-2-nsg@linux.ibm.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Use tcg_constant_i64. Adjust in2_mri2_* to allocate a new
temporary for the output, using gen_ri2 for the address.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
In preparation for returning the number of insns generated
via the same pointer. Adjust only the prototypes so far.
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
probe_access_flags() as it is today uses probe_access_full(), which in
turn uses probe_access_internal() with size = 0. probe_access_internal()
then uses the size to call the tlb_fill() callback for the given CPU.
This size param ('fault_size' as probe_access_internal() calls it) is
ignored by most existing .tlb_fill callback implementations, e.g.
arm_cpu_tlb_fill(), ppc_cpu_tlb_fill(), x86_cpu_tlb_fill() and
mips_cpu_tlb_fill() to name a few.
But RISC-V riscv_cpu_tlb_fill() actually uses it. The 'size' parameter
is used to check for PMP (Physical Memory Protection) access. This is
necessary because PMP does not make any guarantees about all the bytes
of the same page having the same permissions, i.e. the same page can
have different PMP properties, so we're forced to make sub-page range
checks. To allow RISC-V emulation to do a probe_acess_flags() that
covers PMP, we need to either add a 'size' param to the existing
probe_acess_flags() or create a new interface (e.g.
probe_access_range_flags).
There are quite a few probe_* APIs already, so let's add a 'size' param
to probe_access_flags() and re-use this API. This is done by open coding
what probe_access_full() does inside probe_acess_flags() and passing the
'size' param to probe_acess_internal(). Existing probe_access_flags()
callers use size = 0 to not change their current API usage. 'size' is
asserted to enforce single page access like probe_access() already does.
No behavioral changes intended.
Signed-off-by: Daniel Henrique Barboza <dbarboza@ventanamicro.com>
Message-Id: <20230223234427.521114-2-dbarboza@ventanamicro.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
In most cases, this is a simple local allocate and free
replaced by tcg_constant_*. In three cases, a variable
temp was initialized with a constant value -- reorg to
localize the constant. In gen_acc, this fixes a leak.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Ilya Leoshkevich <iii@linux.ibm.com>
Message-Id: <20230220184052.163465-5-richard.henderson@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Return a constant or NULL, which means the free may be
removed from all callers of fpinst_extract_m34.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Ilya Leoshkevich <iii@linux.ibm.com>
Message-Id: <20230220184052.163465-4-richard.henderson@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
The a and b fields are not modified by the consumer,
and while we need not free a constant, tcg will quietly
ignore such frees, so free_compare need not be changed.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Ilya Leoshkevich <iii@linux.ibm.com>
Message-Id: <20230220184052.163465-3-richard.henderson@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Replace tcg_const_* with tcg_constant_* in contexts
where the free to remove is nearby.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: Ilya Leoshkevich <iii@linux.ibm.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20230220184052.163465-2-richard.henderson@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Ensure that the total length is in a local variable
across the byte loop. Compute size1 difference once.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: David Hildenbrand <david@redhat.com>
Message-Id: <20230109201856.3916639-8-richard.henderson@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Inline into the parent functions with a simple test
to select the page, and a new define to remove ifdefs.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: David Hildenbrand <david@redhat.com>
Message-Id: <20230109201856.3916639-7-richard.henderson@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
When this code was written, it was using tlb_vaddr_to_host,
which does not handle TLB_DIRTY. Since then, it has been
converted to probe_access_flags, which does.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Acked-by: David Hildenbrand <david@redhat.com>
Message-Id: <20230109201856.3916639-6-richard.henderson@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Assign to access struct immediately, rather than waiting
until the end of the function. This means we can pass
address of haddr struct members instead of allocating
extra space on the local stack.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: David Hildenbrand <david@redhat.com>
Message-Id: <20230109201856.3916639-5-richard.henderson@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
The interface from probe_access_flags is void*, and matching
that will be helpful. We already rely on the gcc extension
for byte arithmetic on void*.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: David Hildenbrand <david@redhat.com>
Message-Id: <20230109201856.3916639-4-richard.henderson@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Passing a pointer from the caller down to access_prepare_nf
eliminates a structure copy.
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: David Hildenbrand <david@redhat.com>
Message-Id: <20230109201856.3916639-3-richard.henderson@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
In db9aab5783 we broke the contract of s390_probe_access, in that it
no longer returned an exception code, nor set __excp_addr. Fix both.
Reported-by: David Hildenbrand <david@redhat.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: David Hildenbrand <david@redhat.com>
Message-Id: <20230109201856.3916639-2-richard.henderson@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
This case is trivial to implement inline.
Reviewed-by: David Hildenbrand <david@redhat.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
