mirrors/qemu
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
0aca736433
qemu/target/i386/tcg/decode-new.c.inc

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

2816 lines
111 KiB
PHP
Raw Normal View History

target/i386: add core of new i386 decoder The new decoder is based on three principles: - use mostly table-driven decoding, using tables derived as much as possible from the Intel manual. Centralizing the decode the operands makes it more homogeneous, for example all immediates are signed. All modrm handling is in one function, and can be shared between SSE and ALU instructions (including XMM<->GPR instructions). The SSE/AVX decoder will also not have duplicated code between the 0F, 0F38 and 0F3A tables. - keep the code as "non-branchy" as possible. Generally, the code for the new decoder is more verbose, but the control flow is simpler. Conditionals are not nested and have small bodies. All instruction groups are resolved even before operands are decoded, and code generation is separated as much as possible within small functions that only handle one instruction each. - keep address generation and (for ALU operands) memory loads and writeback as much in common code as possible. All ALU operations for example are implemented as T0=f(T0,T1). For non-ALU instructions, read-modify-write memory operations are rare, but registers do not have TCGv equivalents: therefore, the common logic sets up pointer temporaries with the operands, while load and writeback are handled by gvec or by helpers. These principles make future code review and extensibility simpler, at the cost of having a relatively large amount of code in the form of this patch. Even EVEX should not be _too_ hard to implement (it's just a crazy large amount of possibilities). This patch introduces the main decoder flow, and integrates the old decoder with the new one. The old decoder takes care of parsing prefixes and then optionally drops to the new one. The changes to the old decoder are minimal and allow it to be replaced incrementally with the new one. There is a debugging mechanism through a "LIMIT" environment variable. In user-mode emulation, the variable is the number of instructions decoded by the new decoder before permanently switching to the old one. In system emulation, the variable is the highest opcode that is decoded by the new decoder (this is less friendly, but it's the best that can be done without requiring deterministic execution). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-23 12:20:55 +03:00
/*
* New-style decoder for i386 instructions
*
* Copyright (c) 2022 Red Hat, Inc.
*
* Author: Paolo Bonzini <pbonzini@redhat.com>
*
* This library is free software; you can redistribute it and/or
* modify it under the terms of the GNU Lesser General Public
* License as published by the Free Software Foundation; either
* version 2.1 of the License, or (at your option) any later version.
*
* This library is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public
* License along with this library; if not, see <http://www.gnu.org/licenses/>.
*/
/*
* The decoder is mostly based on tables copied from the Intel SDM. As
* a result, most operand load and writeback is done entirely in common
* table-driven code using the same operand type (X86_TYPE_*) and
target/i386: validate VEX.W for AVX instructions Instructions in VEX exception class 6 generally look at the value of VEX.W. Note that the manual places some instructions incorrectly in class 4, for example VPERMQ which has no non-VEX encoding and no legacy SSE analogue. AMD does a mess of its own, as documented in the comment that this patch adds. Most of them are checked for VEX.W=0, and are listed in the manual (though with an omission) in table 2-16; VPERMQ and VPERMPD check for VEX.W=1, which is only listed in the instruction description. Others, such as VPSRLV, VPSLLV and the FMA3 instructions, use VEX.W to switch between a 32-bit and 64-bit operation. Fix more of the class 4/class 6 mismatches, and implement the check for VEX.W in TCG. Acked-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-09 19:16:27 +03:00
* size (X86_SIZE_*) codes used in the manual. There are a few differences
* though.
*
target/i386: document more deviations from the manual Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-23 10:58:48 +03:00
* Operand sizes
* -------------
*
* The manual lists d64 ("cannot encode 32-bit size in 64-bit mode") and f64
* ("cannot encode 16-bit or 32-bit size in 64-bit mode") as modifiers of the
* "v" or "z" sizes. The decoder simply makes them separate operand sizes.
*
target/i386: move 60-BF opcodes to new decoder Compared to the old decoder, the main differences in translation are for the little-used ARPL instruction. IMUL is adjusted a bit to share more code to produce flags, but is otherwise very similar. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-11 11:55:16 +03:00
* The manual lists immediate far destinations as Ap (technically an implicit
* argument). The decoder splits them into two immediates, using "Ip" for
* the offset part (that comes first in the instruction stream) and "Iw" for
* the segment/selector part. The size of the offset is given by s->dflag
* and the instructions are illegal in 64-bit mode, so the choice of "Ip"
* is somewhat arbitrary; "Iv" or "Iz" would work just as well.
*
* Operand types
* -------------
*
target/i386: move C0-FF opcodes to new decoder (except for x87) The shift instructions are rewritten instead of reusing code from the old decoder. Rotates use CC_OP_ADCOX more extensively and generally rely more on the optimizer, so that the code generators are shared between the immediate-count and variable-count cases. In particular, this makes gen_RCL and gen_RCR pretty efficient for the count == 1 case, which becomes (apart from a few extra movs) something like: (compute_cc_all if needed) // save old value for OF calculation mov cc_src2, T0 // the bulk of RCL is just this! deposit T0, cc_src, T0, 1, TARGET_LONG_BITS - 1 // compute carry shr cc_dst, cc_src2, length - 1 and cc_dst, cc_dst, 1 // compute overflow xor cc_src2, cc_src2, T0 extract cc_src2, cc_src2, length - 1, 1 32-bit MUL and IMUL are also slightly more efficient on 64-bit hosts. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-21 18:36:34 +03:00
* For memory-only operands, if the emitter functions wants to rely on
* generic load and writeback, the decoder needs to know the type of the
* operand. Therefore, M is often replaced by the more specific EM and WM
* (respectively selecting an ALU operand, like the operand type E, or a
* vector operand like the operand type W).
*
target/i386: move 60-BF opcodes to new decoder Compared to the old decoder, the main differences in translation are for the little-used ARPL instruction. IMUL is adjusted a bit to share more code to produce flags, but is otherwise very similar. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-11 11:55:16 +03:00
* Immediates are almost always signed or masked away in helpers. Two
* common exceptions are IN/OUT and absolute jumps. For these, there is
* an additional custom operand type "I_unsigned". Alternatively, the
* mask could be applied (and the original sign-extended value would be
* optimized away by TCG) in the emitter function.
*
target/i386: move prefetch and multi-byte UD/NOP to new decoder These are trivial to add, and moving them to the new decoder fixes some corner cases: raising #UD instead of an instruction fetch page fault for the undefined opcodes, and incorrectly rejecting 0F 18 prefetches with register operands (which are treated as reserved NOPs). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Zhao Liu <zhao1.liu@intel.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-06 18:34:55 +03:00
* Finally, a "nop" operand type is used for multi-byte NOPs. It accepts
* any value of mod including 11b (unlike M) but it does not try to
* interpret the operand (like M).
*
target/i386: validate VEX.W for AVX instructions Instructions in VEX exception class 6 generally look at the value of VEX.W. Note that the manual places some instructions incorrectly in class 4, for example VPERMQ which has no non-VEX encoding and no legacy SSE analogue. AMD does a mess of its own, as documented in the comment that this patch adds. Most of them are checked for VEX.W=0, and are listed in the manual (though with an omission) in table 2-16; VPERMQ and VPERMPD check for VEX.W=1, which is only listed in the instruction description. Others, such as VPSRLV, VPSLLV and the FMA3 instructions, use VEX.W to switch between a 32-bit and 64-bit operation. Fix more of the class 4/class 6 mismatches, and implement the check for VEX.W in TCG. Acked-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-09 19:16:27 +03:00
* Vector operands
* ---------------
target/i386: add core of new i386 decoder The new decoder is based on three principles: - use mostly table-driven decoding, using tables derived as much as possible from the Intel manual. Centralizing the decode the operands makes it more homogeneous, for example all immediates are signed. All modrm handling is in one function, and can be shared between SSE and ALU instructions (including XMM<->GPR instructions). The SSE/AVX decoder will also not have duplicated code between the 0F, 0F38 and 0F3A tables. - keep the code as "non-branchy" as possible. Generally, the code for the new decoder is more verbose, but the control flow is simpler. Conditionals are not nested and have small bodies. All instruction groups are resolved even before operands are decoded, and code generation is separated as much as possible within small functions that only handle one instruction each. - keep address generation and (for ALU operands) memory loads and writeback as much in common code as possible. All ALU operations for example are implemented as T0=f(T0,T1). For non-ALU instructions, read-modify-write memory operations are rare, but registers do not have TCGv equivalents: therefore, the common logic sets up pointer temporaries with the operands, while load and writeback are handled by gvec or by helpers. These principles make future code review and extensibility simpler, at the cost of having a relatively large amount of code in the form of this patch. Even EVEX should not be _too_ hard to implement (it's just a crazy large amount of possibilities). This patch introduces the main decoder flow, and integrates the old decoder with the new one. The old decoder takes care of parsing prefixes and then optionally drops to the new one. The changes to the old decoder are minimal and allow it to be replaced incrementally with the new one. There is a debugging mechanism through a "LIMIT" environment variable. In user-mode emulation, the variable is the number of instructions decoded by the new decoder before permanently switching to the old one. In system emulation, the variable is the highest opcode that is decoded by the new decoder (this is less friendly, but it's the best that can be done without requiring deterministic execution). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-23 12:20:55 +03:00
*
* The main difference is that the V, U and W types are extended to
* cover MMX as well; if an instruction is like
*
* por Pq, Qq
* 66 por Vx, Hx, Wx
*
* only the second row is included and the instruction is marked as a
* valid MMX instruction. The MMX flag directs the decoder to rewrite
* the V/U/H/W types to P/N/P/Q if there is no prefix, as well as changing
* "x" to "q" if there is no prefix.
*
* In addition, the ss/ps/sd/pd types are sometimes mushed together as "x"
* if the difference is expressed via prefixes. Individual instructions
* are separated by prefix in the generator functions.
*
target/i386: document more deviations from the manual Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-23 10:58:48 +03:00
* There is a custom size "xh" used to address half of a SSE/AVX operand.
* This points to a 64-bit operand for SSE operations, 128-bit operand
* for 256-bit AVX operands, etc. It is used for conversion operations
* such as VCVTPH2PS or VCVTSS2SD.
*
target/i386: add core of new i386 decoder The new decoder is based on three principles: - use mostly table-driven decoding, using tables derived as much as possible from the Intel manual. Centralizing the decode the operands makes it more homogeneous, for example all immediates are signed. All modrm handling is in one function, and can be shared between SSE and ALU instructions (including XMM<->GPR instructions). The SSE/AVX decoder will also not have duplicated code between the 0F, 0F38 and 0F3A tables. - keep the code as "non-branchy" as possible. Generally, the code for the new decoder is more verbose, but the control flow is simpler. Conditionals are not nested and have small bodies. All instruction groups are resolved even before operands are decoded, and code generation is separated as much as possible within small functions that only handle one instruction each. - keep address generation and (for ALU operands) memory loads and writeback as much in common code as possible. All ALU operations for example are implemented as T0=f(T0,T1). For non-ALU instructions, read-modify-write memory operations are rare, but registers do not have TCGv equivalents: therefore, the common logic sets up pointer temporaries with the operands, while load and writeback are handled by gvec or by helpers. These principles make future code review and extensibility simpler, at the cost of having a relatively large amount of code in the form of this patch. Even EVEX should not be _too_ hard to implement (it's just a crazy large amount of possibilities). This patch introduces the main decoder flow, and integrates the old decoder with the new one. The old decoder takes care of parsing prefixes and then optionally drops to the new one. The changes to the old decoder are minimal and allow it to be replaced incrementally with the new one. There is a debugging mechanism through a "LIMIT" environment variable. In user-mode emulation, the variable is the number of instructions decoded by the new decoder before permanently switching to the old one. In system emulation, the variable is the highest opcode that is decoded by the new decoder (this is less friendly, but it's the best that can be done without requiring deterministic execution). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-23 12:20:55 +03:00
* There are a couple cases in which instructions (e.g. MOVD) write the
* whole XMM or MM register but are established incorrectly in the manual
* as "d" or "q". These have to be fixed for the decoder to work correctly.
target/i386: validate VEX.W for AVX instructions Instructions in VEX exception class 6 generally look at the value of VEX.W. Note that the manual places some instructions incorrectly in class 4, for example VPERMQ which has no non-VEX encoding and no legacy SSE analogue. AMD does a mess of its own, as documented in the comment that this patch adds. Most of them are checked for VEX.W=0, and are listed in the manual (though with an omission) in table 2-16; VPERMQ and VPERMPD check for VEX.W=1, which is only listed in the instruction description. Others, such as VPSRLV, VPSLLV and the FMA3 instructions, use VEX.W to switch between a 32-bit and 64-bit operation. Fix more of the class 4/class 6 mismatches, and implement the check for VEX.W in TCG. Acked-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-09 19:16:27 +03:00
*
* VEX exception classes
* ---------------------
*
* Speaking about imprecisions in the manual, the decoder treats all
* exception-class 4 instructions as having an optional VEX prefix, and
* all exception-class 6 instructions as having a mandatory VEX prefix.
* This is true except for a dozen instructions; these are in exception
* class 4 but do not ignore the VEX.W bit (which does not even exist
* without a VEX prefix). These instructions are mostly listed in Intel's
* table 2-16, but with a few exceptions.
*
* The AMD manual has more precise subclasses for exceptions, and unlike Intel
* they list the VEX.W requirements in the exception classes as well (except
* when they don't). AMD describes class 6 as "AVX Mixed Memory Argument"
* without defining what a mixed memory argument is, but still use 4 as the
* primary exception class... except when they don't.
*
* The summary is:
* Intel AMD VEX.W note
* -------------------------------------------------------------------
* vpblendd 4 4J 0
* vpblendvb 4 4E-X 0 (*)
* vpbroadcastq 6 6D 0 (+)
* vpermd/vpermps 4 4H 0 (§)
* vpermq/vpermpd 4 4H-1 1 (§)
* vpermilpd/vpermilps 4 6E 0 (^)
* vpmaskmovd 6 4K significant (^)
* vpsllv 4 4K significant
* vpsrav 4 4J 0
* vpsrlv 4 4K significant
* vtestps/vtestpd 4 4G 0
*
* (*) AMD lists VPBLENDVB as related to SSE4.1 PBLENDVB, which may
* explain why it is considered exception class 4. However,
* Intel says that VEX-only instructions should be in class 6...
*
* (+) Not found in Intel's table 2-16
*
* (§) 4H and 4H-1 do not mention VEX.W requirements, which are
* however present in the description of the instruction
*
* (^) these are the two cases in which Intel and AMD disagree on the
* primary exception class
target/i386: add core of new i386 decoder The new decoder is based on three principles: - use mostly table-driven decoding, using tables derived as much as possible from the Intel manual. Centralizing the decode the operands makes it more homogeneous, for example all immediates are signed. All modrm handling is in one function, and can be shared between SSE and ALU instructions (including XMM<->GPR instructions). The SSE/AVX decoder will also not have duplicated code between the 0F, 0F38 and 0F3A tables. - keep the code as "non-branchy" as possible. Generally, the code for the new decoder is more verbose, but the control flow is simpler. Conditionals are not nested and have small bodies. All instruction groups are resolved even before operands are decoded, and code generation is separated as much as possible within small functions that only handle one instruction each. - keep address generation and (for ALU operands) memory loads and writeback as much in common code as possible. All ALU operations for example are implemented as T0=f(T0,T1). For non-ALU instructions, read-modify-write memory operations are rare, but registers do not have TCGv equivalents: therefore, the common logic sets up pointer temporaries with the operands, while load and writeback are handled by gvec or by helpers. These principles make future code review and extensibility simpler, at the cost of having a relatively large amount of code in the form of this patch. Even EVEX should not be _too_ hard to implement (it's just a crazy large amount of possibilities). This patch introduces the main decoder flow, and integrates the old decoder with the new one. The old decoder takes care of parsing prefixes and then optionally drops to the new one. The changes to the old decoder are minimal and allow it to be replaced incrementally with the new one. There is a debugging mechanism through a "LIMIT" environment variable. In user-mode emulation, the variable is the number of instructions decoded by the new decoder before permanently switching to the old one. In system emulation, the variable is the highest opcode that is decoded by the new decoder (this is less friendly, but it's the best that can be done without requiring deterministic execution). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-23 12:20:55 +03:00
*/
#define X86_OP_NONE { 0 },
#define X86_OP_GROUP3(op, op0_, s0_, op1_, s1_, op2_, s2_, ...) { \
.decode = glue(decode_, op), \
.op0 = glue(X86_TYPE_, op0_), \
.s0 = glue(X86_SIZE_, s0_), \
.op1 = glue(X86_TYPE_, op1_), \
.s1 = glue(X86_SIZE_, s1_), \
.op2 = glue(X86_TYPE_, op2_), \
.s2 = glue(X86_SIZE_, s2_), \
.is_decode = true, \
## __VA_ARGS__ \
}
target/i386: move C0-FF opcodes to new decoder (except for x87) The shift instructions are rewritten instead of reusing code from the old decoder. Rotates use CC_OP_ADCOX more extensively and generally rely more on the optimizer, so that the code generators are shared between the immediate-count and variable-count cases. In particular, this makes gen_RCL and gen_RCR pretty efficient for the count == 1 case, which becomes (apart from a few extra movs) something like: (compute_cc_all if needed) // save old value for OF calculation mov cc_src2, T0 // the bulk of RCL is just this! deposit T0, cc_src, T0, 1, TARGET_LONG_BITS - 1 // compute carry shr cc_dst, cc_src2, length - 1 and cc_dst, cc_dst, 1 // compute overflow xor cc_src2, cc_src2, T0 extract cc_src2, cc_src2, length - 1, 1 32-bit MUL and IMUL are also slightly more efficient on 64-bit hosts. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-21 18:36:34 +03:00
#define X86_OP_GROUP1(op, op0, s0, ...) \
X86_OP_GROUP3(op, op0, s0, 2op, s0, None, None, ## __VA_ARGS__)
target/i386: add core of new i386 decoder The new decoder is based on three principles: - use mostly table-driven decoding, using tables derived as much as possible from the Intel manual. Centralizing the decode the operands makes it more homogeneous, for example all immediates are signed. All modrm handling is in one function, and can be shared between SSE and ALU instructions (including XMM<->GPR instructions). The SSE/AVX decoder will also not have duplicated code between the 0F, 0F38 and 0F3A tables. - keep the code as "non-branchy" as possible. Generally, the code for the new decoder is more verbose, but the control flow is simpler. Conditionals are not nested and have small bodies. All instruction groups are resolved even before operands are decoded, and code generation is separated as much as possible within small functions that only handle one instruction each. - keep address generation and (for ALU operands) memory loads and writeback as much in common code as possible. All ALU operations for example are implemented as T0=f(T0,T1). For non-ALU instructions, read-modify-write memory operations are rare, but registers do not have TCGv equivalents: therefore, the common logic sets up pointer temporaries with the operands, while load and writeback are handled by gvec or by helpers. These principles make future code review and extensibility simpler, at the cost of having a relatively large amount of code in the form of this patch. Even EVEX should not be _too_ hard to implement (it's just a crazy large amount of possibilities). This patch introduces the main decoder flow, and integrates the old decoder with the new one. The old decoder takes care of parsing prefixes and then optionally drops to the new one. The changes to the old decoder are minimal and allow it to be replaced incrementally with the new one. There is a debugging mechanism through a "LIMIT" environment variable. In user-mode emulation, the variable is the number of instructions decoded by the new decoder before permanently switching to the old one. In system emulation, the variable is the highest opcode that is decoded by the new decoder (this is less friendly, but it's the best that can be done without requiring deterministic execution). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-23 12:20:55 +03:00
#define X86_OP_GROUP2(op, op0, s0, op1, s1, ...) \
X86_OP_GROUP3(op, op0, s0, 2op, s0, op1, s1, ## __VA_ARGS__)
target/i386: move 00-5F opcodes to new decoder Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-20 01:30:21 +03:00
#define X86_OP_GROUPw(op, op0, s0, ...) \
X86_OP_GROUP3(op, op0, s0, None, None, None, None, ## __VA_ARGS__)
target/i386: convert MOV from/to CR and DR to new decoder Complete implementation of C and D operand types, then the operations are just MOVs. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-29 16:55:22 +03:00
#define X86_OP_GROUPwr(op, op0, s0, op1, s1, ...) \
X86_OP_GROUP3(op, op0, s0, op1, s1, None, None, ## __VA_ARGS__)
target/i386: add core of new i386 decoder The new decoder is based on three principles: - use mostly table-driven decoding, using tables derived as much as possible from the Intel manual. Centralizing the decode the operands makes it more homogeneous, for example all immediates are signed. All modrm handling is in one function, and can be shared between SSE and ALU instructions (including XMM<->GPR instructions). The SSE/AVX decoder will also not have duplicated code between the 0F, 0F38 and 0F3A tables. - keep the code as "non-branchy" as possible. Generally, the code for the new decoder is more verbose, but the control flow is simpler. Conditionals are not nested and have small bodies. All instruction groups are resolved even before operands are decoded, and code generation is separated as much as possible within small functions that only handle one instruction each. - keep address generation and (for ALU operands) memory loads and writeback as much in common code as possible. All ALU operations for example are implemented as T0=f(T0,T1). For non-ALU instructions, read-modify-write memory operations are rare, but registers do not have TCGv equivalents: therefore, the common logic sets up pointer temporaries with the operands, while load and writeback are handled by gvec or by helpers. These principles make future code review and extensibility simpler, at the cost of having a relatively large amount of code in the form of this patch. Even EVEX should not be _too_ hard to implement (it's just a crazy large amount of possibilities). This patch introduces the main decoder flow, and integrates the old decoder with the new one. The old decoder takes care of parsing prefixes and then optionally drops to the new one. The changes to the old decoder are minimal and allow it to be replaced incrementally with the new one. There is a debugging mechanism through a "LIMIT" environment variable. In user-mode emulation, the variable is the number of instructions decoded by the new decoder before permanently switching to the old one. In system emulation, the variable is the highest opcode that is decoded by the new decoder (this is less friendly, but it's the best that can be done without requiring deterministic execution). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-23 12:20:55 +03:00
#define X86_OP_GROUP0(op, ...) \
X86_OP_GROUP3(op, None, None, None, None, None, None, ## __VA_ARGS__)
#define X86_OP_ENTRY3(op, op0_, s0_, op1_, s1_, op2_, s2_, ...) { \
.gen = glue(gen_, op), \
.op0 = glue(X86_TYPE_, op0_), \
.s0 = glue(X86_SIZE_, s0_), \
.op1 = glue(X86_TYPE_, op1_), \
.s1 = glue(X86_SIZE_, s1_), \
.op2 = glue(X86_TYPE_, op2_), \
.s2 = glue(X86_SIZE_, s2_), \
## __VA_ARGS__ \
}
#define X86_OP_ENTRY4(op, op0_, s0_, op1_, s1_, op2_, s2_, ...) \
X86_OP_ENTRY3(op, op0_, s0_, op1_, s1_, op2_, s2_, \
.op3 = X86_TYPE_I, .s3 = X86_SIZE_b, \
## __VA_ARGS__)
target/i386: move 00-5F opcodes to new decoder Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-20 01:30:21 +03:00
/*
* Short forms that are mostly useful for ALU opcodes and other
* one-byte opcodes. For vector instructions it is usually
* clearer to write all three operands explicitly, because the
* corresponding gen_* function will use OP_PTRn rather than s->T0
* and s->T1.
*/
#define X86_OP_ENTRYrr(op, op0, s0, op1, s1, ...) \
X86_OP_ENTRY3(op, None, None, op0, s0, op1, s1, ## __VA_ARGS__)
target/i386: move 60-BF opcodes to new decoder Compared to the old decoder, the main differences in translation are for the little-used ARPL instruction. IMUL is adjusted a bit to share more code to produce flags, but is otherwise very similar. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-11 11:55:16 +03:00
#define X86_OP_ENTRYwr(op, op0, s0, op1, s1, ...) \
target/i386: change X86_ENTRYwr to use T0, use it for moves Just like X86_ENTRYr, X86_ENTRYwr is easily changed to use only T0. In this case, the motivation is to use it for the MOV instruction family. The case when you need to preserve the input value is the odd one, as it is used basically only for BLS* instructions. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-08 12:06:50 +03:00
X86_OP_ENTRY3(op, op0, s0, op1, s1, None, None, ## __VA_ARGS__)
target/i386: add core of new i386 decoder The new decoder is based on three principles: - use mostly table-driven decoding, using tables derived as much as possible from the Intel manual. Centralizing the decode the operands makes it more homogeneous, for example all immediates are signed. All modrm handling is in one function, and can be shared between SSE and ALU instructions (including XMM<->GPR instructions). The SSE/AVX decoder will also not have duplicated code between the 0F, 0F38 and 0F3A tables. - keep the code as "non-branchy" as possible. Generally, the code for the new decoder is more verbose, but the control flow is simpler. Conditionals are not nested and have small bodies. All instruction groups are resolved even before operands are decoded, and code generation is separated as much as possible within small functions that only handle one instruction each. - keep address generation and (for ALU operands) memory loads and writeback as much in common code as possible. All ALU operations for example are implemented as T0=f(T0,T1). For non-ALU instructions, read-modify-write memory operations are rare, but registers do not have TCGv equivalents: therefore, the common logic sets up pointer temporaries with the operands, while load and writeback are handled by gvec or by helpers. These principles make future code review and extensibility simpler, at the cost of having a relatively large amount of code in the form of this patch. Even EVEX should not be _too_ hard to implement (it's just a crazy large amount of possibilities). This patch introduces the main decoder flow, and integrates the old decoder with the new one. The old decoder takes care of parsing prefixes and then optionally drops to the new one. The changes to the old decoder are minimal and allow it to be replaced incrementally with the new one. There is a debugging mechanism through a "LIMIT" environment variable. In user-mode emulation, the variable is the number of instructions decoded by the new decoder before permanently switching to the old one. In system emulation, the variable is the highest opcode that is decoded by the new decoder (this is less friendly, but it's the best that can be done without requiring deterministic execution). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-23 12:20:55 +03:00
#define X86_OP_ENTRY2(op, op0, s0, op1, s1, ...) \
X86_OP_ENTRY3(op, op0, s0, 2op, s0, op1, s1, ## __VA_ARGS__)
target/i386: implement VLDMXCSR/VSTMXCSR These are exactly the same as the non-VEX version, but one has to be careful that only VEX.L=0 is allowed. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-11 14:22:32 +03:00
#define X86_OP_ENTRYw(op, op0, s0, ...) \
X86_OP_ENTRY3(op, op0, s0, None, None, None, None, ## __VA_ARGS__)
#define X86_OP_ENTRYr(op, op0, s0, ...) \
target/i386: change X86_ENTRYr to use T0 I am not sure why I made it use T1. It is a bit more symmetric with respect to X86_ENTRYwr (which uses T0 for the "w"ritten operand and T1 for the "r"ead operand), but it is also less flexible because it does not let you apply zextT0/sextT0. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-08 12:06:50 +03:00
X86_OP_ENTRY3(op, None, None, op0, s0, None, None, ## __VA_ARGS__)
target/i386: move 00-5F opcodes to new decoder Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-20 01:30:21 +03:00
#define X86_OP_ENTRY1(op, op0, s0, ...) \
X86_OP_ENTRY3(op, op0, s0, 2op, s0, None, None, ## __VA_ARGS__)
target/i386: add core of new i386 decoder The new decoder is based on three principles: - use mostly table-driven decoding, using tables derived as much as possible from the Intel manual. Centralizing the decode the operands makes it more homogeneous, for example all immediates are signed. All modrm handling is in one function, and can be shared between SSE and ALU instructions (including XMM<->GPR instructions). The SSE/AVX decoder will also not have duplicated code between the 0F, 0F38 and 0F3A tables. - keep the code as "non-branchy" as possible. Generally, the code for the new decoder is more verbose, but the control flow is simpler. Conditionals are not nested and have small bodies. All instruction groups are resolved even before operands are decoded, and code generation is separated as much as possible within small functions that only handle one instruction each. - keep address generation and (for ALU operands) memory loads and writeback as much in common code as possible. All ALU operations for example are implemented as T0=f(T0,T1). For non-ALU instructions, read-modify-write memory operations are rare, but registers do not have TCGv equivalents: therefore, the common logic sets up pointer temporaries with the operands, while load and writeback are handled by gvec or by helpers. These principles make future code review and extensibility simpler, at the cost of having a relatively large amount of code in the form of this patch. Even EVEX should not be _too_ hard to implement (it's just a crazy large amount of possibilities). This patch introduces the main decoder flow, and integrates the old decoder with the new one. The old decoder takes care of parsing prefixes and then optionally drops to the new one. The changes to the old decoder are minimal and allow it to be replaced incrementally with the new one. There is a debugging mechanism through a "LIMIT" environment variable. In user-mode emulation, the variable is the number of instructions decoded by the new decoder before permanently switching to the old one. In system emulation, the variable is the highest opcode that is decoded by the new decoder (this is less friendly, but it's the best that can be done without requiring deterministic execution). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-23 12:20:55 +03:00
#define X86_OP_ENTRY0(op, ...) \
X86_OP_ENTRY3(op, None, None, None, None, None, None, ## __VA_ARGS__)
target/i386: add CPUID feature checks to new decoder Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-01 15:51:35 +03:00
#define cpuid(feat) .cpuid = X86_FEAT_##feat,
target/i386: replace NoSeg special with NoLoadEA This is a bit more generic, as it can be applied to MPX as well. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-09 17:56:26 +03:00
#define nolea .special = X86_SPECIAL_NoLoadEA,
target/i386: add core of new i386 decoder The new decoder is based on three principles: - use mostly table-driven decoding, using tables derived as much as possible from the Intel manual. Centralizing the decode the operands makes it more homogeneous, for example all immediates are signed. All modrm handling is in one function, and can be shared between SSE and ALU instructions (including XMM<->GPR instructions). The SSE/AVX decoder will also not have duplicated code between the 0F, 0F38 and 0F3A tables. - keep the code as "non-branchy" as possible. Generally, the code for the new decoder is more verbose, but the control flow is simpler. Conditionals are not nested and have small bodies. All instruction groups are resolved even before operands are decoded, and code generation is separated as much as possible within small functions that only handle one instruction each. - keep address generation and (for ALU operands) memory loads and writeback as much in common code as possible. All ALU operations for example are implemented as T0=f(T0,T1). For non-ALU instructions, read-modify-write memory operations are rare, but registers do not have TCGv equivalents: therefore, the common logic sets up pointer temporaries with the operands, while load and writeback are handled by gvec or by helpers. These principles make future code review and extensibility simpler, at the cost of having a relatively large amount of code in the form of this patch. Even EVEX should not be _too_ hard to implement (it's just a crazy large amount of possibilities). This patch introduces the main decoder flow, and integrates the old decoder with the new one. The old decoder takes care of parsing prefixes and then optionally drops to the new one. The changes to the old decoder are minimal and allow it to be replaced incrementally with the new one. There is a debugging mechanism through a "LIMIT" environment variable. In user-mode emulation, the variable is the number of instructions decoded by the new decoder before permanently switching to the old one. In system emulation, the variable is the highest opcode that is decoded by the new decoder (this is less friendly, but it's the best that can be done without requiring deterministic execution). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-23 12:20:55 +03:00
#define xchg .special = X86_SPECIAL_Locked,
target/i386: reimplement check for validity of LOCK prefix The previous check erroneously allowed CMP to be modified with LOCK. Instead, tag explicitly the instructions that do support LOCK. Acked-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-19 15:22:53 +03:00
#define lock .special = X86_SPECIAL_HasLock,
target/i386: add core of new i386 decoder The new decoder is based on three principles: - use mostly table-driven decoding, using tables derived as much as possible from the Intel manual. Centralizing the decode the operands makes it more homogeneous, for example all immediates are signed. All modrm handling is in one function, and can be shared between SSE and ALU instructions (including XMM<->GPR instructions). The SSE/AVX decoder will also not have duplicated code between the 0F, 0F38 and 0F3A tables. - keep the code as "non-branchy" as possible. Generally, the code for the new decoder is more verbose, but the control flow is simpler. Conditionals are not nested and have small bodies. All instruction groups are resolved even before operands are decoded, and code generation is separated as much as possible within small functions that only handle one instruction each. - keep address generation and (for ALU operands) memory loads and writeback as much in common code as possible. All ALU operations for example are implemented as T0=f(T0,T1). For non-ALU instructions, read-modify-write memory operations are rare, but registers do not have TCGv equivalents: therefore, the common logic sets up pointer temporaries with the operands, while load and writeback are handled by gvec or by helpers. These principles make future code review and extensibility simpler, at the cost of having a relatively large amount of code in the form of this patch. Even EVEX should not be _too_ hard to implement (it's just a crazy large amount of possibilities). This patch introduces the main decoder flow, and integrates the old decoder with the new one. The old decoder takes care of parsing prefixes and then optionally drops to the new one. The changes to the old decoder are minimal and allow it to be replaced incrementally with the new one. There is a debugging mechanism through a "LIMIT" environment variable. In user-mode emulation, the variable is the number of instructions decoded by the new decoder before permanently switching to the old one. In system emulation, the variable is the highest opcode that is decoded by the new decoder (this is less friendly, but it's the best that can be done without requiring deterministic execution). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-23 12:20:55 +03:00
#define mmx .special = X86_SPECIAL_MMX,
target/i386: rename zext0/zext2 and make them closer to the manual X86_SPECIAL_ZExtOp0 and X86_SPECIAL_ZExtOp2 are poorly named; they are a hack that is needed by scalar insertion and extraction instructions, and not really related to zero extension: for PEXTR the zero extension is done by the generation functions, for PINSR the high bits are not used at all and in fact are *not* filled with zeroes when loaded into s->T1. Rename the values to match the effect described in the manual, and explain better in the comments. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-20 00:14:34 +03:00
#define op0_Rd .special = X86_SPECIAL_Op0_Rd,
#define op2_Ry .special = X86_SPECIAL_Op2_Ry,
target/i386: reimplement 0x0f 0x38, add AVX There are several special cases here: 1) extending moves have different widths for the helpers vs. for the memory loads, and the width for memory loads depends on VEX.L too. This is represented by X86_SPECIAL_AVXExtMov. 2) some instructions, such as variable-width shifts, select the vector element size via REX.W. 3) VSIB instructions (VGATHERxPy, VPGATHERxy) are also part of this group, and they have (among other things) two output operands. 3) the macros for 4-operand blends (which are under 0x0f 0x3a) have to be extended to support 2-operand blends. The 2-operand variant actually came a few years earlier, but it is clearer to implement them in the opposite order. X86_TYPE_WM, introduced earlier for unaligned loads, is reused for helpers that accept a Reg* but have a M argument. These three-byte opcodes also include AVX new instructions, for which the helpers were originally implemented by Paul Brook <paul@nowt.org>. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-14 19:52:44 +03:00
#define avx_movx .special = X86_SPECIAL_AVXExtMov,
target/i386: add X86_SPECIALs for MOVSX and MOVZX Usually the registers are just moved into s->T0 without much care for their operand size. However, in some cases we can get more efficient code if the operand fetching logic syncs with the emission function on what is nicer. All the current uses are mostly demonstrative and only reduce the code in the emission functions, because the instructions do not support memory operands. However the logic is generic and applies to several more instructions such as MOVSXD (aka movslq), one-byte shift instructions, multiplications, XLAT, and indirect calls/jumps. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-19 16:40:54 +03:00
#define sextT0 .special = X86_SPECIAL_SExtT0,
#define zextT0 .special = X86_SPECIAL_ZExtT0,
target/i386: fix memory opsize for Mov to/from Seg This commit fixes an issue with MOV instructions (0x8C and 0x8E) involving segment registers; MOV to segment register's source is 16-bit, while MOV from segment register has to explicitly set the memory operand size to 16 bits. Introduce a new flag X86_SPECIAL_Op0_Mw to handle this specification correctly. Signed-off-by: Xinyu Li <lixinyu20s@ict.ac.cn> Message-ID: <20240602100528.2135717-1-lixinyu20s@ict.ac.cn> Fixes: 5e9e21bcc4d ("target/i386: move 60-BF opcodes to new decoder", 2024-05-07) Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-06-02 13:05:28 +03:00
#define op0_Mw .special = X86_SPECIAL_Op0_Mw,
target/i386: add core of new i386 decoder The new decoder is based on three principles: - use mostly table-driven decoding, using tables derived as much as possible from the Intel manual. Centralizing the decode the operands makes it more homogeneous, for example all immediates are signed. All modrm handling is in one function, and can be shared between SSE and ALU instructions (including XMM<->GPR instructions). The SSE/AVX decoder will also not have duplicated code between the 0F, 0F38 and 0F3A tables. - keep the code as "non-branchy" as possible. Generally, the code for the new decoder is more verbose, but the control flow is simpler. Conditionals are not nested and have small bodies. All instruction groups are resolved even before operands are decoded, and code generation is separated as much as possible within small functions that only handle one instruction each. - keep address generation and (for ALU operands) memory loads and writeback as much in common code as possible. All ALU operations for example are implemented as T0=f(T0,T1). For non-ALU instructions, read-modify-write memory operations are rare, but registers do not have TCGv equivalents: therefore, the common logic sets up pointer temporaries with the operands, while load and writeback are handled by gvec or by helpers. These principles make future code review and extensibility simpler, at the cost of having a relatively large amount of code in the form of this patch. Even EVEX should not be _too_ hard to implement (it's just a crazy large amount of possibilities). This patch introduces the main decoder flow, and integrates the old decoder with the new one. The old decoder takes care of parsing prefixes and then optionally drops to the new one. The changes to the old decoder are minimal and allow it to be replaced incrementally with the new one. There is a debugging mechanism through a "LIMIT" environment variable. In user-mode emulation, the variable is the number of instructions decoded by the new decoder before permanently switching to the old one. In system emulation, the variable is the highest opcode that is decoded by the new decoder (this is less friendly, but it's the best that can be done without requiring deterministic execution). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-23 12:20:55 +03:00
target/i386: validate VEX prefixes via the instructions' exception classes Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-18 01:43:52 +03:00
#define vex1 .vex_class = 1,
#define vex1_rep3 .vex_class = 1, .vex_special = X86_VEX_REPScalar,
#define vex2 .vex_class = 2,
#define vex2_rep3 .vex_class = 2, .vex_special = X86_VEX_REPScalar,
#define vex3 .vex_class = 3,
#define vex4 .vex_class = 4,
#define vex4_unal .vex_class = 4, .vex_special = X86_VEX_SSEUnaligned,
target/i386: fix operand size of unary SSE operations VRCPSS, VRSQRTSS and VCVTSx2Sx have a 32-bit or 64-bit memory operand, which is represented in the decoding tables by X86_VEX_REPScalar. Add it to the tables, and make validate_vex() handle the case of an instruction that is in exception type 4 without the REP prefix and exception type 5 with it; this is the cas of VRCP and VRSQRT. Reported-by: yongwoo <https://gitlab.com/yongwoo36> Resolves: https://gitlab.com/qemu-project/qemu/-/issues/1377 Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-01-07 20:14:20 +03:00
#define vex4_rep5 .vex_class = 4, .vex_special = X86_VEX_REPScalar,
target/i386: validate VEX prefixes via the instructions' exception classes Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-18 01:43:52 +03:00
#define vex5 .vex_class = 5,
#define vex6 .vex_class = 6,
#define vex7 .vex_class = 7,
#define vex8 .vex_class = 8,
#define vex11 .vex_class = 11,
#define vex12 .vex_class = 12,
#define vex13 .vex_class = 13,
target/i386: group common checks in the decoding phase In preparation for adding more similar checks, move the VEX.L=0 check and several X86_SPECIAL_* checks to a new field, where each bit represent a common check on unused bits, or a restriction on the processor mode. Likewise, many SVM intercepts can be checked during the decoding phase, the main exception being the selective CR0 write, MSR and IOIO intercepts. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-09 18:43:12 +03:00
#define chk(a) .check = X86_CHECK_##a,
target/i386: convert non-grouped, helper-based 2-byte opcodes These have very simple generators and no need for complex group decoding. Apart from LAR/LSL which are simplified to use gen_op_deposit_reg_v and movcond, the code is generally lifted from translate.c into the generators. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-25 11:49:26 +03:00
#define chk2(a, b) .check = X86_CHECK_##a | X86_CHECK_##b,
#define chk3(a, b, c) .check = X86_CHECK_##a | X86_CHECK_##b | X86_CHECK_##c,
target/i386: fix processing of intercept 0 (read CR0) Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-09 15:34:24 +03:00
#define svm(a) .intercept = SVM_EXIT_##a, .has_intercept = true,
target/i386: group common checks in the decoding phase In preparation for adding more similar checks, move the VEX.L=0 check and several X86_SPECIAL_* checks to a new field, where each bit represent a common check on unused bits, or a restriction on the processor mode. Likewise, many SVM intercepts can be checked during the decoding phase, the main exception being the selective CR0 write, MSR and IOIO intercepts. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-09 18:43:12 +03:00
target/i386: validate VEX prefixes via the instructions' exception classes Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-18 01:43:52 +03:00
#define avx2_256 .vex_special = X86_VEX_AVX2_256,
target/i386: validate SSE prefixes directly in the decoding table Many SSE and AVX instructions are only valid with specific prefixes (none, 66, F3, F2). Introduce a direct way to encode this in the decoding table to avoid using decode groups too much. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-05 16:30:02 +03:00
#define P_00 1
#define P_66 (1 << PREFIX_DATA)
#define P_F3 (1 << PREFIX_REPZ)
#define P_F2 (1 << PREFIX_REPNZ)
#define p_00 .valid_prefix = P_00,
#define p_66 .valid_prefix = P_66,
#define p_f3 .valid_prefix = P_F3,
#define p_f2 .valid_prefix = P_F2,
#define p_00_66 .valid_prefix = P_00 | P_66,
#define p_00_f3 .valid_prefix = P_00 | P_F3,
#define p_66_f2 .valid_prefix = P_66 | P_F2,
#define p_00_66_f3 .valid_prefix = P_00 | P_66 | P_F3,
#define p_66_f3_f2 .valid_prefix = P_66 | P_F3 | P_F2,
#define p_00_66_f3_f2 .valid_prefix = P_00 | P_66 | P_F3 | P_F2,
target/i386: move 60-BF opcodes to new decoder Compared to the old decoder, the main differences in translation are for the little-used ARPL instruction. IMUL is adjusted a bit to share more code to produce flags, but is otherwise very similar. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-11 11:55:16 +03:00
#define UNKNOWN_OPCODE ((X86OpEntry) {})
target/i386: add core of new i386 decoder The new decoder is based on three principles: - use mostly table-driven decoding, using tables derived as much as possible from the Intel manual. Centralizing the decode the operands makes it more homogeneous, for example all immediates are signed. All modrm handling is in one function, and can be shared between SSE and ALU instructions (including XMM<->GPR instructions). The SSE/AVX decoder will also not have duplicated code between the 0F, 0F38 and 0F3A tables. - keep the code as "non-branchy" as possible. Generally, the code for the new decoder is more verbose, but the control flow is simpler. Conditionals are not nested and have small bodies. All instruction groups are resolved even before operands are decoded, and code generation is separated as much as possible within small functions that only handle one instruction each. - keep address generation and (for ALU operands) memory loads and writeback as much in common code as possible. All ALU operations for example are implemented as T0=f(T0,T1). For non-ALU instructions, read-modify-write memory operations are rare, but registers do not have TCGv equivalents: therefore, the common logic sets up pointer temporaries with the operands, while load and writeback are handled by gvec or by helpers. These principles make future code review and extensibility simpler, at the cost of having a relatively large amount of code in the form of this patch. Even EVEX should not be _too_ hard to implement (it's just a crazy large amount of possibilities). This patch introduces the main decoder flow, and integrates the old decoder with the new one. The old decoder takes care of parsing prefixes and then optionally drops to the new one. The changes to the old decoder are minimal and allow it to be replaced incrementally with the new one. There is a debugging mechanism through a "LIMIT" environment variable. In user-mode emulation, the variable is the number of instructions decoded by the new decoder before permanently switching to the old one. In system emulation, the variable is the highest opcode that is decoded by the new decoder (this is less friendly, but it's the best that can be done without requiring deterministic execution). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-23 12:20:55 +03:00
static uint8_t get_modrm(DisasContext *s, CPUX86State *env)
{
if (!s->has_modrm) {
s->modrm = x86_ldub_code(env, s);
s->has_modrm = true;
}
return s->modrm;
}
target/i386: reimplement 0x0f 0x60-0x6f, add AVX These are both MMX and SSE/AVX instructions, except for vmovdqu. In both cases the inputs and output is in s->ptr{0,1,2}, so the only difference between MMX, SSE, and AVX is which helper to call. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-20 12:42:45 +03:00
static inline const X86OpEntry *decode_by_prefix(DisasContext *s, const X86OpEntry entries[4])
{
if (s->prefix & PREFIX_REPNZ) {
return &entries[3];
} else if (s->prefix & PREFIX_REPZ) {
return &entries[2];
} else if (s->prefix & PREFIX_DATA) {
return &entries[1];
} else {
return &entries[0];
}
}
target/i386: implement VLDMXCSR/VSTMXCSR These are exactly the same as the non-VEX version, but one has to be careful that only VEX.L=0 is allowed. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-11 14:22:32 +03:00
static void decode_group15(DisasContext *s, CPUX86State *env, X86OpEntry *entry, uint8_t *b)
{
static const X86OpEntry group15_reg[8] = {
target/i386: finish converting 0F AE to the new decoder This is already partly implemented due to VLDMXCSR and VSTMXCSR; finish the job. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-08 18:45:53 +03:00
[0] = X86_OP_ENTRYw(RDxxBASE, R,y, cpuid(FSGSBASE) chk(o64) p_f3),
[1] = X86_OP_ENTRYw(RDxxBASE, R,y, cpuid(FSGSBASE) chk(o64) p_f3),
[2] = X86_OP_ENTRYr(WRxxBASE, R,y, cpuid(FSGSBASE) chk(o64) p_f3 zextT0),
[3] = X86_OP_ENTRYr(WRxxBASE, R,y, cpuid(FSGSBASE) chk(o64) p_f3 zextT0),
[5] = X86_OP_ENTRY0(LFENCE, cpuid(SSE2) p_00),
[6] = X86_OP_ENTRY0(MFENCE, cpuid(SSE2) p_00),
[7] = X86_OP_ENTRY0(SFENCE, cpuid(SSE2) p_00),
target/i386: implement VLDMXCSR/VSTMXCSR These are exactly the same as the non-VEX version, but one has to be careful that only VEX.L=0 is allowed. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-11 14:22:32 +03:00
};
static const X86OpEntry group15_mem[8] = {
target/i386: finish converting 0F AE to the new decoder This is already partly implemented due to VLDMXCSR and VSTMXCSR; finish the job. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-08 18:45:53 +03:00
[0] = X86_OP_ENTRYw(FXSAVE, M,y, cpuid(FXSR) p_00),
[1] = X86_OP_ENTRYr(FXRSTOR, M,y, cpuid(FXSR) p_00),
[2] = X86_OP_ENTRYr(LDMXCSR, E,d, vex5 chk(VEX128) p_00),
[3] = X86_OP_ENTRYw(STMXCSR, E,d, vex5 chk(VEX128) p_00),
[4] = X86_OP_ENTRYw(XSAVE, M,y, cpuid(XSAVE) p_00),
[5] = X86_OP_ENTRYr(XRSTOR, M,y, cpuid(XSAVE) p_00),
[6] = X86_OP_ENTRYw(XSAVEOPT, M,b, cpuid(XSAVEOPT) p_00),
[7] = X86_OP_ENTRYw(NOP, M,b, cpuid(CLFLUSH) p_00),
};
static const X86OpEntry group15_mem_66[8] = {
[6] = X86_OP_ENTRYw(NOP, M,b, cpuid(CLWB)),
[7] = X86_OP_ENTRYw(NOP, M,b, cpuid(CLFLUSHOPT)),
target/i386: implement VLDMXCSR/VSTMXCSR These are exactly the same as the non-VEX version, but one has to be careful that only VEX.L=0 is allowed. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-11 14:22:32 +03:00
};
uint8_t modrm = get_modrm(s, env);
target/i386: finish converting 0F AE to the new decoder This is already partly implemented due to VLDMXCSR and VSTMXCSR; finish the job. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-08 18:45:53 +03:00
int op = (modrm >> 3) & 7;
target/i386: implement VLDMXCSR/VSTMXCSR These are exactly the same as the non-VEX version, but one has to be careful that only VEX.L=0 is allowed. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-11 14:22:32 +03:00
if ((modrm >> 6) == 3) {
target/i386: finish converting 0F AE to the new decoder This is already partly implemented due to VLDMXCSR and VSTMXCSR; finish the job. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-08 18:45:53 +03:00
*entry = group15_reg[op];
} else if (s->prefix & PREFIX_DATA) {
*entry = group15_mem_66[op];
target/i386: implement VLDMXCSR/VSTMXCSR These are exactly the same as the non-VEX version, but one has to be careful that only VEX.L=0 is allowed. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-11 14:22:32 +03:00
} else {
target/i386: finish converting 0F AE to the new decoder This is already partly implemented due to VLDMXCSR and VSTMXCSR; finish the job. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-08 18:45:53 +03:00
*entry = group15_mem[op];
target/i386: implement VLDMXCSR/VSTMXCSR These are exactly the same as the non-VEX version, but one has to be careful that only VEX.L=0 is allowed. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-11 14:22:32 +03:00
}
}
target/i386: move scalar 0F 38 and 0F 3A instruction to new decoder Because these are the only VEX instructions that QEMU supports, the new decoder is entered on the first byte of a valid VEX prefix, and VEX decoding only needs to be done in decode-new.c.inc. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-24 19:01:41 +03:00
static void decode_group17(DisasContext *s, CPUX86State *env, X86OpEntry *entry, uint8_t *b)
{
static const X86GenFunc group17_gen[8] = {
NULL, gen_BLSR, gen_BLSMSK, gen_BLSI,
};
int op = (get_modrm(s, env) >> 3) & 7;
entry->gen = group17_gen[op];
}
target/i386: reimplement 0x0f 0x70-0x77, add AVX This includes shifts by immediate, which use bits 3-5 of the ModRM byte as an opcode extension. With the exception of 128-bit shifts, they are implemented using gvec. This also covers VZEROALL and VZEROUPPER, which use the same opcode as EMMS. If we were wanting to optimize out gen_clear_ymmh then this would be one of the starting points. The implementation of the VZEROALL and VZEROUPPER helpers is by Paul Brook. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-02 19:19:06 +03:00
static void decode_group12(DisasContext *s, CPUX86State *env, X86OpEntry *entry, uint8_t *b)
{
static const X86OpEntry opcodes_group12[8] = {
{},
{},
X86_OP_ENTRY3(PSRLW_i, H,x, U,x, I,b, vex7 mmx avx2_256 p_00_66),
{},
X86_OP_ENTRY3(PSRAW_i, H,x, U,x, I,b, vex7 mmx avx2_256 p_00_66),
{},
X86_OP_ENTRY3(PSLLW_i, H,x, U,x, I,b, vex7 mmx avx2_256 p_00_66),
{},
};
int op = (get_modrm(s, env) >> 3) & 7;
*entry = opcodes_group12[op];
}
static void decode_group13(DisasContext *s, CPUX86State *env, X86OpEntry *entry, uint8_t *b)
{
static const X86OpEntry opcodes_group13[8] = {
{},
{},
X86_OP_ENTRY3(PSRLD_i, H,x, U,x, I,b, vex7 mmx avx2_256 p_00_66),
{},
X86_OP_ENTRY3(PSRAD_i, H,x, U,x, I,b, vex7 mmx avx2_256 p_00_66),
{},
X86_OP_ENTRY3(PSLLD_i, H,x, U,x, I,b, vex7 mmx avx2_256 p_00_66),
{},
};
int op = (get_modrm(s, env) >> 3) & 7;
*entry = opcodes_group13[op];
}
static void decode_group14(DisasContext *s, CPUX86State *env, X86OpEntry *entry, uint8_t *b)
{
static const X86OpEntry opcodes_group14[8] = {
/* grp14 */
{},
{},
X86_OP_ENTRY3(PSRLQ_i, H,x, U,x, I,b, vex7 mmx avx2_256 p_00_66),
X86_OP_ENTRY3(PSRLDQ_i, H,x, U,x, I,b, vex7 avx2_256 p_66),
{},
{},
X86_OP_ENTRY3(PSLLQ_i, H,x, U,x, I,b, vex7 mmx avx2_256 p_00_66),
X86_OP_ENTRY3(PSLLDQ_i, H,x, U,x, I,b, vex7 avx2_256 p_66),
};
int op = (get_modrm(s, env) >> 3) & 7;
*entry = opcodes_group14[op];
}
target/i386: reimplement 0x0f 0x60-0x6f, add AVX These are both MMX and SSE/AVX instructions, except for vmovdqu. In both cases the inputs and output is in s->ptr{0,1,2}, so the only difference between MMX, SSE, and AVX is which helper to call. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-20 12:42:45 +03:00
static void decode_0F6F(DisasContext *s, CPUX86State *env, X86OpEntry *entry, uint8_t *b)
{
static const X86OpEntry opcodes_0F6F[4] = {
target/i386: Fix exception classes for SSE/AVX instructions. Fix the exception classes for some SSE/AVX instructions to match what is documented in the Intel manual. These changes are expected to have no functional effect on the behavior that qemu implements (primarily >= 16-byte memory alignment checks). For instance, since qemu does not implement the AC flag, there is no difference in behavior between Exception Classes 4 and 5 for instructions where the SSE version only takes <16 byte memory operands. Message-Id: <20230501111428.95998-2-ricky@rzhou.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-05-01 14:14:27 +03:00
X86_OP_ENTRY3(MOVDQ, P,q, None,None, Q,q, vex5 mmx), /* movq */
target/i386: reimplement 0x0f 0x60-0x6f, add AVX These are both MMX and SSE/AVX instructions, except for vmovdqu. In both cases the inputs and output is in s->ptr{0,1,2}, so the only difference between MMX, SSE, and AVX is which helper to call. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-20 12:42:45 +03:00
X86_OP_ENTRY3(MOVDQ, V,x, None,None, W,x, vex1), /* movdqa */
X86_OP_ENTRY3(MOVDQ, V,x, None,None, W,x, vex4_unal), /* movdqu */
{},
};
*entry = *decode_by_prefix(s, opcodes_0F6F);
}
target/i386: reimplement 0x0f 0x70-0x77, add AVX This includes shifts by immediate, which use bits 3-5 of the ModRM byte as an opcode extension. With the exception of 128-bit shifts, they are implemented using gvec. This also covers VZEROALL and VZEROUPPER, which use the same opcode as EMMS. If we were wanting to optimize out gen_clear_ymmh then this would be one of the starting points. The implementation of the VZEROALL and VZEROUPPER helpers is by Paul Brook. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-02 19:19:06 +03:00
static void decode_0F70(DisasContext *s, CPUX86State *env, X86OpEntry *entry, uint8_t *b)
{
static const X86OpEntry pshufw[4] = {
X86_OP_ENTRY3(PSHUFW, P,q, Q,q, I,b, vex4 mmx),
X86_OP_ENTRY3(PSHUFD, V,x, W,x, I,b, vex4 avx2_256),
X86_OP_ENTRY3(PSHUFHW, V,x, W,x, I,b, vex4 avx2_256),
X86_OP_ENTRY3(PSHUFLW, V,x, W,x, I,b, vex4 avx2_256),
};
*entry = *decode_by_prefix(s, pshufw);
}
static void decode_0F77(DisasContext *s, CPUX86State *env, X86OpEntry *entry, uint8_t *b)
{
if (!(s->prefix & PREFIX_VEX)) {
entry->gen = gen_EMMS;
} else if (!s->vex_l) {
entry->gen = gen_VZEROUPPER;
entry->vex_class = 8;
} else {
entry->gen = gen_VZEROALL;
entry->vex_class = 8;
}
}
target/i386: reimplement 0x0f 0x78-0x7f, add AVX These are a mixed batch, including the first two horizontal (66 and F2 only) operations, more moves, and SSE4a extract/insert. Because SSE4a is pretty rare, I chose to leave the helper as they are, but it is possible to unify them by loading index and length from the source XMM register and generating deposit or extract TCG ops. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-01 15:27:55 +03:00
static void decode_0F78(DisasContext *s, CPUX86State *env, X86OpEntry *entry, uint8_t *b)
{
static const X86OpEntry opcodes_0F78[4] = {
{},
target/i386: Fix and add some comments next to SSE/AVX instructions. Adds some comments describing what instructions correspond to decoding table entries and fixes some existing comments which named the wrong instruction. Message-Id: <20230501111428.95998-1-ricky@rzhou.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-05-01 14:14:26 +03:00
X86_OP_ENTRY3(EXTRQ_i, V,x, None,None, I,w, cpuid(SSE4A)), /* AMD extension */
target/i386: reimplement 0x0f 0x78-0x7f, add AVX These are a mixed batch, including the first two horizontal (66 and F2 only) operations, more moves, and SSE4a extract/insert. Because SSE4a is pretty rare, I chose to leave the helper as they are, but it is possible to unify them by loading index and length from the source XMM register and generating deposit or extract TCG ops. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-01 15:27:55 +03:00
{},
target/i386: Fix and add some comments next to SSE/AVX instructions. Adds some comments describing what instructions correspond to decoding table entries and fixes some existing comments which named the wrong instruction. Message-Id: <20230501111428.95998-1-ricky@rzhou.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-05-01 14:14:26 +03:00
X86_OP_ENTRY3(INSERTQ_i, V,x, U,x, I,w, cpuid(SSE4A)), /* AMD extension */
target/i386: reimplement 0x0f 0x78-0x7f, add AVX These are a mixed batch, including the first two horizontal (66 and F2 only) operations, more moves, and SSE4a extract/insert. Because SSE4a is pretty rare, I chose to leave the helper as they are, but it is possible to unify them by loading index and length from the source XMM register and generating deposit or extract TCG ops. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-01 15:27:55 +03:00
};
*entry = *decode_by_prefix(s, opcodes_0F78);
}
static void decode_0F79(DisasContext *s, CPUX86State *env, X86OpEntry *entry, uint8_t *b)
{
if (s->prefix & PREFIX_REPNZ) {
target/i386: Fix and add some comments next to SSE/AVX instructions. Adds some comments describing what instructions correspond to decoding table entries and fixes some existing comments which named the wrong instruction. Message-Id: <20230501111428.95998-1-ricky@rzhou.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-05-01 14:14:26 +03:00
entry->gen = gen_INSERTQ_r; /* AMD extension */
target/i386: reimplement 0x0f 0x78-0x7f, add AVX These are a mixed batch, including the first two horizontal (66 and F2 only) operations, more moves, and SSE4a extract/insert. Because SSE4a is pretty rare, I chose to leave the helper as they are, but it is possible to unify them by loading index and length from the source XMM register and generating deposit or extract TCG ops. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-01 15:27:55 +03:00
} else if (s->prefix & PREFIX_DATA) {
target/i386: Fix and add some comments next to SSE/AVX instructions. Adds some comments describing what instructions correspond to decoding table entries and fixes some existing comments which named the wrong instruction. Message-Id: <20230501111428.95998-1-ricky@rzhou.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-05-01 14:14:26 +03:00
entry->gen = gen_EXTRQ_r; /* AMD extension */
target/i386: reimplement 0x0f 0x78-0x7f, add AVX These are a mixed batch, including the first two horizontal (66 and F2 only) operations, more moves, and SSE4a extract/insert. Because SSE4a is pretty rare, I chose to leave the helper as they are, but it is possible to unify them by loading index and length from the source XMM register and generating deposit or extract TCG ops. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-01 15:27:55 +03:00
} else {
entry->gen = NULL;
};
}
static void decode_0F7E(DisasContext *s, CPUX86State *env, X86OpEntry *entry, uint8_t *b)
{
static const X86OpEntry opcodes_0F7E[4] = {
X86_OP_ENTRY3(MOVD_from, E,y, None,None, P,y, vex5 mmx),
X86_OP_ENTRY3(MOVD_from, E,y, None,None, V,y, vex5),
X86_OP_ENTRY3(MOVQ, V,x, None,None, W,q, vex5), /* wrong dest Vy on SDM! */
{},
};
*entry = *decode_by_prefix(s, opcodes_0F7E);
}
static void decode_0F7F(DisasContext *s, CPUX86State *env, X86OpEntry *entry, uint8_t *b)
{
static const X86OpEntry opcodes_0F7F[4] = {
target/i386: Fix exception classes for SSE/AVX instructions. Fix the exception classes for some SSE/AVX instructions to match what is documented in the Intel manual. These changes are expected to have no functional effect on the behavior that qemu implements (primarily >= 16-byte memory alignment checks). For instance, since qemu does not implement the AC flag, there is no difference in behavior between Exception Classes 4 and 5 for instructions where the SSE version only takes <16 byte memory operands. Message-Id: <20230501111428.95998-2-ricky@rzhou.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-05-01 14:14:27 +03:00
X86_OP_ENTRY3(MOVDQ, W,x, None,None, V,x, vex5 mmx), /* movq */
target/i386: reimplement 0x0f 0x78-0x7f, add AVX These are a mixed batch, including the first two horizontal (66 and F2 only) operations, more moves, and SSE4a extract/insert. Because SSE4a is pretty rare, I chose to leave the helper as they are, but it is possible to unify them by loading index and length from the source XMM register and generating deposit or extract TCG ops. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-01 15:27:55 +03:00
X86_OP_ENTRY3(MOVDQ, W,x, None,None, V,x, vex1), /* movdqa */
X86_OP_ENTRY3(MOVDQ, W,x, None,None, V,x, vex4_unal), /* movdqu */
{},
};
*entry = *decode_by_prefix(s, opcodes_0F7F);
}
target/i386: convert LZCNT/TZCNT/BSF/BSR/POPCNT to new decoder Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-09 16:11:41 +03:00
static void decode_0FB8(DisasContext *s, CPUX86State *env, X86OpEntry *entry, uint8_t *b)
{
static const X86OpEntry popcnt =
X86_OP_ENTRYwr(POPCNT, G,v, E,v, cpuid(POPCNT) zextT0);
if (s->prefix & PREFIX_REPZ) {
*entry = popcnt;
} else {
memset(entry, 0, sizeof(*entry));
}
}
static void decode_0FBC(DisasContext *s, CPUX86State *env, X86OpEntry *entry, uint8_t *b)
{
/* For BSF, pass 2op as the third operand so that we can use zextT0 */
static const X86OpEntry opcodes_0FBC[4] = {
X86_OP_ENTRY3(BSF, G,v, E,v, 2op,v, zextT0),
X86_OP_ENTRY3(BSF, G,v, E,v, 2op,v, zextT0), /* 0x66 */
X86_OP_ENTRYwr(TZCNT, G,v, E,v, zextT0), /* 0xf3 */
X86_OP_ENTRY3(BSF, G,v, E,v, 2op,v, zextT0), /* 0xf2 */
};
if (!(s->cpuid_ext3_features & CPUID_EXT3_ABM)) {
*entry = opcodes_0FBC[0];
} else {
*entry = *decode_by_prefix(s, opcodes_0FBC);
}
}
static void decode_0FBD(DisasContext *s, CPUX86State *env, X86OpEntry *entry, uint8_t *b)
{
/* For BSR, pass 2op as the third operand so that we can use zextT0 */
static const X86OpEntry opcodes_0FBD[4] = {
X86_OP_ENTRY3(BSR, G,v, E,v, 2op,v, zextT0),
X86_OP_ENTRY3(BSR, G,v, E,v, 2op,v, zextT0), /* 0x66 */
X86_OP_ENTRYwr(LZCNT, G,v, E,v, zextT0), /* 0xf3 */
X86_OP_ENTRY3(BSR, G,v, E,v, 2op,v, zextT0), /* 0xf2 */
};
if (!(s->cpuid_7_0_ebx_features & CPUID_7_0_EBX_BMI1)) {
*entry = opcodes_0FBD[0];
} else {
*entry = *decode_by_prefix(s, opcodes_0FBD);
}
}
target/i386: reimplement 0x0f 0xd0-0xd7, 0xe0-0xe7, 0xf0-0xf7, add AVX The more complicated ones here are d6-d7, e6-e7, f7. The others are trivial. For LDDQU, using gen_load_sse directly might corrupt the register if the second part of the load fails. Therefore, add a custom X86_TYPE_WM value; like X86_TYPE_W it does call gen_load(), but it also rejects a value of 11 in the ModRM field like X86_TYPE_M. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-01 15:27:55 +03:00
static void decode_0FD6(DisasContext *s, CPUX86State *env, X86OpEntry *entry, uint8_t *b)
{
static const X86OpEntry movq[4] = {
{},
X86_OP_ENTRY3(MOVQ, W,x, None, None, V,q, vex5),
X86_OP_ENTRY3(MOVq_dq, V,dq, None, None, N,q),
X86_OP_ENTRY3(MOVq_dq, P,q, None, None, U,q),
};
*entry = *decode_by_prefix(s, movq);
}
target/i386: add core of new i386 decoder The new decoder is based on three principles: - use mostly table-driven decoding, using tables derived as much as possible from the Intel manual. Centralizing the decode the operands makes it more homogeneous, for example all immediates are signed. All modrm handling is in one function, and can be shared between SSE and ALU instructions (including XMM<->GPR instructions). The SSE/AVX decoder will also not have duplicated code between the 0F, 0F38 and 0F3A tables. - keep the code as "non-branchy" as possible. Generally, the code for the new decoder is more verbose, but the control flow is simpler. Conditionals are not nested and have small bodies. All instruction groups are resolved even before operands are decoded, and code generation is separated as much as possible within small functions that only handle one instruction each. - keep address generation and (for ALU operands) memory loads and writeback as much in common code as possible. All ALU operations for example are implemented as T0=f(T0,T1). For non-ALU instructions, read-modify-write memory operations are rare, but registers do not have TCGv equivalents: therefore, the common logic sets up pointer temporaries with the operands, while load and writeback are handled by gvec or by helpers. These principles make future code review and extensibility simpler, at the cost of having a relatively large amount of code in the form of this patch. Even EVEX should not be _too_ hard to implement (it's just a crazy large amount of possibilities). This patch introduces the main decoder flow, and integrates the old decoder with the new one. The old decoder takes care of parsing prefixes and then optionally drops to the new one. The changes to the old decoder are minimal and allow it to be replaced incrementally with the new one. There is a debugging mechanism through a "LIMIT" environment variable. In user-mode emulation, the variable is the number of instructions decoded by the new decoder before permanently switching to the old one. In system emulation, the variable is the highest opcode that is decoded by the new decoder (this is less friendly, but it's the best that can be done without requiring deterministic execution). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-23 12:20:55 +03:00
static const X86OpEntry opcodes_0F38_00toEF[240] = {
target/i386: reimplement 0x0f 0x38, add AVX There are several special cases here: 1) extending moves have different widths for the helpers vs. for the memory loads, and the width for memory loads depends on VEX.L too. This is represented by X86_SPECIAL_AVXExtMov. 2) some instructions, such as variable-width shifts, select the vector element size via REX.W. 3) VSIB instructions (VGATHERxPy, VPGATHERxy) are also part of this group, and they have (among other things) two output operands. 3) the macros for 4-operand blends (which are under 0x0f 0x3a) have to be extended to support 2-operand blends. The 2-operand variant actually came a few years earlier, but it is clearer to implement them in the opposite order. X86_TYPE_WM, introduced earlier for unaligned loads, is reused for helpers that accept a Reg* but have a M argument. These three-byte opcodes also include AVX new instructions, for which the helpers were originally implemented by Paul Brook <paul@nowt.org>. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-14 19:52:44 +03:00
[0x00] = X86_OP_ENTRY3(PSHUFB, V,x, H,x, W,x, vex4 cpuid(SSSE3) mmx avx2_256 p_00_66),
[0x01] = X86_OP_ENTRY3(PHADDW, V,x, H,x, W,x, vex4 cpuid(SSSE3) mmx avx2_256 p_00_66),
[0x02] = X86_OP_ENTRY3(PHADDD, V,x, H,x, W,x, vex4 cpuid(SSSE3) mmx avx2_256 p_00_66),
[0x03] = X86_OP_ENTRY3(PHADDSW, V,x, H,x, W,x, vex4 cpuid(SSSE3) mmx avx2_256 p_00_66),
[0x04] = X86_OP_ENTRY3(PMADDUBSW, V,x, H,x, W,x, vex4 cpuid(SSSE3) mmx avx2_256 p_00_66),
[0x05] = X86_OP_ENTRY3(PHSUBW, V,x, H,x, W,x, vex4 cpuid(SSSE3) mmx avx2_256 p_00_66),
[0x06] = X86_OP_ENTRY3(PHSUBD, V,x, H,x, W,x, vex4 cpuid(SSSE3) mmx avx2_256 p_00_66),
[0x07] = X86_OP_ENTRY3(PHSUBSW, V,x, H,x, W,x, vex4 cpuid(SSSE3) mmx avx2_256 p_00_66),
[0x10] = X86_OP_ENTRY2(PBLENDVB, V,x, W,x, vex4 cpuid(SSE41) avx2_256 p_66),
target/i386: validate VEX.W for AVX instructions Instructions in VEX exception class 6 generally look at the value of VEX.W. Note that the manual places some instructions incorrectly in class 4, for example VPERMQ which has no non-VEX encoding and no legacy SSE analogue. AMD does a mess of its own, as documented in the comment that this patch adds. Most of them are checked for VEX.W=0, and are listed in the manual (though with an omission) in table 2-16; VPERMQ and VPERMPD check for VEX.W=1, which is only listed in the instruction description. Others, such as VPSRLV, VPSLLV and the FMA3 instructions, use VEX.W to switch between a 32-bit and 64-bit operation. Fix more of the class 4/class 6 mismatches, and implement the check for VEX.W in TCG. Acked-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-09 19:16:27 +03:00
[0x13] = X86_OP_ENTRY2(VCVTPH2PS, V,x, W,xh, vex11 chk(W0) cpuid(F16C) p_66),
target/i386: reimplement 0x0f 0x38, add AVX There are several special cases here: 1) extending moves have different widths for the helpers vs. for the memory loads, and the width for memory loads depends on VEX.L too. This is represented by X86_SPECIAL_AVXExtMov. 2) some instructions, such as variable-width shifts, select the vector element size via REX.W. 3) VSIB instructions (VGATHERxPy, VPGATHERxy) are also part of this group, and they have (among other things) two output operands. 3) the macros for 4-operand blends (which are under 0x0f 0x3a) have to be extended to support 2-operand blends. The 2-operand variant actually came a few years earlier, but it is clearer to implement them in the opposite order. X86_TYPE_WM, introduced earlier for unaligned loads, is reused for helpers that accept a Reg* but have a M argument. These three-byte opcodes also include AVX new instructions, for which the helpers were originally implemented by Paul Brook <paul@nowt.org>. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-14 19:52:44 +03:00
[0x14] = X86_OP_ENTRY2(BLENDVPS, V,x, W,x, vex4 cpuid(SSE41) p_66),
[0x15] = X86_OP_ENTRY2(BLENDVPD, V,x, W,x, vex4 cpuid(SSE41) p_66),
/* Listed incorrectly as type 4 */
target/i386: validate VEX.W for AVX instructions Instructions in VEX exception class 6 generally look at the value of VEX.W. Note that the manual places some instructions incorrectly in class 4, for example VPERMQ which has no non-VEX encoding and no legacy SSE analogue. AMD does a mess of its own, as documented in the comment that this patch adds. Most of them are checked for VEX.W=0, and are listed in the manual (though with an omission) in table 2-16; VPERMQ and VPERMPD check for VEX.W=1, which is only listed in the instruction description. Others, such as VPSRLV, VPSLLV and the FMA3 instructions, use VEX.W to switch between a 32-bit and 64-bit operation. Fix more of the class 4/class 6 mismatches, and implement the check for VEX.W in TCG. Acked-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-09 19:16:27 +03:00
[0x16] = X86_OP_ENTRY3(VPERMD, V,qq, H,qq, W,qq, vex6 chk(W0) cpuid(AVX2) p_66), /* vpermps */
target/i386: reimplement 0x0f 0x38, add AVX There are several special cases here: 1) extending moves have different widths for the helpers vs. for the memory loads, and the width for memory loads depends on VEX.L too. This is represented by X86_SPECIAL_AVXExtMov. 2) some instructions, such as variable-width shifts, select the vector element size via REX.W. 3) VSIB instructions (VGATHERxPy, VPGATHERxy) are also part of this group, and they have (among other things) two output operands. 3) the macros for 4-operand blends (which are under 0x0f 0x3a) have to be extended to support 2-operand blends. The 2-operand variant actually came a few years earlier, but it is clearer to implement them in the opposite order. X86_TYPE_WM, introduced earlier for unaligned loads, is reused for helpers that accept a Reg* but have a M argument. These three-byte opcodes also include AVX new instructions, for which the helpers were originally implemented by Paul Brook <paul@nowt.org>. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-14 19:52:44 +03:00
[0x17] = X86_OP_ENTRY3(VPTEST, None,None, V,x, W,x, vex4 cpuid(SSE41) p_66),
/*
* Source operand listed as Mq/Ux and similar in the manual; incorrectly listed
* as 128-bit only in 2-17.
*/
[0x20] = X86_OP_ENTRY3(VPMOVSXBW, V,x, None,None, W,q, vex5 cpuid(SSE41) avx_movx avx2_256 p_66),
[0x21] = X86_OP_ENTRY3(VPMOVSXBD, V,x, None,None, W,d, vex5 cpuid(SSE41) avx_movx avx2_256 p_66),
[0x22] = X86_OP_ENTRY3(VPMOVSXBQ, V,x, None,None, W,w, vex5 cpuid(SSE41) avx_movx avx2_256 p_66),
[0x23] = X86_OP_ENTRY3(VPMOVSXWD, V,x, None,None, W,q, vex5 cpuid(SSE41) avx_movx avx2_256 p_66),
[0x24] = X86_OP_ENTRY3(VPMOVSXWQ, V,x, None,None, W,d, vex5 cpuid(SSE41) avx_movx avx2_256 p_66),
[0x25] = X86_OP_ENTRY3(VPMOVSXDQ, V,x, None,None, W,q, vex5 cpuid(SSE41) avx_movx avx2_256 p_66),
/* Same as PMOVSX. */
[0x30] = X86_OP_ENTRY3(VPMOVZXBW, V,x, None,None, W,q, vex5 cpuid(SSE41) avx_movx avx2_256 p_66),
[0x31] = X86_OP_ENTRY3(VPMOVZXBD, V,x, None,None, W,d, vex5 cpuid(SSE41) avx_movx avx2_256 p_66),
[0x32] = X86_OP_ENTRY3(VPMOVZXBQ, V,x, None,None, W,w, vex5 cpuid(SSE41) avx_movx avx2_256 p_66),
[0x33] = X86_OP_ENTRY3(VPMOVZXWD, V,x, None,None, W,q, vex5 cpuid(SSE41) avx_movx avx2_256 p_66),
[0x34] = X86_OP_ENTRY3(VPMOVZXWQ, V,x, None,None, W,d, vex5 cpuid(SSE41) avx_movx avx2_256 p_66),
[0x35] = X86_OP_ENTRY3(VPMOVZXDQ, V,x, None,None, W,q, vex5 cpuid(SSE41) avx_movx avx2_256 p_66),
target/i386: validate VEX.W for AVX instructions Instructions in VEX exception class 6 generally look at the value of VEX.W. Note that the manual places some instructions incorrectly in class 4, for example VPERMQ which has no non-VEX encoding and no legacy SSE analogue. AMD does a mess of its own, as documented in the comment that this patch adds. Most of them are checked for VEX.W=0, and are listed in the manual (though with an omission) in table 2-16; VPERMQ and VPERMPD check for VEX.W=1, which is only listed in the instruction description. Others, such as VPSRLV, VPSLLV and the FMA3 instructions, use VEX.W to switch between a 32-bit and 64-bit operation. Fix more of the class 4/class 6 mismatches, and implement the check for VEX.W in TCG. Acked-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-09 19:16:27 +03:00
[0x36] = X86_OP_ENTRY3(VPERMD, V,qq, H,qq, W,qq, vex6 chk(W0) cpuid(AVX2) p_66),
target/i386: reimplement 0x0f 0x38, add AVX There are several special cases here: 1) extending moves have different widths for the helpers vs. for the memory loads, and the width for memory loads depends on VEX.L too. This is represented by X86_SPECIAL_AVXExtMov. 2) some instructions, such as variable-width shifts, select the vector element size via REX.W. 3) VSIB instructions (VGATHERxPy, VPGATHERxy) are also part of this group, and they have (among other things) two output operands. 3) the macros for 4-operand blends (which are under 0x0f 0x3a) have to be extended to support 2-operand blends. The 2-operand variant actually came a few years earlier, but it is clearer to implement them in the opposite order. X86_TYPE_WM, introduced earlier for unaligned loads, is reused for helpers that accept a Reg* but have a M argument. These three-byte opcodes also include AVX new instructions, for which the helpers were originally implemented by Paul Brook <paul@nowt.org>. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-14 19:52:44 +03:00
[0x37] = X86_OP_ENTRY3(PCMPGTQ, V,x, H,x, W,x, vex4 cpuid(SSE42) avx2_256 p_66),
[0x40] = X86_OP_ENTRY3(PMULLD, V,x, H,x, W,x, vex4 cpuid(SSE41) avx2_256 p_66),
[0x41] = X86_OP_ENTRY3(VPHMINPOSUW, V,dq, None,None, W,dq, vex4 cpuid(SSE41) p_66),
/* Listed incorrectly as type 4 */
[0x45] = X86_OP_ENTRY3(VPSRLV, V,x, H,x, W,x, vex6 cpuid(AVX2) p_66),
target/i386: validate VEX.W for AVX instructions Instructions in VEX exception class 6 generally look at the value of VEX.W. Note that the manual places some instructions incorrectly in class 4, for example VPERMQ which has no non-VEX encoding and no legacy SSE analogue. AMD does a mess of its own, as documented in the comment that this patch adds. Most of them are checked for VEX.W=0, and are listed in the manual (though with an omission) in table 2-16; VPERMQ and VPERMPD check for VEX.W=1, which is only listed in the instruction description. Others, such as VPSRLV, VPSLLV and the FMA3 instructions, use VEX.W to switch between a 32-bit and 64-bit operation. Fix more of the class 4/class 6 mismatches, and implement the check for VEX.W in TCG. Acked-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-09 19:16:27 +03:00
[0x46] = X86_OP_ENTRY3(VPSRAV, V,x, H,x, W,x, vex6 chk(W0) cpuid(AVX2) p_66),
target/i386: reimplement 0x0f 0x38, add AVX There are several special cases here: 1) extending moves have different widths for the helpers vs. for the memory loads, and the width for memory loads depends on VEX.L too. This is represented by X86_SPECIAL_AVXExtMov. 2) some instructions, such as variable-width shifts, select the vector element size via REX.W. 3) VSIB instructions (VGATHERxPy, VPGATHERxy) are also part of this group, and they have (among other things) two output operands. 3) the macros for 4-operand blends (which are under 0x0f 0x3a) have to be extended to support 2-operand blends. The 2-operand variant actually came a few years earlier, but it is clearer to implement them in the opposite order. X86_TYPE_WM, introduced earlier for unaligned loads, is reused for helpers that accept a Reg* but have a M argument. These three-byte opcodes also include AVX new instructions, for which the helpers were originally implemented by Paul Brook <paul@nowt.org>. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-14 19:52:44 +03:00
[0x47] = X86_OP_ENTRY3(VPSLLV, V,x, H,x, W,x, vex6 cpuid(AVX2) p_66),
[0x90] = X86_OP_ENTRY3(VPGATHERD, V,x, H,x, M,d, vex12 cpuid(AVX2) p_66), /* vpgatherdd/q */
[0x91] = X86_OP_ENTRY3(VPGATHERQ, V,x, H,x, M,q, vex12 cpuid(AVX2) p_66), /* vpgatherqd/q */
[0x92] = X86_OP_ENTRY3(VPGATHERD, V,x, H,x, M,d, vex12 cpuid(AVX2) p_66), /* vgatherdps/d */
[0x93] = X86_OP_ENTRY3(VPGATHERQ, V,x, H,x, M,q, vex12 cpuid(AVX2) p_66), /* vgatherqps/d */
target/i386: implement FMA instructions The only issue with FMA instructions is that there are _a lot_ of them (30 opcodes, each of which comes in up to 4 versions depending on VEX.W and VEX.L; a total of 96 possibilities). However, they can be implement with only 6 helpers, two for scalar operations and four for packed operations. (Scalar versions do not do any merging; they only affect the bottom 32 or 64 bits of the output operand. Therefore, there is no separate XMM and YMM of the scalar helpers). First, we can reduce the number of helpers to one third by passing four operands (one output and three inputs); the reordering of which operands go to the multiply and which go to the add is done in emit.c. Second, the different instructions also dispatch to the same softfloat function, so the flags for float32_muladd and float64_muladd are passed in the helper as int arguments, with a little extra complication to handle FMADDSUB and FMSUBADD. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-10-19 14:22:06 +03:00
/* Should be exception type 2 but they do not have legacy SSE equivalents? */
[0x96] = X86_OP_ENTRY3(VFMADDSUB132Px, V,x, H,x, W,x, vex6 cpuid(FMA) p_66),
[0x97] = X86_OP_ENTRY3(VFMSUBADD132Px, V,x, H,x, W,x, vex6 cpuid(FMA) p_66),
[0xa6] = X86_OP_ENTRY3(VFMADDSUB213Px, V,x, H,x, W,x, vex6 cpuid(FMA) p_66),
[0xa7] = X86_OP_ENTRY3(VFMSUBADD213Px, V,x, H,x, W,x, vex6 cpuid(FMA) p_66),
[0xb6] = X86_OP_ENTRY3(VFMADDSUB231Px, V,x, H,x, W,x, vex6 cpuid(FMA) p_66),
[0xb7] = X86_OP_ENTRY3(VFMSUBADD231Px, V,x, H,x, W,x, vex6 cpuid(FMA) p_66),
target/i386: reimplement 0x0f 0x38, add AVX There are several special cases here: 1) extending moves have different widths for the helpers vs. for the memory loads, and the width for memory loads depends on VEX.L too. This is represented by X86_SPECIAL_AVXExtMov. 2) some instructions, such as variable-width shifts, select the vector element size via REX.W. 3) VSIB instructions (VGATHERxPy, VPGATHERxy) are also part of this group, and they have (among other things) two output operands. 3) the macros for 4-operand blends (which are under 0x0f 0x3a) have to be extended to support 2-operand blends. The 2-operand variant actually came a few years earlier, but it is clearer to implement them in the opposite order. X86_TYPE_WM, introduced earlier for unaligned loads, is reused for helpers that accept a Reg* but have a M argument. These three-byte opcodes also include AVX new instructions, for which the helpers were originally implemented by Paul Brook <paul@nowt.org>. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-14 19:52:44 +03:00
[0x08] = X86_OP_ENTRY3(PSIGNB, V,x, H,x, W,x, vex4 cpuid(SSSE3) mmx avx2_256 p_00_66),
[0x09] = X86_OP_ENTRY3(PSIGNW, V,x, H,x, W,x, vex4 cpuid(SSSE3) mmx avx2_256 p_00_66),
[0x0a] = X86_OP_ENTRY3(PSIGND, V,x, H,x, W,x, vex4 cpuid(SSSE3) mmx avx2_256 p_00_66),
[0x0b] = X86_OP_ENTRY3(PMULHRSW, V,x, H,x, W,x, vex4 cpuid(SSSE3) mmx avx2_256 p_00_66),
target/i386: validate VEX.W for AVX instructions Instructions in VEX exception class 6 generally look at the value of VEX.W. Note that the manual places some instructions incorrectly in class 4, for example VPERMQ which has no non-VEX encoding and no legacy SSE analogue. AMD does a mess of its own, as documented in the comment that this patch adds. Most of them are checked for VEX.W=0, and are listed in the manual (though with an omission) in table 2-16; VPERMQ and VPERMPD check for VEX.W=1, which is only listed in the instruction description. Others, such as VPSRLV, VPSLLV and the FMA3 instructions, use VEX.W to switch between a 32-bit and 64-bit operation. Fix more of the class 4/class 6 mismatches, and implement the check for VEX.W in TCG. Acked-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-09 19:16:27 +03:00
/* Listed incorrectly as type 4 */
[0x0c] = X86_OP_ENTRY3(VPERMILPS, V,x, H,x, W,x, vex6 chk(W0) cpuid(AVX) p_00_66),
[0x0d] = X86_OP_ENTRY3(VPERMILPD, V,x, H,x, W,x, vex6 chk(W0) cpuid(AVX) p_66),
[0x0e] = X86_OP_ENTRY3(VTESTPS, None,None, V,x, W,x, vex6 chk(W0) cpuid(AVX) p_66),
[0x0f] = X86_OP_ENTRY3(VTESTPD, None,None, V,x, W,x, vex6 chk(W0) cpuid(AVX) p_66),
[0x18] = X86_OP_ENTRY3(VPBROADCASTD, V,x, None,None, W,d, vex6 chk(W0) cpuid(AVX) p_66), /* vbroadcastss */
[0x19] = X86_OP_ENTRY3(VPBROADCASTQ, V,qq, None,None, W,q, vex6 chk(W0) cpuid(AVX) p_66), /* vbroadcastsd */
[0x1a] = X86_OP_ENTRY3(VBROADCASTx128, V,qq, None,None, WM,dq,vex6 chk(W0) cpuid(AVX) p_66),
target/i386: reimplement 0x0f 0x38, add AVX There are several special cases here: 1) extending moves have different widths for the helpers vs. for the memory loads, and the width for memory loads depends on VEX.L too. This is represented by X86_SPECIAL_AVXExtMov. 2) some instructions, such as variable-width shifts, select the vector element size via REX.W. 3) VSIB instructions (VGATHERxPy, VPGATHERxy) are also part of this group, and they have (among other things) two output operands. 3) the macros for 4-operand blends (which are under 0x0f 0x3a) have to be extended to support 2-operand blends. The 2-operand variant actually came a few years earlier, but it is clearer to implement them in the opposite order. X86_TYPE_WM, introduced earlier for unaligned loads, is reused for helpers that accept a Reg* but have a M argument. These three-byte opcodes also include AVX new instructions, for which the helpers were originally implemented by Paul Brook <paul@nowt.org>. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-14 19:52:44 +03:00
[0x1c] = X86_OP_ENTRY3(PABSB, V,x, None,None, W,x, vex4 cpuid(SSSE3) mmx avx2_256 p_00_66),
[0x1d] = X86_OP_ENTRY3(PABSW, V,x, None,None, W,x, vex4 cpuid(SSSE3) mmx avx2_256 p_00_66),
[0x1e] = X86_OP_ENTRY3(PABSD, V,x, None,None, W,x, vex4 cpuid(SSSE3) mmx avx2_256 p_00_66),
[0x28] = X86_OP_ENTRY3(PMULDQ, V,x, H,x, W,x, vex4 cpuid(SSE41) avx2_256 p_66),
[0x29] = X86_OP_ENTRY3(PCMPEQQ, V,x, H,x, W,x, vex4 cpuid(SSE41) avx2_256 p_66),
[0x2a] = X86_OP_ENTRY3(MOVDQ, V,x, None,None, WM,x, vex1 cpuid(SSE41) avx2_256 p_66), /* movntdqa */
[0x2b] = X86_OP_ENTRY3(VPACKUSDW, V,x, H,x, W,x, vex4 cpuid(SSE41) avx2_256 p_66),
target/i386: validate VEX.W for AVX instructions Instructions in VEX exception class 6 generally look at the value of VEX.W. Note that the manual places some instructions incorrectly in class 4, for example VPERMQ which has no non-VEX encoding and no legacy SSE analogue. AMD does a mess of its own, as documented in the comment that this patch adds. Most of them are checked for VEX.W=0, and are listed in the manual (though with an omission) in table 2-16; VPERMQ and VPERMPD check for VEX.W=1, which is only listed in the instruction description. Others, such as VPSRLV, VPSLLV and the FMA3 instructions, use VEX.W to switch between a 32-bit and 64-bit operation. Fix more of the class 4/class 6 mismatches, and implement the check for VEX.W in TCG. Acked-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-09 19:16:27 +03:00
[0x2c] = X86_OP_ENTRY3(VMASKMOVPS, V,x, H,x, WM,x, vex6 chk(W0) cpuid(AVX) p_66),
[0x2d] = X86_OP_ENTRY3(VMASKMOVPD, V,x, H,x, WM,x, vex6 chk(W0) cpuid(AVX) p_66),
target/i386: reimplement 0x0f 0x38, add AVX There are several special cases here: 1) extending moves have different widths for the helpers vs. for the memory loads, and the width for memory loads depends on VEX.L too. This is represented by X86_SPECIAL_AVXExtMov. 2) some instructions, such as variable-width shifts, select the vector element size via REX.W. 3) VSIB instructions (VGATHERxPy, VPGATHERxy) are also part of this group, and they have (among other things) two output operands. 3) the macros for 4-operand blends (which are under 0x0f 0x3a) have to be extended to support 2-operand blends. The 2-operand variant actually came a few years earlier, but it is clearer to implement them in the opposite order. X86_TYPE_WM, introduced earlier for unaligned loads, is reused for helpers that accept a Reg* but have a M argument. These three-byte opcodes also include AVX new instructions, for which the helpers were originally implemented by Paul Brook <paul@nowt.org>. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-14 19:52:44 +03:00
/* Incorrectly listed as Mx,Hx,Vx in the manual */
target/i386: validate VEX.W for AVX instructions Instructions in VEX exception class 6 generally look at the value of VEX.W. Note that the manual places some instructions incorrectly in class 4, for example VPERMQ which has no non-VEX encoding and no legacy SSE analogue. AMD does a mess of its own, as documented in the comment that this patch adds. Most of them are checked for VEX.W=0, and are listed in the manual (though with an omission) in table 2-16; VPERMQ and VPERMPD check for VEX.W=1, which is only listed in the instruction description. Others, such as VPSRLV, VPSLLV and the FMA3 instructions, use VEX.W to switch between a 32-bit and 64-bit operation. Fix more of the class 4/class 6 mismatches, and implement the check for VEX.W in TCG. Acked-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-09 19:16:27 +03:00
[0x2e] = X86_OP_ENTRY3(VMASKMOVPS_st, M,x, V,x, H,x, vex6 chk(W0) cpuid(AVX) p_66),
[0x2f] = X86_OP_ENTRY3(VMASKMOVPD_st, M,x, V,x, H,x, vex6 chk(W0) cpuid(AVX) p_66),
target/i386: reimplement 0x0f 0x38, add AVX There are several special cases here: 1) extending moves have different widths for the helpers vs. for the memory loads, and the width for memory loads depends on VEX.L too. This is represented by X86_SPECIAL_AVXExtMov. 2) some instructions, such as variable-width shifts, select the vector element size via REX.W. 3) VSIB instructions (VGATHERxPy, VPGATHERxy) are also part of this group, and they have (among other things) two output operands. 3) the macros for 4-operand blends (which are under 0x0f 0x3a) have to be extended to support 2-operand blends. The 2-operand variant actually came a few years earlier, but it is clearer to implement them in the opposite order. X86_TYPE_WM, introduced earlier for unaligned loads, is reused for helpers that accept a Reg* but have a M argument. These three-byte opcodes also include AVX new instructions, for which the helpers were originally implemented by Paul Brook <paul@nowt.org>. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-14 19:52:44 +03:00
[0x38] = X86_OP_ENTRY3(PMINSB, V,x, H,x, W,x, vex4 cpuid(SSE41) avx2_256 p_66),
[0x39] = X86_OP_ENTRY3(PMINSD, V,x, H,x, W,x, vex4 cpuid(SSE41) avx2_256 p_66),
[0x3a] = X86_OP_ENTRY3(PMINUW, V,x, H,x, W,x, vex4 cpuid(SSE41) avx2_256 p_66),
[0x3b] = X86_OP_ENTRY3(PMINUD, V,x, H,x, W,x, vex4 cpuid(SSE41) avx2_256 p_66),
[0x3c] = X86_OP_ENTRY3(PMAXSB, V,x, H,x, W,x, vex4 cpuid(SSE41) avx2_256 p_66),
[0x3d] = X86_OP_ENTRY3(PMAXSD, V,x, H,x, W,x, vex4 cpuid(SSE41) avx2_256 p_66),
[0x3e] = X86_OP_ENTRY3(PMAXUW, V,x, H,x, W,x, vex4 cpuid(SSE41) avx2_256 p_66),
[0x3f] = X86_OP_ENTRY3(PMAXUD, V,x, H,x, W,x, vex4 cpuid(SSE41) avx2_256 p_66),
target/i386: validate VEX.W for AVX instructions Instructions in VEX exception class 6 generally look at the value of VEX.W. Note that the manual places some instructions incorrectly in class 4, for example VPERMQ which has no non-VEX encoding and no legacy SSE analogue. AMD does a mess of its own, as documented in the comment that this patch adds. Most of them are checked for VEX.W=0, and are listed in the manual (though with an omission) in table 2-16; VPERMQ and VPERMPD check for VEX.W=1, which is only listed in the instruction description. Others, such as VPSRLV, VPSLLV and the FMA3 instructions, use VEX.W to switch between a 32-bit and 64-bit operation. Fix more of the class 4/class 6 mismatches, and implement the check for VEX.W in TCG. Acked-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-09 19:16:27 +03:00
/* VPBROADCASTQ not listed as W0 in table 2-16 */
[0x58] = X86_OP_ENTRY3(VPBROADCASTD, V,x, None,None, W,d, vex6 chk(W0) cpuid(AVX2) p_66),
[0x59] = X86_OP_ENTRY3(VPBROADCASTQ, V,x, None,None, W,q, vex6 chk(W0) cpuid(AVX2) p_66),
[0x5a] = X86_OP_ENTRY3(VBROADCASTx128, V,qq, None,None, WM,dq,vex6 chk(W0) cpuid(AVX2) p_66),
target/i386: reimplement 0x0f 0x38, add AVX There are several special cases here: 1) extending moves have different widths for the helpers vs. for the memory loads, and the width for memory loads depends on VEX.L too. This is represented by X86_SPECIAL_AVXExtMov. 2) some instructions, such as variable-width shifts, select the vector element size via REX.W. 3) VSIB instructions (VGATHERxPy, VPGATHERxy) are also part of this group, and they have (among other things) two output operands. 3) the macros for 4-operand blends (which are under 0x0f 0x3a) have to be extended to support 2-operand blends. The 2-operand variant actually came a few years earlier, but it is clearer to implement them in the opposite order. X86_TYPE_WM, introduced earlier for unaligned loads, is reused for helpers that accept a Reg* but have a M argument. These three-byte opcodes also include AVX new instructions, for which the helpers were originally implemented by Paul Brook <paul@nowt.org>. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-14 19:52:44 +03:00
target/i386: validate VEX.W for AVX instructions Instructions in VEX exception class 6 generally look at the value of VEX.W. Note that the manual places some instructions incorrectly in class 4, for example VPERMQ which has no non-VEX encoding and no legacy SSE analogue. AMD does a mess of its own, as documented in the comment that this patch adds. Most of them are checked for VEX.W=0, and are listed in the manual (though with an omission) in table 2-16; VPERMQ and VPERMPD check for VEX.W=1, which is only listed in the instruction description. Others, such as VPSRLV, VPSLLV and the FMA3 instructions, use VEX.W to switch between a 32-bit and 64-bit operation. Fix more of the class 4/class 6 mismatches, and implement the check for VEX.W in TCG. Acked-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-09 19:16:27 +03:00
[0x78] = X86_OP_ENTRY3(VPBROADCASTB, V,x, None,None, W,b, vex6 chk(W0) cpuid(AVX2) p_66),
[0x79] = X86_OP_ENTRY3(VPBROADCASTW, V,x, None,None, W,w, vex6 chk(W0) cpuid(AVX2) p_66),
target/i386: reimplement 0x0f 0x38, add AVX There are several special cases here: 1) extending moves have different widths for the helpers vs. for the memory loads, and the width for memory loads depends on VEX.L too. This is represented by X86_SPECIAL_AVXExtMov. 2) some instructions, such as variable-width shifts, select the vector element size via REX.W. 3) VSIB instructions (VGATHERxPy, VPGATHERxy) are also part of this group, and they have (among other things) two output operands. 3) the macros for 4-operand blends (which are under 0x0f 0x3a) have to be extended to support 2-operand blends. The 2-operand variant actually came a few years earlier, but it is clearer to implement them in the opposite order. X86_TYPE_WM, introduced earlier for unaligned loads, is reused for helpers that accept a Reg* but have a M argument. These three-byte opcodes also include AVX new instructions, for which the helpers were originally implemented by Paul Brook <paul@nowt.org>. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-14 19:52:44 +03:00
[0x8c] = X86_OP_ENTRY3(VPMASKMOV, V,x, H,x, WM,x, vex6 cpuid(AVX2) p_66),
[0x8e] = X86_OP_ENTRY3(VPMASKMOV_st, M,x, V,x, H,x, vex6 cpuid(AVX2) p_66),
target/i386: implement FMA instructions The only issue with FMA instructions is that there are _a lot_ of them (30 opcodes, each of which comes in up to 4 versions depending on VEX.W and VEX.L; a total of 96 possibilities). However, they can be implement with only 6 helpers, two for scalar operations and four for packed operations. (Scalar versions do not do any merging; they only affect the bottom 32 or 64 bits of the output operand. Therefore, there is no separate XMM and YMM of the scalar helpers). First, we can reduce the number of helpers to one third by passing four operands (one output and three inputs); the reordering of which operands go to the multiply and which go to the add is done in emit.c. Second, the different instructions also dispatch to the same softfloat function, so the flags for float32_muladd and float64_muladd are passed in the helper as int arguments, with a little extra complication to handle FMADDSUB and FMSUBADD. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-10-19 14:22:06 +03:00
/* Should be exception type 2 or 3 but they do not have legacy SSE equivalents? */
[0x98] = X86_OP_ENTRY3(VFMADD132Px, V,x, H,x, W,x, vex6 cpuid(FMA) p_66),
[0x99] = X86_OP_ENTRY3(VFMADD132Sx, V,x, H,x, W,x, vex6 cpuid(FMA) p_66),
[0x9a] = X86_OP_ENTRY3(VFMSUB132Px, V,x, H,x, W,x, vex6 cpuid(FMA) p_66),
[0x9b] = X86_OP_ENTRY3(VFMSUB132Sx, V,x, H,x, W,x, vex6 cpuid(FMA) p_66),
[0x9c] = X86_OP_ENTRY3(VFNMADD132Px, V,x, H,x, W,x, vex6 cpuid(FMA) p_66),
[0x9d] = X86_OP_ENTRY3(VFNMADD132Sx, V,x, H,x, W,x, vex6 cpuid(FMA) p_66),
[0x9e] = X86_OP_ENTRY3(VFNMSUB132Px, V,x, H,x, W,x, vex6 cpuid(FMA) p_66),
[0x9f] = X86_OP_ENTRY3(VFNMSUB132Sx, V,x, H,x, W,x, vex6 cpuid(FMA) p_66),
[0xa8] = X86_OP_ENTRY3(VFMADD213Px, V,x, H,x, W,x, vex6 cpuid(FMA) p_66),
[0xa9] = X86_OP_ENTRY3(VFMADD213Sx, V,x, H,x, W,x, vex6 cpuid(FMA) p_66),
[0xaa] = X86_OP_ENTRY3(VFMSUB213Px, V,x, H,x, W,x, vex6 cpuid(FMA) p_66),
[0xab] = X86_OP_ENTRY3(VFMSUB213Sx, V,x, H,x, W,x, vex6 cpuid(FMA) p_66),
[0xac] = X86_OP_ENTRY3(VFNMADD213Px, V,x, H,x, W,x, vex6 cpuid(FMA) p_66),
[0xad] = X86_OP_ENTRY3(VFNMADD213Sx, V,x, H,x, W,x, vex6 cpuid(FMA) p_66),
[0xae] = X86_OP_ENTRY3(VFNMSUB213Px, V,x, H,x, W,x, vex6 cpuid(FMA) p_66),
[0xaf] = X86_OP_ENTRY3(VFNMSUB213Sx, V,x, H,x, W,x, vex6 cpuid(FMA) p_66),
[0xb8] = X86_OP_ENTRY3(VFMADD231Px, V,x, H,x, W,x, vex6 cpuid(FMA) p_66),
[0xb9] = X86_OP_ENTRY3(VFMADD231Sx, V,x, H,x, W,x, vex6 cpuid(FMA) p_66),
[0xba] = X86_OP_ENTRY3(VFMSUB231Px, V,x, H,x, W,x, vex6 cpuid(FMA) p_66),
[0xbb] = X86_OP_ENTRY3(VFMSUB231Sx, V,x, H,x, W,x, vex6 cpuid(FMA) p_66),
[0xbc] = X86_OP_ENTRY3(VFNMADD231Px, V,x, H,x, W,x, vex6 cpuid(FMA) p_66),
[0xbd] = X86_OP_ENTRY3(VFNMADD231Sx, V,x, H,x, W,x, vex6 cpuid(FMA) p_66),
[0xbe] = X86_OP_ENTRY3(VFNMSUB231Px, V,x, H,x, W,x, vex6 cpuid(FMA) p_66),
[0xbf] = X86_OP_ENTRY3(VFNMSUB231Sx, V,x, H,x, W,x, vex6 cpuid(FMA) p_66),
target/i386: implement SHA instructions The implementation was validated with OpenSSL and with the test vectors in https://github.com/rust-lang/stdarch/blob/master/crates/core_arch/src/x86/sha.rs. The instructions provide a ~25% improvement on hashing a 64 MiB file: runtime goes down from 1.8 seconds to 1.4 seconds; instruction count on the host goes down from 5.8 billion to 4.8 billion with slightly better IPC too. Good job Intel. ;) Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-10 11:31:17 +03:00
[0xc8] = X86_OP_ENTRY2(SHA1NEXTE, V,dq, W,dq, cpuid(SHA_NI)),
[0xc9] = X86_OP_ENTRY2(SHA1MSG1, V,dq, W,dq, cpuid(SHA_NI)),
[0xca] = X86_OP_ENTRY2(SHA1MSG2, V,dq, W,dq, cpuid(SHA_NI)),
[0xcb] = X86_OP_ENTRY2(SHA256RNDS2, V,dq, W,dq, cpuid(SHA_NI)),
[0xcc] = X86_OP_ENTRY2(SHA256MSG1, V,dq, W,dq, cpuid(SHA_NI)),
[0xcd] = X86_OP_ENTRY2(SHA256MSG2, V,dq, W,dq, cpuid(SHA_NI)),
target/i386: reimplement 0x0f 0x38, add AVX There are several special cases here: 1) extending moves have different widths for the helpers vs. for the memory loads, and the width for memory loads depends on VEX.L too. This is represented by X86_SPECIAL_AVXExtMov. 2) some instructions, such as variable-width shifts, select the vector element size via REX.W. 3) VSIB instructions (VGATHERxPy, VPGATHERxy) are also part of this group, and they have (among other things) two output operands. 3) the macros for 4-operand blends (which are under 0x0f 0x3a) have to be extended to support 2-operand blends. The 2-operand variant actually came a few years earlier, but it is clearer to implement them in the opposite order. X86_TYPE_WM, introduced earlier for unaligned loads, is reused for helpers that accept a Reg* but have a M argument. These three-byte opcodes also include AVX new instructions, for which the helpers were originally implemented by Paul Brook <paul@nowt.org>. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-14 19:52:44 +03:00
[0xdb] = X86_OP_ENTRY3(VAESIMC, V,dq, None,None, W,dq, vex4 cpuid(AES) p_66),
[0xdc] = X86_OP_ENTRY3(VAESENC, V,x, H,x, W,x, vex4 cpuid(AES) p_66),
[0xdd] = X86_OP_ENTRY3(VAESENCLAST, V,x, H,x, W,x, vex4 cpuid(AES) p_66),
[0xde] = X86_OP_ENTRY3(VAESDEC, V,x, H,x, W,x, vex4 cpuid(AES) p_66),
[0xdf] = X86_OP_ENTRY3(VAESDECLAST, V,x, H,x, W,x, vex4 cpuid(AES) p_66),
target/i386: implement CMPccXADD The main difficulty here is that a page fault when writing to the destination must not overwrite the flags. Therefore, the flags computation must be inlined instead of using gen_jcc1*. For simplicity, I am using an unconditional cmpxchg operation, that becomes a NOP if the comparison fails. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-10 11:31:39 +03:00
/*
* REG selects srcdest2 operand, VEX.vvvv selects src3. VEX class not found
* in manual, assumed to be 13 from the VEX.L0 constraint.
*/
[0xe0] = X86_OP_ENTRY3(CMPccXADD, M,y, G,y, B,y, vex13 xchg chk(o64) cpuid(CMPCCXADD) p_66),
[0xe1] = X86_OP_ENTRY3(CMPccXADD, M,y, G,y, B,y, vex13 xchg chk(o64) cpuid(CMPCCXADD) p_66),
[0xe2] = X86_OP_ENTRY3(CMPccXADD, M,y, G,y, B,y, vex13 xchg chk(o64) cpuid(CMPCCXADD) p_66),
[0xe3] = X86_OP_ENTRY3(CMPccXADD, M,y, G,y, B,y, vex13 xchg chk(o64) cpuid(CMPCCXADD) p_66),
[0xe4] = X86_OP_ENTRY3(CMPccXADD, M,y, G,y, B,y, vex13 xchg chk(o64) cpuid(CMPCCXADD) p_66),
[0xe5] = X86_OP_ENTRY3(CMPccXADD, M,y, G,y, B,y, vex13 xchg chk(o64) cpuid(CMPCCXADD) p_66),
[0xe6] = X86_OP_ENTRY3(CMPccXADD, M,y, G,y, B,y, vex13 xchg chk(o64) cpuid(CMPCCXADD) p_66),
[0xe7] = X86_OP_ENTRY3(CMPccXADD, M,y, G,y, B,y, vex13 xchg chk(o64) cpuid(CMPCCXADD) p_66),
[0xe8] = X86_OP_ENTRY3(CMPccXADD, M,y, G,y, B,y, vex13 xchg chk(o64) cpuid(CMPCCXADD) p_66),
[0xe9] = X86_OP_ENTRY3(CMPccXADD, M,y, G,y, B,y, vex13 xchg chk(o64) cpuid(CMPCCXADD) p_66),
[0xea] = X86_OP_ENTRY3(CMPccXADD, M,y, G,y, B,y, vex13 xchg chk(o64) cpuid(CMPCCXADD) p_66),
[0xeb] = X86_OP_ENTRY3(CMPccXADD, M,y, G,y, B,y, vex13 xchg chk(o64) cpuid(CMPCCXADD) p_66),
[0xec] = X86_OP_ENTRY3(CMPccXADD, M,y, G,y, B,y, vex13 xchg chk(o64) cpuid(CMPCCXADD) p_66),
[0xed] = X86_OP_ENTRY3(CMPccXADD, M,y, G,y, B,y, vex13 xchg chk(o64) cpuid(CMPCCXADD) p_66),
[0xee] = X86_OP_ENTRY3(CMPccXADD, M,y, G,y, B,y, vex13 xchg chk(o64) cpuid(CMPCCXADD) p_66),
[0xef] = X86_OP_ENTRY3(CMPccXADD, M,y, G,y, B,y, vex13 xchg chk(o64) cpuid(CMPCCXADD) p_66),
target/i386: add core of new i386 decoder The new decoder is based on three principles: - use mostly table-driven decoding, using tables derived as much as possible from the Intel manual. Centralizing the decode the operands makes it more homogeneous, for example all immediates are signed. All modrm handling is in one function, and can be shared between SSE and ALU instructions (including XMM<->GPR instructions). The SSE/AVX decoder will also not have duplicated code between the 0F, 0F38 and 0F3A tables. - keep the code as "non-branchy" as possible. Generally, the code for the new decoder is more verbose, but the control flow is simpler. Conditionals are not nested and have small bodies. All instruction groups are resolved even before operands are decoded, and code generation is separated as much as possible within small functions that only handle one instruction each. - keep address generation and (for ALU operands) memory loads and writeback as much in common code as possible. All ALU operations for example are implemented as T0=f(T0,T1). For non-ALU instructions, read-modify-write memory operations are rare, but registers do not have TCGv equivalents: therefore, the common logic sets up pointer temporaries with the operands, while load and writeback are handled by gvec or by helpers. These principles make future code review and extensibility simpler, at the cost of having a relatively large amount of code in the form of this patch. Even EVEX should not be _too_ hard to implement (it's just a crazy large amount of possibilities). This patch introduces the main decoder flow, and integrates the old decoder with the new one. The old decoder takes care of parsing prefixes and then optionally drops to the new one. The changes to the old decoder are minimal and allow it to be replaced incrementally with the new one. There is a debugging mechanism through a "LIMIT" environment variable. In user-mode emulation, the variable is the number of instructions decoded by the new decoder before permanently switching to the old one. In system emulation, the variable is the highest opcode that is decoded by the new decoder (this is less friendly, but it's the best that can be done without requiring deterministic execution). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-23 12:20:55 +03:00
};
/* five rows for no prefix, 66, F3, F2, 66+F2 */
static const X86OpEntry opcodes_0F38_F0toFF[16][5] = {
target/i386: move scalar 0F 38 and 0F 3A instruction to new decoder Because these are the only VEX instructions that QEMU supports, the new decoder is entered on the first byte of a valid VEX prefix, and VEX decoding only needs to be done in decode-new.c.inc. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-24 19:01:41 +03:00
[0] = {
target/i386: change X86_ENTRYwr to use T0, use it for moves Just like X86_ENTRYr, X86_ENTRYwr is easily changed to use only T0. In this case, the motivation is to use it for the MOV instruction family. The case when you need to preserve the input value is the odd one, as it is used basically only for BLS* instructions. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-08 12:06:50 +03:00
X86_OP_ENTRYwr(MOVBE, G,y, M,y, cpuid(MOVBE)),
X86_OP_ENTRYwr(MOVBE, G,w, M,w, cpuid(MOVBE)),
target/i386: move scalar 0F 38 and 0F 3A instruction to new decoder Because these are the only VEX instructions that QEMU supports, the new decoder is entered on the first byte of a valid VEX prefix, and VEX decoding only needs to be done in decode-new.c.inc. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-24 19:01:41 +03:00
{},
X86_OP_ENTRY2(CRC32, G,d, E,b, cpuid(SSE42)),
X86_OP_ENTRY2(CRC32, G,d, E,b, cpuid(SSE42)),
},
[1] = {
target/i386: change X86_ENTRYwr to use T0, use it for moves Just like X86_ENTRYr, X86_ENTRYwr is easily changed to use only T0. In this case, the motivation is to use it for the MOV instruction family. The case when you need to preserve the input value is the odd one, as it is used basically only for BLS* instructions. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-08 12:06:50 +03:00
X86_OP_ENTRYwr(MOVBE, M,y, G,y, cpuid(MOVBE)),
X86_OP_ENTRYwr(MOVBE, M,w, G,w, cpuid(MOVBE)),
target/i386: move scalar 0F 38 and 0F 3A instruction to new decoder Because these are the only VEX instructions that QEMU supports, the new decoder is entered on the first byte of a valid VEX prefix, and VEX decoding only needs to be done in decode-new.c.inc. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-24 19:01:41 +03:00
{},
X86_OP_ENTRY2(CRC32, G,d, E,y, cpuid(SSE42)),
X86_OP_ENTRY2(CRC32, G,d, E,w, cpuid(SSE42)),
},
[2] = {
X86_OP_ENTRY3(ANDN, G,y, B,y, E,y, vex13 cpuid(BMI1)),
{},
{},
{},
{},
},
[3] = {
target/i386: put BLS* input in T1, use generic flag writeback This makes for easier cpu_cc_* setup, and not using set_cc_op() should come in handy if QEMU ever implements APX. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-23 10:33:22 +03:00
X86_OP_GROUP3(group17, B,y, None,None, E,y, vex13 cpuid(BMI1)),
target/i386: move scalar 0F 38 and 0F 3A instruction to new decoder Because these are the only VEX instructions that QEMU supports, the new decoder is entered on the first byte of a valid VEX prefix, and VEX decoding only needs to be done in decode-new.c.inc. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-24 19:01:41 +03:00
{},
{},
{},
{},
},
[5] = {
X86_OP_ENTRY3(BZHI, G,y, E,y, B,y, vex13 cpuid(BMI1)),
{},
target/i386: add X86_SPECIALs for MOVSX and MOVZX Usually the registers are just moved into s->T0 without much care for their operand size. However, in some cases we can get more efficient code if the operand fetching logic syncs with the emission function on what is nicer. All the current uses are mostly demonstrative and only reduce the code in the emission functions, because the instructions do not support memory operands. However the logic is generic and applies to several more instructions such as MOVSXD (aka movslq), one-byte shift instructions, multiplications, XLAT, and indirect calls/jumps. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-19 16:40:54 +03:00
X86_OP_ENTRY3(PEXT, G,y, B,y, E,y, vex13 zextT0 cpuid(BMI2)),
X86_OP_ENTRY3(PDEP, G,y, B,y, E,y, vex13 zextT0 cpuid(BMI2)),
target/i386: move scalar 0F 38 and 0F 3A instruction to new decoder Because these are the only VEX instructions that QEMU supports, the new decoder is entered on the first byte of a valid VEX prefix, and VEX decoding only needs to be done in decode-new.c.inc. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-24 19:01:41 +03:00
{},
},
[6] = {
{},
X86_OP_ENTRY2(ADCX, G,y, E,y, cpuid(ADX)),
X86_OP_ENTRY2(ADOX, G,y, E,y, cpuid(ADX)),
X86_OP_ENTRY3(MULX, /* B,y, */ G,y, E,y, 2,y, vex13 cpuid(BMI2)),
{},
},
[7] = {
target/i386: add X86_SPECIALs for MOVSX and MOVZX Usually the registers are just moved into s->T0 without much care for their operand size. However, in some cases we can get more efficient code if the operand fetching logic syncs with the emission function on what is nicer. All the current uses are mostly demonstrative and only reduce the code in the emission functions, because the instructions do not support memory operands. However the logic is generic and applies to several more instructions such as MOVSXD (aka movslq), one-byte shift instructions, multiplications, XLAT, and indirect calls/jumps. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-19 16:40:54 +03:00
X86_OP_ENTRY3(BEXTR, G,y, E,y, B,y, vex13 zextT0 cpuid(BMI1)),
target/i386: move scalar 0F 38 and 0F 3A instruction to new decoder Because these are the only VEX instructions that QEMU supports, the new decoder is entered on the first byte of a valid VEX prefix, and VEX decoding only needs to be done in decode-new.c.inc. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-24 19:01:41 +03:00
X86_OP_ENTRY3(SHLX, G,y, E,y, B,y, vex13 cpuid(BMI1)),
target/i386: add X86_SPECIALs for MOVSX and MOVZX Usually the registers are just moved into s->T0 without much care for their operand size. However, in some cases we can get more efficient code if the operand fetching logic syncs with the emission function on what is nicer. All the current uses are mostly demonstrative and only reduce the code in the emission functions, because the instructions do not support memory operands. However the logic is generic and applies to several more instructions such as MOVSXD (aka movslq), one-byte shift instructions, multiplications, XLAT, and indirect calls/jumps. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-19 16:40:54 +03:00
X86_OP_ENTRY3(SARX, G,y, E,y, B,y, vex13 sextT0 cpuid(BMI1)),
X86_OP_ENTRY3(SHRX, G,y, E,y, B,y, vex13 zextT0 cpuid(BMI1)),
target/i386: move scalar 0F 38 and 0F 3A instruction to new decoder Because these are the only VEX instructions that QEMU supports, the new decoder is entered on the first byte of a valid VEX prefix, and VEX decoding only needs to be done in decode-new.c.inc. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-24 19:01:41 +03:00
{},
},
target/i386: add core of new i386 decoder The new decoder is based on three principles: - use mostly table-driven decoding, using tables derived as much as possible from the Intel manual. Centralizing the decode the operands makes it more homogeneous, for example all immediates are signed. All modrm handling is in one function, and can be shared between SSE and ALU instructions (including XMM<->GPR instructions). The SSE/AVX decoder will also not have duplicated code between the 0F, 0F38 and 0F3A tables. - keep the code as "non-branchy" as possible. Generally, the code for the new decoder is more verbose, but the control flow is simpler. Conditionals are not nested and have small bodies. All instruction groups are resolved even before operands are decoded, and code generation is separated as much as possible within small functions that only handle one instruction each. - keep address generation and (for ALU operands) memory loads and writeback as much in common code as possible. All ALU operations for example are implemented as T0=f(T0,T1). For non-ALU instructions, read-modify-write memory operations are rare, but registers do not have TCGv equivalents: therefore, the common logic sets up pointer temporaries with the operands, while load and writeback are handled by gvec or by helpers. These principles make future code review and extensibility simpler, at the cost of having a relatively large amount of code in the form of this patch. Even EVEX should not be _too_ hard to implement (it's just a crazy large amount of possibilities). This patch introduces the main decoder flow, and integrates the old decoder with the new one. The old decoder takes care of parsing prefixes and then optionally drops to the new one. The changes to the old decoder are minimal and allow it to be replaced incrementally with the new one. There is a debugging mechanism through a "LIMIT" environment variable. In user-mode emulation, the variable is the number of instructions decoded by the new decoder before permanently switching to the old one. In system emulation, the variable is the highest opcode that is decoded by the new decoder (this is less friendly, but it's the best that can be done without requiring deterministic execution). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-23 12:20:55 +03:00
};
static void decode_0F38(DisasContext *s, CPUX86State *env, X86OpEntry *entry, uint8_t *b)
{
*b = x86_ldub_code(env, s);
if (*b < 0xf0) {
*entry = opcodes_0F38_00toEF[*b];
} else {
int row = 0;
if (s->prefix & PREFIX_REPZ) {
/* The REPZ (F3) prefix has priority over 66 */
row = 2;
} else {
row += s->prefix & PREFIX_REPNZ ? 3 : 0;
row += s->prefix & PREFIX_DATA ? 1 : 0;
}
*entry = opcodes_0F38_F0toFF[*b & 15][row];
}
}
target/i386: reimplement 0x0f 0x3a, add AVX The more complicated operations here are insertions and extractions. Otherwise, there are just more entries than usual because the PS/PD/SS/SD variations are encoded in the opcode rater than in the prefixes. These three-byte opcodes also include AVX new instructions, whose implementation in the helpers was originally done by Paul Brook <paul@nowt.org>. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-06 11:34:11 +03:00
static void decode_VINSERTPS(DisasContext *s, CPUX86State *env, X86OpEntry *entry, uint8_t *b)
{
static const X86OpEntry
vinsertps_reg = X86_OP_ENTRY4(VINSERTPS_r, V,dq, H,dq, U,dq, vex5 cpuid(SSE41) p_66),
vinsertps_mem = X86_OP_ENTRY4(VINSERTPS_m, V,dq, H,dq, M,d, vex5 cpuid(SSE41) p_66);
int modrm = get_modrm(s, env);
*entry = (modrm >> 6) == 3 ? vinsertps_reg : vinsertps_mem;
}
target/i386: add core of new i386 decoder The new decoder is based on three principles: - use mostly table-driven decoding, using tables derived as much as possible from the Intel manual. Centralizing the decode the operands makes it more homogeneous, for example all immediates are signed. All modrm handling is in one function, and can be shared between SSE and ALU instructions (including XMM<->GPR instructions). The SSE/AVX decoder will also not have duplicated code between the 0F, 0F38 and 0F3A tables. - keep the code as "non-branchy" as possible. Generally, the code for the new decoder is more verbose, but the control flow is simpler. Conditionals are not nested and have small bodies. All instruction groups are resolved even before operands are decoded, and code generation is separated as much as possible within small functions that only handle one instruction each. - keep address generation and (for ALU operands) memory loads and writeback as much in common code as possible. All ALU operations for example are implemented as T0=f(T0,T1). For non-ALU instructions, read-modify-write memory operations are rare, but registers do not have TCGv equivalents: therefore, the common logic sets up pointer temporaries with the operands, while load and writeback are handled by gvec or by helpers. These principles make future code review and extensibility simpler, at the cost of having a relatively large amount of code in the form of this patch. Even EVEX should not be _too_ hard to implement (it's just a crazy large amount of possibilities). This patch introduces the main decoder flow, and integrates the old decoder with the new one. The old decoder takes care of parsing prefixes and then optionally drops to the new one. The changes to the old decoder are minimal and allow it to be replaced incrementally with the new one. There is a debugging mechanism through a "LIMIT" environment variable. In user-mode emulation, the variable is the number of instructions decoded by the new decoder before permanently switching to the old one. In system emulation, the variable is the highest opcode that is decoded by the new decoder (this is less friendly, but it's the best that can be done without requiring deterministic execution). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-23 12:20:55 +03:00
static const X86OpEntry opcodes_0F3A[256] = {
target/i386: reimplement 0x0f 0x3a, add AVX The more complicated operations here are insertions and extractions. Otherwise, there are just more entries than usual because the PS/PD/SS/SD variations are encoded in the opcode rater than in the prefixes. These three-byte opcodes also include AVX new instructions, whose implementation in the helpers was originally done by Paul Brook <paul@nowt.org>. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-06 11:34:11 +03:00
/*
* These are VEX-only, but incorrectly listed in the manual as exception type 4.
* Also the "qq" instructions are sometimes omitted by Table 2-17, but are VEX256
* only.
*/
target/i386: validate VEX.W for AVX instructions Instructions in VEX exception class 6 generally look at the value of VEX.W. Note that the manual places some instructions incorrectly in class 4, for example VPERMQ which has no non-VEX encoding and no legacy SSE analogue. AMD does a mess of its own, as documented in the comment that this patch adds. Most of them are checked for VEX.W=0, and are listed in the manual (though with an omission) in table 2-16; VPERMQ and VPERMPD check for VEX.W=1, which is only listed in the instruction description. Others, such as VPSRLV, VPSLLV and the FMA3 instructions, use VEX.W to switch between a 32-bit and 64-bit operation. Fix more of the class 4/class 6 mismatches, and implement the check for VEX.W in TCG. Acked-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-09 19:16:27 +03:00
[0x00] = X86_OP_ENTRY3(VPERMQ, V,qq, W,qq, I,b, vex6 chk(W1) cpuid(AVX2) p_66),
[0x01] = X86_OP_ENTRY3(VPERMQ, V,qq, W,qq, I,b, vex6 chk(W1) cpuid(AVX2) p_66), /* VPERMPD */
[0x02] = X86_OP_ENTRY4(VBLENDPS, V,x, H,x, W,x, vex6 chk(W0) cpuid(AVX2) p_66), /* VPBLENDD */
[0x04] = X86_OP_ENTRY3(VPERMILPS_i, V,x, W,x, I,b, vex6 chk(W0) cpuid(AVX) p_66),
[0x05] = X86_OP_ENTRY3(VPERMILPD_i, V,x, W,x, I,b, vex6 chk(W0) cpuid(AVX) p_66),
[0x06] = X86_OP_ENTRY4(VPERM2x128, V,qq, H,qq, W,qq, vex6 chk(W0) cpuid(AVX) p_66),
target/i386: reimplement 0x0f 0x3a, add AVX The more complicated operations here are insertions and extractions. Otherwise, there are just more entries than usual because the PS/PD/SS/SD variations are encoded in the opcode rater than in the prefixes. These three-byte opcodes also include AVX new instructions, whose implementation in the helpers was originally done by Paul Brook <paul@nowt.org>. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-06 11:34:11 +03:00
target/i386: rename zext0/zext2 and make them closer to the manual X86_SPECIAL_ZExtOp0 and X86_SPECIAL_ZExtOp2 are poorly named; they are a hack that is needed by scalar insertion and extraction instructions, and not really related to zero extension: for PEXTR the zero extension is done by the generation functions, for PINSR the high bits are not used at all and in fact are *not* filled with zeroes when loaded into s->T1. Rename the values to match the effect described in the manual, and explain better in the comments. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-20 00:14:34 +03:00
[0x14] = X86_OP_ENTRY3(PEXTRB, E,b, V,dq, I,b, vex5 cpuid(SSE41) op0_Rd p_66),
[0x15] = X86_OP_ENTRY3(PEXTRW, E,w, V,dq, I,b, vex5 cpuid(SSE41) op0_Rd p_66),
target/i386: reimplement 0x0f 0x3a, add AVX The more complicated operations here are insertions and extractions. Otherwise, there are just more entries than usual because the PS/PD/SS/SD variations are encoded in the opcode rater than in the prefixes. These three-byte opcodes also include AVX new instructions, whose implementation in the helpers was originally done by Paul Brook <paul@nowt.org>. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-06 11:34:11 +03:00
[0x16] = X86_OP_ENTRY3(PEXTR, E,y, V,dq, I,b, vex5 cpuid(SSE41) p_66),
[0x17] = X86_OP_ENTRY3(VEXTRACTPS, E,d, V,dq, I,b, vex5 cpuid(SSE41) p_66),
target/i386: validate VEX.W for AVX instructions Instructions in VEX exception class 6 generally look at the value of VEX.W. Note that the manual places some instructions incorrectly in class 4, for example VPERMQ which has no non-VEX encoding and no legacy SSE analogue. AMD does a mess of its own, as documented in the comment that this patch adds. Most of them are checked for VEX.W=0, and are listed in the manual (though with an omission) in table 2-16; VPERMQ and VPERMPD check for VEX.W=1, which is only listed in the instruction description. Others, such as VPSRLV, VPSLLV and the FMA3 instructions, use VEX.W to switch between a 32-bit and 64-bit operation. Fix more of the class 4/class 6 mismatches, and implement the check for VEX.W in TCG. Acked-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-09 19:16:27 +03:00
[0x1d] = X86_OP_ENTRY3(VCVTPS2PH, W,xh, V,x, I,b, vex11 chk(W0) cpuid(F16C) p_66),
target/i386: reimplement 0x0f 0x3a, add AVX The more complicated operations here are insertions and extractions. Otherwise, there are just more entries than usual because the PS/PD/SS/SD variations are encoded in the opcode rater than in the prefixes. These three-byte opcodes also include AVX new instructions, whose implementation in the helpers was originally done by Paul Brook <paul@nowt.org>. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-06 11:34:11 +03:00
target/i386: rename zext0/zext2 and make them closer to the manual X86_SPECIAL_ZExtOp0 and X86_SPECIAL_ZExtOp2 are poorly named; they are a hack that is needed by scalar insertion and extraction instructions, and not really related to zero extension: for PEXTR the zero extension is done by the generation functions, for PINSR the high bits are not used at all and in fact are *not* filled with zeroes when loaded into s->T1. Rename the values to match the effect described in the manual, and explain better in the comments. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-20 00:14:34 +03:00
[0x20] = X86_OP_ENTRY4(PINSRB, V,dq, H,dq, E,b, vex5 cpuid(SSE41) op2_Ry p_66),
target/i386: reimplement 0x0f 0x3a, add AVX The more complicated operations here are insertions and extractions. Otherwise, there are just more entries than usual because the PS/PD/SS/SD variations are encoded in the opcode rater than in the prefixes. These three-byte opcodes also include AVX new instructions, whose implementation in the helpers was originally done by Paul Brook <paul@nowt.org>. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-06 11:34:11 +03:00
[0x21] = X86_OP_GROUP0(VINSERTPS),
[0x22] = X86_OP_ENTRY4(PINSR, V,dq, H,dq, E,y, vex5 cpuid(SSE41) p_66),
[0x40] = X86_OP_ENTRY4(VDDPS, V,x, H,x, W,x, vex2 cpuid(SSE41) p_66),
[0x41] = X86_OP_ENTRY4(VDDPD, V,dq, H,dq, W,dq, vex2 cpuid(SSE41) p_66),
[0x42] = X86_OP_ENTRY4(VMPSADBW, V,x, H,x, W,x, vex2 cpuid(SSE41) avx2_256 p_66),
[0x44] = X86_OP_ENTRY4(PCLMULQDQ, V,dq, H,dq, W,dq, vex4 cpuid(PCLMULQDQ) p_66),
target/i386: validate VEX.W for AVX instructions Instructions in VEX exception class 6 generally look at the value of VEX.W. Note that the manual places some instructions incorrectly in class 4, for example VPERMQ which has no non-VEX encoding and no legacy SSE analogue. AMD does a mess of its own, as documented in the comment that this patch adds. Most of them are checked for VEX.W=0, and are listed in the manual (though with an omission) in table 2-16; VPERMQ and VPERMPD check for VEX.W=1, which is only listed in the instruction description. Others, such as VPSRLV, VPSLLV and the FMA3 instructions, use VEX.W to switch between a 32-bit and 64-bit operation. Fix more of the class 4/class 6 mismatches, and implement the check for VEX.W in TCG. Acked-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-09 19:16:27 +03:00
[0x46] = X86_OP_ENTRY4(VPERM2x128, V,qq, H,qq, W,qq, vex6 chk(W0) cpuid(AVX2) p_66),
target/i386: reimplement 0x0f 0x3a, add AVX The more complicated operations here are insertions and extractions. Otherwise, there are just more entries than usual because the PS/PD/SS/SD variations are encoded in the opcode rater than in the prefixes. These three-byte opcodes also include AVX new instructions, whose implementation in the helpers was originally done by Paul Brook <paul@nowt.org>. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-06 11:34:11 +03:00
[0x60] = X86_OP_ENTRY4(PCMPESTRM, None,None, V,dq, W,dq, vex4_unal cpuid(SSE42) p_66),
[0x61] = X86_OP_ENTRY4(PCMPESTRI, None,None, V,dq, W,dq, vex4_unal cpuid(SSE42) p_66),
[0x62] = X86_OP_ENTRY4(PCMPISTRM, None,None, V,dq, W,dq, vex4_unal cpuid(SSE42) p_66),
[0x63] = X86_OP_ENTRY4(PCMPISTRI, None,None, V,dq, W,dq, vex4_unal cpuid(SSE42) p_66),
[0x08] = X86_OP_ENTRY3(VROUNDPS, V,x, W,x, I,b, vex2 cpuid(SSE41) p_66),
[0x09] = X86_OP_ENTRY3(VROUNDPD, V,x, W,x, I,b, vex2 cpuid(SSE41) p_66),
/*
* Not listed as four operand in the manual. Also writes and reads 128-bits
* from the first two operands due to the V operand picking higher entries of
* the H operand; the "Vss,Hss,Wss" description from the manual is incorrect.
* For other unary operations such as VSQRTSx this is hidden by the "REPScalar"
* value of vex_special, because the table lists the operand types of VSQRTPx.
*/
[0x0a] = X86_OP_ENTRY4(VROUNDSS, V,x, H,x, W,ss, vex3 cpuid(SSE41) p_66),
[0x0b] = X86_OP_ENTRY4(VROUNDSD, V,x, H,x, W,sd, vex3 cpuid(SSE41) p_66),
[0x0c] = X86_OP_ENTRY4(VBLENDPS, V,x, H,x, W,x, vex4 cpuid(SSE41) p_66),
[0x0d] = X86_OP_ENTRY4(VBLENDPD, V,x, H,x, W,x, vex4 cpuid(SSE41) p_66),
target/i386: reimplement 0x0f 0x38, add AVX There are several special cases here: 1) extending moves have different widths for the helpers vs. for the memory loads, and the width for memory loads depends on VEX.L too. This is represented by X86_SPECIAL_AVXExtMov. 2) some instructions, such as variable-width shifts, select the vector element size via REX.W. 3) VSIB instructions (VGATHERxPy, VPGATHERxy) are also part of this group, and they have (among other things) two output operands. 3) the macros for 4-operand blends (which are under 0x0f 0x3a) have to be extended to support 2-operand blends. The 2-operand variant actually came a few years earlier, but it is clearer to implement them in the opposite order. X86_TYPE_WM, introduced earlier for unaligned loads, is reused for helpers that accept a Reg* but have a M argument. These three-byte opcodes also include AVX new instructions, for which the helpers were originally implemented by Paul Brook <paul@nowt.org>. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-14 19:52:44 +03:00
[0x0e] = X86_OP_ENTRY4(VPBLENDW, V,x, H,x, W,x, vex4 cpuid(SSE41) avx2_256 p_66),
[0x0f] = X86_OP_ENTRY4(PALIGNR, V,x, H,x, W,x, vex4 cpuid(SSSE3) mmx avx2_256 p_00_66),
target/i386: reimplement 0x0f 0x3a, add AVX The more complicated operations here are insertions and extractions. Otherwise, there are just more entries than usual because the PS/PD/SS/SD variations are encoded in the opcode rater than in the prefixes. These three-byte opcodes also include AVX new instructions, whose implementation in the helpers was originally done by Paul Brook <paul@nowt.org>. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-06 11:34:11 +03:00
target/i386: validate VEX.W for AVX instructions Instructions in VEX exception class 6 generally look at the value of VEX.W. Note that the manual places some instructions incorrectly in class 4, for example VPERMQ which has no non-VEX encoding and no legacy SSE analogue. AMD does a mess of its own, as documented in the comment that this patch adds. Most of them are checked for VEX.W=0, and are listed in the manual (though with an omission) in table 2-16; VPERMQ and VPERMPD check for VEX.W=1, which is only listed in the instruction description. Others, such as VPSRLV, VPSLLV and the FMA3 instructions, use VEX.W to switch between a 32-bit and 64-bit operation. Fix more of the class 4/class 6 mismatches, and implement the check for VEX.W in TCG. Acked-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-09 19:16:27 +03:00
[0x18] = X86_OP_ENTRY4(VINSERTx128, V,qq, H,qq, W,qq, vex6 chk(W0) cpuid(AVX) p_66),
[0x19] = X86_OP_ENTRY3(VEXTRACTx128, W,dq, V,qq, I,b, vex6 chk(W0) cpuid(AVX) p_66),
target/i386: reimplement 0x0f 0x3a, add AVX The more complicated operations here are insertions and extractions. Otherwise, there are just more entries than usual because the PS/PD/SS/SD variations are encoded in the opcode rater than in the prefixes. These three-byte opcodes also include AVX new instructions, whose implementation in the helpers was originally done by Paul Brook <paul@nowt.org>. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-06 11:34:11 +03:00
target/i386: validate VEX.W for AVX instructions Instructions in VEX exception class 6 generally look at the value of VEX.W. Note that the manual places some instructions incorrectly in class 4, for example VPERMQ which has no non-VEX encoding and no legacy SSE analogue. AMD does a mess of its own, as documented in the comment that this patch adds. Most of them are checked for VEX.W=0, and are listed in the manual (though with an omission) in table 2-16; VPERMQ and VPERMPD check for VEX.W=1, which is only listed in the instruction description. Others, such as VPSRLV, VPSLLV and the FMA3 instructions, use VEX.W to switch between a 32-bit and 64-bit operation. Fix more of the class 4/class 6 mismatches, and implement the check for VEX.W in TCG. Acked-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-09 19:16:27 +03:00
[0x38] = X86_OP_ENTRY4(VINSERTx128, V,qq, H,qq, W,qq, vex6 chk(W0) cpuid(AVX2) p_66),
[0x39] = X86_OP_ENTRY3(VEXTRACTx128, W,dq, V,qq, I,b, vex6 chk(W0) cpuid(AVX2) p_66),
target/i386: reimplement 0x0f 0x3a, add AVX The more complicated operations here are insertions and extractions. Otherwise, there are just more entries than usual because the PS/PD/SS/SD variations are encoded in the opcode rater than in the prefixes. These three-byte opcodes also include AVX new instructions, whose implementation in the helpers was originally done by Paul Brook <paul@nowt.org>. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-06 11:34:11 +03:00
/* Listed incorrectly as type 4 */
target/i386: validate VEX.W for AVX instructions Instructions in VEX exception class 6 generally look at the value of VEX.W. Note that the manual places some instructions incorrectly in class 4, for example VPERMQ which has no non-VEX encoding and no legacy SSE analogue. AMD does a mess of its own, as documented in the comment that this patch adds. Most of them are checked for VEX.W=0, and are listed in the manual (though with an omission) in table 2-16; VPERMQ and VPERMPD check for VEX.W=1, which is only listed in the instruction description. Others, such as VPSRLV, VPSLLV and the FMA3 instructions, use VEX.W to switch between a 32-bit and 64-bit operation. Fix more of the class 4/class 6 mismatches, and implement the check for VEX.W in TCG. Acked-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-09 19:16:27 +03:00
[0x4a] = X86_OP_ENTRY4(VBLENDVPS, V,x, H,x, W,x, vex6 chk(W0) cpuid(AVX) p_66),
[0x4b] = X86_OP_ENTRY4(VBLENDVPD, V,x, H,x, W,x, vex6 chk(W0) cpuid(AVX) p_66),
[0x4c] = X86_OP_ENTRY4(VPBLENDVB, V,x, H,x, W,x, vex6 chk(W0) cpuid(AVX) p_66 avx2_256),
target/i386: reimplement 0x0f 0x3a, add AVX The more complicated operations here are insertions and extractions. Otherwise, there are just more entries than usual because the PS/PD/SS/SD variations are encoded in the opcode rater than in the prefixes. These three-byte opcodes also include AVX new instructions, whose implementation in the helpers was originally done by Paul Brook <paul@nowt.org>. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-06 11:34:11 +03:00
target/i386: implement SHA instructions The implementation was validated with OpenSSL and with the test vectors in https://github.com/rust-lang/stdarch/blob/master/crates/core_arch/src/x86/sha.rs. The instructions provide a ~25% improvement on hashing a 64 MiB file: runtime goes down from 1.8 seconds to 1.4 seconds; instruction count on the host goes down from 5.8 billion to 4.8 billion with slightly better IPC too. Good job Intel. ;) Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-10 11:31:17 +03:00
[0xcc] = X86_OP_ENTRY3(SHA1RNDS4, V,dq, W,dq, I,b, cpuid(SHA_NI)),
target/i386: reimplement 0x0f 0x3a, add AVX The more complicated operations here are insertions and extractions. Otherwise, there are just more entries than usual because the PS/PD/SS/SD variations are encoded in the opcode rater than in the prefixes. These three-byte opcodes also include AVX new instructions, whose implementation in the helpers was originally done by Paul Brook <paul@nowt.org>. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-06 11:34:11 +03:00
[0xdf] = X86_OP_ENTRY3(VAESKEYGEN, V,dq, W,dq, I,b, vex4 cpuid(AES) p_66),
target/i386: move scalar 0F 38 and 0F 3A instruction to new decoder Because these are the only VEX instructions that QEMU supports, the new decoder is entered on the first byte of a valid VEX prefix, and VEX decoding only needs to be done in decode-new.c.inc. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-24 19:01:41 +03:00
[0xF0] = X86_OP_ENTRY3(RORX, G,y, E,y, I,b, vex13 cpuid(BMI2) p_f2),
target/i386: add core of new i386 decoder The new decoder is based on three principles: - use mostly table-driven decoding, using tables derived as much as possible from the Intel manual. Centralizing the decode the operands makes it more homogeneous, for example all immediates are signed. All modrm handling is in one function, and can be shared between SSE and ALU instructions (including XMM<->GPR instructions). The SSE/AVX decoder will also not have duplicated code between the 0F, 0F38 and 0F3A tables. - keep the code as "non-branchy" as possible. Generally, the code for the new decoder is more verbose, but the control flow is simpler. Conditionals are not nested and have small bodies. All instruction groups are resolved even before operands are decoded, and code generation is separated as much as possible within small functions that only handle one instruction each. - keep address generation and (for ALU operands) memory loads and writeback as much in common code as possible. All ALU operations for example are implemented as T0=f(T0,T1). For non-ALU instructions, read-modify-write memory operations are rare, but registers do not have TCGv equivalents: therefore, the common logic sets up pointer temporaries with the operands, while load and writeback are handled by gvec or by helpers. These principles make future code review and extensibility simpler, at the cost of having a relatively large amount of code in the form of this patch. Even EVEX should not be _too_ hard to implement (it's just a crazy large amount of possibilities). This patch introduces the main decoder flow, and integrates the old decoder with the new one. The old decoder takes care of parsing prefixes and then optionally drops to the new one. The changes to the old decoder are minimal and allow it to be replaced incrementally with the new one. There is a debugging mechanism through a "LIMIT" environment variable. In user-mode emulation, the variable is the number of instructions decoded by the new decoder before permanently switching to the old one. In system emulation, the variable is the highest opcode that is decoded by the new decoder (this is less friendly, but it's the best that can be done without requiring deterministic execution). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-23 12:20:55 +03:00
};
static void decode_0F3A(DisasContext *s, CPUX86State *env, X86OpEntry *entry, uint8_t *b)
{
*b = x86_ldub_code(env, s);
*entry = opcodes_0F3A[*b];
}
target/i386: reimplement 0x0f 0x10-0x17, add AVX These are mostly moves, and yet are a total pain. The main issue is that: 1) some instructions are selected by mod==11 (register operand) vs. mod=00/01/10 (memory operand) 2) stores to memory are two-operand operations, while the 3-register and load-from-memory versions operate on the entire contents of the destination; this makes it easier to separate the gen_* function for the store case 3) it's inefficient to load into xmm_T0 only to move the value out again, so the gen_* function for the load case is separated too The manual also has various mistakes in the operands here, for example the store case of MOVHPS operates on a 128-bit source (albeit discarding the bottom 64 bits) and therefore should be Mq,Vdq rather than Mq,Vq. Likewise for the destination and source of MOVHLPS. VUNPCK?PS and VUNPCK?PD are the same as VUNPCK?DQ and VUNPCK?QDQ, but encoded as prefixes rather than separate operands. The helpers can be reused however. For MOVSLDUP, MOVSHDUP and MOVDDUP I chose to reimplement them as helpers. I named the helper for MOVDDUP "movdldup" in preparation for possible future introduction of MOVDHDUP and to clarify the similarity with MOVSLDUP. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-18 00:22:36 +03:00
/*
* There are some mistakes in the operands in the manual, and the load/store/register
* cases are easiest to keep separate, so the entries for 10-17 follow simplicity and
* efficiency of implementation rather than copying what the manual says.
*
* In particular:
*
* 1) "VMOVSS m32, xmm1" and "VMOVSD m64, xmm1" do not support VEX.vvvv != 1111b,
* but this is not mentioned in the tables.
*
* 2) MOVHLPS, MOVHPS, MOVHPD, MOVLPD, MOVLPS read the high quadword of one of their
* operands, which must therefore be dq; MOVLPD and MOVLPS also write the high
* quadword of the V operand.
*/
static void decode_0F10(DisasContext *s, CPUX86State *env, X86OpEntry *entry, uint8_t *b)
{
static const X86OpEntry opcodes_0F10_reg[4] = {
X86_OP_ENTRY3(MOVDQ, V,x, None,None, W,x, vex4_unal), /* MOVUPS */
X86_OP_ENTRY3(MOVDQ, V,x, None,None, W,x, vex4_unal), /* MOVUPD */
target/i386: Fix exception classes for SSE/AVX instructions. Fix the exception classes for some SSE/AVX instructions to match what is documented in the Intel manual. These changes are expected to have no functional effect on the behavior that qemu implements (primarily >= 16-byte memory alignment checks). For instance, since qemu does not implement the AC flag, there is no difference in behavior between Exception Classes 4 and 5 for instructions where the SSE version only takes <16 byte memory operands. Message-Id: <20230501111428.95998-2-ricky@rzhou.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-05-01 14:14:27 +03:00
X86_OP_ENTRY3(VMOVSS, V,x, H,x, W,x, vex5),
X86_OP_ENTRY3(VMOVLPx, V,x, H,x, W,x, vex5), /* MOVSD */
target/i386: reimplement 0x0f 0x10-0x17, add AVX These are mostly moves, and yet are a total pain. The main issue is that: 1) some instructions are selected by mod==11 (register operand) vs. mod=00/01/10 (memory operand) 2) stores to memory are two-operand operations, while the 3-register and load-from-memory versions operate on the entire contents of the destination; this makes it easier to separate the gen_* function for the store case 3) it's inefficient to load into xmm_T0 only to move the value out again, so the gen_* function for the load case is separated too The manual also has various mistakes in the operands here, for example the store case of MOVHPS operates on a 128-bit source (albeit discarding the bottom 64 bits) and therefore should be Mq,Vdq rather than Mq,Vq. Likewise for the destination and source of MOVHLPS. VUNPCK?PS and VUNPCK?PD are the same as VUNPCK?DQ and VUNPCK?QDQ, but encoded as prefixes rather than separate operands. The helpers can be reused however. For MOVSLDUP, MOVSHDUP and MOVDDUP I chose to reimplement them as helpers. I named the helper for MOVDDUP "movdldup" in preparation for possible future introduction of MOVDHDUP and to clarify the similarity with MOVSLDUP. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-18 00:22:36 +03:00
};
static const X86OpEntry opcodes_0F10_mem[4] = {
X86_OP_ENTRY3(MOVDQ, V,x, None,None, W,x, vex4_unal), /* MOVUPS */
X86_OP_ENTRY3(MOVDQ, V,x, None,None, W,x, vex4_unal), /* MOVUPD */
target/i386: Fix exception classes for SSE/AVX instructions. Fix the exception classes for some SSE/AVX instructions to match what is documented in the Intel manual. These changes are expected to have no functional effect on the behavior that qemu implements (primarily >= 16-byte memory alignment checks). For instance, since qemu does not implement the AC flag, there is no difference in behavior between Exception Classes 4 and 5 for instructions where the SSE version only takes <16 byte memory operands. Message-Id: <20230501111428.95998-2-ricky@rzhou.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-05-01 14:14:27 +03:00
X86_OP_ENTRY3(VMOVSS_ld, V,x, H,x, M,ss, vex5),
X86_OP_ENTRY3(VMOVSD_ld, V,x, H,x, M,sd, vex5),
target/i386: reimplement 0x0f 0x10-0x17, add AVX These are mostly moves, and yet are a total pain. The main issue is that: 1) some instructions are selected by mod==11 (register operand) vs. mod=00/01/10 (memory operand) 2) stores to memory are two-operand operations, while the 3-register and load-from-memory versions operate on the entire contents of the destination; this makes it easier to separate the gen_* function for the store case 3) it's inefficient to load into xmm_T0 only to move the value out again, so the gen_* function for the load case is separated too The manual also has various mistakes in the operands here, for example the store case of MOVHPS operates on a 128-bit source (albeit discarding the bottom 64 bits) and therefore should be Mq,Vdq rather than Mq,Vq. Likewise for the destination and source of MOVHLPS. VUNPCK?PS and VUNPCK?PD are the same as VUNPCK?DQ and VUNPCK?QDQ, but encoded as prefixes rather than separate operands. The helpers can be reused however. For MOVSLDUP, MOVSHDUP and MOVDDUP I chose to reimplement them as helpers. I named the helper for MOVDDUP "movdldup" in preparation for possible future introduction of MOVDHDUP and to clarify the similarity with MOVSLDUP. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-18 00:22:36 +03:00
};
if ((get_modrm(s, env) >> 6) == 3) {
*entry = *decode_by_prefix(s, opcodes_0F10_reg);
} else {
*entry = *decode_by_prefix(s, opcodes_0F10_mem);
}
}
static void decode_0F11(DisasContext *s, CPUX86State *env, X86OpEntry *entry, uint8_t *b)
{
static const X86OpEntry opcodes_0F11_reg[4] = {
target/i386: Fix and add some comments next to SSE/AVX instructions. Adds some comments describing what instructions correspond to decoding table entries and fixes some existing comments which named the wrong instruction. Message-Id: <20230501111428.95998-1-ricky@rzhou.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-05-01 14:14:26 +03:00
X86_OP_ENTRY3(MOVDQ, W,x, None,None, V,x, vex4), /* MOVUPS */
X86_OP_ENTRY3(MOVDQ, W,x, None,None, V,x, vex4), /* MOVUPD */
target/i386: Fix exception classes for SSE/AVX instructions. Fix the exception classes for some SSE/AVX instructions to match what is documented in the Intel manual. These changes are expected to have no functional effect on the behavior that qemu implements (primarily >= 16-byte memory alignment checks). For instance, since qemu does not implement the AC flag, there is no difference in behavior between Exception Classes 4 and 5 for instructions where the SSE version only takes <16 byte memory operands. Message-Id: <20230501111428.95998-2-ricky@rzhou.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-05-01 14:14:27 +03:00
X86_OP_ENTRY3(VMOVSS, W,x, H,x, V,x, vex5),
X86_OP_ENTRY3(VMOVLPx, W,x, H,x, V,q, vex5), /* MOVSD */
target/i386: reimplement 0x0f 0x10-0x17, add AVX These are mostly moves, and yet are a total pain. The main issue is that: 1) some instructions are selected by mod==11 (register operand) vs. mod=00/01/10 (memory operand) 2) stores to memory are two-operand operations, while the 3-register and load-from-memory versions operate on the entire contents of the destination; this makes it easier to separate the gen_* function for the store case 3) it's inefficient to load into xmm_T0 only to move the value out again, so the gen_* function for the load case is separated too The manual also has various mistakes in the operands here, for example the store case of MOVHPS operates on a 128-bit source (albeit discarding the bottom 64 bits) and therefore should be Mq,Vdq rather than Mq,Vq. Likewise for the destination and source of MOVHLPS. VUNPCK?PS and VUNPCK?PD are the same as VUNPCK?DQ and VUNPCK?QDQ, but encoded as prefixes rather than separate operands. The helpers can be reused however. For MOVSLDUP, MOVSHDUP and MOVDDUP I chose to reimplement them as helpers. I named the helper for MOVDDUP "movdldup" in preparation for possible future introduction of MOVDHDUP and to clarify the similarity with MOVSLDUP. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-18 00:22:36 +03:00
};
static const X86OpEntry opcodes_0F11_mem[4] = {
target/i386: Fix and add some comments next to SSE/AVX instructions. Adds some comments describing what instructions correspond to decoding table entries and fixes some existing comments which named the wrong instruction. Message-Id: <20230501111428.95998-1-ricky@rzhou.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-05-01 14:14:26 +03:00
X86_OP_ENTRY3(MOVDQ, W,x, None,None, V,x, vex4), /* MOVUPS */
X86_OP_ENTRY3(MOVDQ, W,x, None,None, V,x, vex4), /* MOVUPD */
target/i386: Fix exception classes for SSE/AVX instructions. Fix the exception classes for some SSE/AVX instructions to match what is documented in the Intel manual. These changes are expected to have no functional effect on the behavior that qemu implements (primarily >= 16-byte memory alignment checks). For instance, since qemu does not implement the AC flag, there is no difference in behavior between Exception Classes 4 and 5 for instructions where the SSE version only takes <16 byte memory operands. Message-Id: <20230501111428.95998-2-ricky@rzhou.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-05-01 14:14:27 +03:00
X86_OP_ENTRY3(VMOVSS_st, M,ss, None,None, V,x, vex5),
X86_OP_ENTRY3(VMOVLPx_st, M,sd, None,None, V,x, vex5), /* MOVSD */
target/i386: reimplement 0x0f 0x10-0x17, add AVX These are mostly moves, and yet are a total pain. The main issue is that: 1) some instructions are selected by mod==11 (register operand) vs. mod=00/01/10 (memory operand) 2) stores to memory are two-operand operations, while the 3-register and load-from-memory versions operate on the entire contents of the destination; this makes it easier to separate the gen_* function for the store case 3) it's inefficient to load into xmm_T0 only to move the value out again, so the gen_* function for the load case is separated too The manual also has various mistakes in the operands here, for example the store case of MOVHPS operates on a 128-bit source (albeit discarding the bottom 64 bits) and therefore should be Mq,Vdq rather than Mq,Vq. Likewise for the destination and source of MOVHLPS. VUNPCK?PS and VUNPCK?PD are the same as VUNPCK?DQ and VUNPCK?QDQ, but encoded as prefixes rather than separate operands. The helpers can be reused however. For MOVSLDUP, MOVSHDUP and MOVDDUP I chose to reimplement them as helpers. I named the helper for MOVDDUP "movdldup" in preparation for possible future introduction of MOVDHDUP and to clarify the similarity with MOVSLDUP. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-18 00:22:36 +03:00
};
if ((get_modrm(s, env) >> 6) == 3) {
*entry = *decode_by_prefix(s, opcodes_0F11_reg);
} else {
*entry = *decode_by_prefix(s, opcodes_0F11_mem);
}
}
static void decode_0F12(DisasContext *s, CPUX86State *env, X86OpEntry *entry, uint8_t *b)
{
static const X86OpEntry opcodes_0F12_mem[4] = {
/*
* Use dq for operand for compatibility with gen_MOVSD and
* to allow VEX128 only.
*/
target/i386: Fix exception classes for SSE/AVX instructions. Fix the exception classes for some SSE/AVX instructions to match what is documented in the Intel manual. These changes are expected to have no functional effect on the behavior that qemu implements (primarily >= 16-byte memory alignment checks). For instance, since qemu does not implement the AC flag, there is no difference in behavior between Exception Classes 4 and 5 for instructions where the SSE version only takes <16 byte memory operands. Message-Id: <20230501111428.95998-2-ricky@rzhou.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-05-01 14:14:27 +03:00
X86_OP_ENTRY3(VMOVLPx_ld, V,dq, H,dq, M,q, vex5), /* MOVLPS */
X86_OP_ENTRY3(VMOVLPx_ld, V,dq, H,dq, M,q, vex5), /* MOVLPD */
target/i386: reimplement 0x0f 0x10-0x17, add AVX These are mostly moves, and yet are a total pain. The main issue is that: 1) some instructions are selected by mod==11 (register operand) vs. mod=00/01/10 (memory operand) 2) stores to memory are two-operand operations, while the 3-register and load-from-memory versions operate on the entire contents of the destination; this makes it easier to separate the gen_* function for the store case 3) it's inefficient to load into xmm_T0 only to move the value out again, so the gen_* function for the load case is separated too The manual also has various mistakes in the operands here, for example the store case of MOVHPS operates on a 128-bit source (albeit discarding the bottom 64 bits) and therefore should be Mq,Vdq rather than Mq,Vq. Likewise for the destination and source of MOVHLPS. VUNPCK?PS and VUNPCK?PD are the same as VUNPCK?DQ and VUNPCK?QDQ, but encoded as prefixes rather than separate operands. The helpers can be reused however. For MOVSLDUP, MOVSHDUP and MOVDDUP I chose to reimplement them as helpers. I named the helper for MOVDDUP "movdldup" in preparation for possible future introduction of MOVDHDUP and to clarify the similarity with MOVSLDUP. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-18 00:22:36 +03:00
X86_OP_ENTRY3(VMOVSLDUP, V,x, None,None, W,x, vex4 cpuid(SSE3)),
target/i386: Fix exception classes for SSE/AVX instructions. Fix the exception classes for some SSE/AVX instructions to match what is documented in the Intel manual. These changes are expected to have no functional effect on the behavior that qemu implements (primarily >= 16-byte memory alignment checks). For instance, since qemu does not implement the AC flag, there is no difference in behavior between Exception Classes 4 and 5 for instructions where the SSE version only takes <16 byte memory operands. Message-Id: <20230501111428.95998-2-ricky@rzhou.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-05-01 14:14:27 +03:00
X86_OP_ENTRY3(VMOVDDUP, V,x, None,None, WM,q, vex5 cpuid(SSE3)), /* qq if VEX.256 */
target/i386: reimplement 0x0f 0x10-0x17, add AVX These are mostly moves, and yet are a total pain. The main issue is that: 1) some instructions are selected by mod==11 (register operand) vs. mod=00/01/10 (memory operand) 2) stores to memory are two-operand operations, while the 3-register and load-from-memory versions operate on the entire contents of the destination; this makes it easier to separate the gen_* function for the store case 3) it's inefficient to load into xmm_T0 only to move the value out again, so the gen_* function for the load case is separated too The manual also has various mistakes in the operands here, for example the store case of MOVHPS operates on a 128-bit source (albeit discarding the bottom 64 bits) and therefore should be Mq,Vdq rather than Mq,Vq. Likewise for the destination and source of MOVHLPS. VUNPCK?PS and VUNPCK?PD are the same as VUNPCK?DQ and VUNPCK?QDQ, but encoded as prefixes rather than separate operands. The helpers can be reused however. For MOVSLDUP, MOVSHDUP and MOVDDUP I chose to reimplement them as helpers. I named the helper for MOVDDUP "movdldup" in preparation for possible future introduction of MOVDHDUP and to clarify the similarity with MOVSLDUP. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-18 00:22:36 +03:00
};
static const X86OpEntry opcodes_0F12_reg[4] = {
target/i386: Fix exception classes for SSE/AVX instructions. Fix the exception classes for some SSE/AVX instructions to match what is documented in the Intel manual. These changes are expected to have no functional effect on the behavior that qemu implements (primarily >= 16-byte memory alignment checks). For instance, since qemu does not implement the AC flag, there is no difference in behavior between Exception Classes 4 and 5 for instructions where the SSE version only takes <16 byte memory operands. Message-Id: <20230501111428.95998-2-ricky@rzhou.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-05-01 14:14:27 +03:00
X86_OP_ENTRY3(VMOVHLPS, V,dq, H,dq, U,dq, vex7),
X86_OP_ENTRY3(VMOVLPx, W,x, H,x, U,q, vex5), /* MOVLPD */
target/i386: reimplement 0x0f 0x10-0x17, add AVX These are mostly moves, and yet are a total pain. The main issue is that: 1) some instructions are selected by mod==11 (register operand) vs. mod=00/01/10 (memory operand) 2) stores to memory are two-operand operations, while the 3-register and load-from-memory versions operate on the entire contents of the destination; this makes it easier to separate the gen_* function for the store case 3) it's inefficient to load into xmm_T0 only to move the value out again, so the gen_* function for the load case is separated too The manual also has various mistakes in the operands here, for example the store case of MOVHPS operates on a 128-bit source (albeit discarding the bottom 64 bits) and therefore should be Mq,Vdq rather than Mq,Vq. Likewise for the destination and source of MOVHLPS. VUNPCK?PS and VUNPCK?PD are the same as VUNPCK?DQ and VUNPCK?QDQ, but encoded as prefixes rather than separate operands. The helpers can be reused however. For MOVSLDUP, MOVSHDUP and MOVDDUP I chose to reimplement them as helpers. I named the helper for MOVDDUP "movdldup" in preparation for possible future introduction of MOVDHDUP and to clarify the similarity with MOVSLDUP. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-18 00:22:36 +03:00
X86_OP_ENTRY3(VMOVSLDUP, V,x, None,None, U,x, vex4 cpuid(SSE3)),
target/i386: Fix exception classes for SSE/AVX instructions. Fix the exception classes for some SSE/AVX instructions to match what is documented in the Intel manual. These changes are expected to have no functional effect on the behavior that qemu implements (primarily >= 16-byte memory alignment checks). For instance, since qemu does not implement the AC flag, there is no difference in behavior between Exception Classes 4 and 5 for instructions where the SSE version only takes <16 byte memory operands. Message-Id: <20230501111428.95998-2-ricky@rzhou.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-05-01 14:14:27 +03:00
X86_OP_ENTRY3(VMOVDDUP, V,x, None,None, U,x, vex5 cpuid(SSE3)),
target/i386: reimplement 0x0f 0x10-0x17, add AVX These are mostly moves, and yet are a total pain. The main issue is that: 1) some instructions are selected by mod==11 (register operand) vs. mod=00/01/10 (memory operand) 2) stores to memory are two-operand operations, while the 3-register and load-from-memory versions operate on the entire contents of the destination; this makes it easier to separate the gen_* function for the store case 3) it's inefficient to load into xmm_T0 only to move the value out again, so the gen_* function for the load case is separated too The manual also has various mistakes in the operands here, for example the store case of MOVHPS operates on a 128-bit source (albeit discarding the bottom 64 bits) and therefore should be Mq,Vdq rather than Mq,Vq. Likewise for the destination and source of MOVHLPS. VUNPCK?PS and VUNPCK?PD are the same as VUNPCK?DQ and VUNPCK?QDQ, but encoded as prefixes rather than separate operands. The helpers can be reused however. For MOVSLDUP, MOVSHDUP and MOVDDUP I chose to reimplement them as helpers. I named the helper for MOVDDUP "movdldup" in preparation for possible future introduction of MOVDHDUP and to clarify the similarity with MOVSLDUP. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-18 00:22:36 +03:00
};
if ((get_modrm(s, env) >> 6) == 3) {
*entry = *decode_by_prefix(s, opcodes_0F12_reg);
} else {
*entry = *decode_by_prefix(s, opcodes_0F12_mem);
if ((s->prefix & PREFIX_REPNZ) && s->vex_l) {
entry->s2 = X86_SIZE_qq;
}
}
}
static void decode_0F16(DisasContext *s, CPUX86State *env, X86OpEntry *entry, uint8_t *b)
{
static const X86OpEntry opcodes_0F16_mem[4] = {
/*
* Operand 1 technically only reads the low 64 bits, but uses dq so that
* it is easier to check for op0 == op1 in an endianness-neutral manner.
*/
target/i386: Fix exception classes for SSE/AVX instructions. Fix the exception classes for some SSE/AVX instructions to match what is documented in the Intel manual. These changes are expected to have no functional effect on the behavior that qemu implements (primarily >= 16-byte memory alignment checks). For instance, since qemu does not implement the AC flag, there is no difference in behavior between Exception Classes 4 and 5 for instructions where the SSE version only takes <16 byte memory operands. Message-Id: <20230501111428.95998-2-ricky@rzhou.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-05-01 14:14:27 +03:00
X86_OP_ENTRY3(VMOVHPx_ld, V,dq, H,dq, M,q, vex5), /* MOVHPS */
X86_OP_ENTRY3(VMOVHPx_ld, V,dq, H,dq, M,q, vex5), /* MOVHPD */
target/i386: reimplement 0x0f 0x10-0x17, add AVX These are mostly moves, and yet are a total pain. The main issue is that: 1) some instructions are selected by mod==11 (register operand) vs. mod=00/01/10 (memory operand) 2) stores to memory are two-operand operations, while the 3-register and load-from-memory versions operate on the entire contents of the destination; this makes it easier to separate the gen_* function for the store case 3) it's inefficient to load into xmm_T0 only to move the value out again, so the gen_* function for the load case is separated too The manual also has various mistakes in the operands here, for example the store case of MOVHPS operates on a 128-bit source (albeit discarding the bottom 64 bits) and therefore should be Mq,Vdq rather than Mq,Vq. Likewise for the destination and source of MOVHLPS. VUNPCK?PS and VUNPCK?PD are the same as VUNPCK?DQ and VUNPCK?QDQ, but encoded as prefixes rather than separate operands. The helpers can be reused however. For MOVSLDUP, MOVSHDUP and MOVDDUP I chose to reimplement them as helpers. I named the helper for MOVDDUP "movdldup" in preparation for possible future introduction of MOVDHDUP and to clarify the similarity with MOVSLDUP. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-18 00:22:36 +03:00
X86_OP_ENTRY3(VMOVSHDUP, V,x, None,None, W,x, vex4 cpuid(SSE3)),
{},
};
static const X86OpEntry opcodes_0F16_reg[4] = {
/* Same as above, operand 1 could be Hq if it wasn't for big-endian. */
target/i386: Fix exception classes for SSE/AVX instructions. Fix the exception classes for some SSE/AVX instructions to match what is documented in the Intel manual. These changes are expected to have no functional effect on the behavior that qemu implements (primarily >= 16-byte memory alignment checks). For instance, since qemu does not implement the AC flag, there is no difference in behavior between Exception Classes 4 and 5 for instructions where the SSE version only takes <16 byte memory operands. Message-Id: <20230501111428.95998-2-ricky@rzhou.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-05-01 14:14:27 +03:00
X86_OP_ENTRY3(VMOVLHPS, V,dq, H,dq, U,q, vex7),
X86_OP_ENTRY3(VMOVHPx, V,x, H,x, U,x, vex5), /* MOVHPD */
target/i386: reimplement 0x0f 0x10-0x17, add AVX These are mostly moves, and yet are a total pain. The main issue is that: 1) some instructions are selected by mod==11 (register operand) vs. mod=00/01/10 (memory operand) 2) stores to memory are two-operand operations, while the 3-register and load-from-memory versions operate on the entire contents of the destination; this makes it easier to separate the gen_* function for the store case 3) it's inefficient to load into xmm_T0 only to move the value out again, so the gen_* function for the load case is separated too The manual also has various mistakes in the operands here, for example the store case of MOVHPS operates on a 128-bit source (albeit discarding the bottom 64 bits) and therefore should be Mq,Vdq rather than Mq,Vq. Likewise for the destination and source of MOVHLPS. VUNPCK?PS and VUNPCK?PD are the same as VUNPCK?DQ and VUNPCK?QDQ, but encoded as prefixes rather than separate operands. The helpers can be reused however. For MOVSLDUP, MOVSHDUP and MOVDDUP I chose to reimplement them as helpers. I named the helper for MOVDDUP "movdldup" in preparation for possible future introduction of MOVDHDUP and to clarify the similarity with MOVSLDUP. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-18 00:22:36 +03:00
X86_OP_ENTRY3(VMOVSHDUP, V,x, None,None, U,x, vex4 cpuid(SSE3)),
{},
};
if ((get_modrm(s, env) >> 6) == 3) {
*entry = *decode_by_prefix(s, opcodes_0F16_reg);
} else {
*entry = *decode_by_prefix(s, opcodes_0F16_mem);
}
}
target/i386: reimplement 0x0f 0x28-0x2f, add AVX Here the code is a bit uglier due to the truncation and extension of registers to and from 32-bit. There is also a mistake in the manual with respect to the size of the memory operand of CVTPS2PI and CVTTPS2PI, reported by Ricky Zhou. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-06 19:44:02 +03:00
static void decode_0F2A(DisasContext *s, CPUX86State *env, X86OpEntry *entry, uint8_t *b)
{
static const X86OpEntry opcodes_0F2A[4] = {
X86_OP_ENTRY3(CVTPI2Px, V,x, None,None, Q,q),
X86_OP_ENTRY3(CVTPI2Px, V,x, None,None, Q,q),
X86_OP_ENTRY3(VCVTSI2Sx, V,x, H,x, E,y, vex3),
X86_OP_ENTRY3(VCVTSI2Sx, V,x, H,x, E,y, vex3),
};
*entry = *decode_by_prefix(s, opcodes_0F2A);
}
static void decode_0F2B(DisasContext *s, CPUX86State *env, X86OpEntry *entry, uint8_t *b)
{
static const X86OpEntry opcodes_0F2B[4] = {
target/i386: Fix exception classes for MOVNTPS/MOVNTPD. Before this change, MOVNTPS and MOVNTPD were labeled as Exception Class 4 (only requiring alignment for legacy SSE instructions). This changes them to Exception Class 1 (always requiring memory alignment), as documented in the Intel manual. Message-Id: <20230501111428.95998-3-ricky@rzhou.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-05-01 14:14:28 +03:00
X86_OP_ENTRY3(MOVDQ, M,x, None,None, V,x, vex1), /* MOVNTPS */
X86_OP_ENTRY3(MOVDQ, M,x, None,None, V,x, vex1), /* MOVNTPD */
/* AMD extensions */
target/i386: reimplement 0x0f 0x28-0x2f, add AVX Here the code is a bit uglier due to the truncation and extension of registers to and from 32-bit. There is also a mistake in the manual with respect to the size of the memory operand of CVTPS2PI and CVTTPS2PI, reported by Ricky Zhou. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-06 19:44:02 +03:00
X86_OP_ENTRY3(VMOVSS_st, M,ss, None,None, V,x, vex4 cpuid(SSE4A)), /* MOVNTSS */
X86_OP_ENTRY3(VMOVLPx_st, M,sd, None,None, V,x, vex4 cpuid(SSE4A)), /* MOVNTSD */
};
*entry = *decode_by_prefix(s, opcodes_0F2B);
}
static void decode_0F2C(DisasContext *s, CPUX86State *env, X86OpEntry *entry, uint8_t *b)
{
static const X86OpEntry opcodes_0F2C[4] = {
/* Listed as ps/pd in the manual, but CVTTPS2PI only reads 64-bit. */
X86_OP_ENTRY3(CVTTPx2PI, P,q, None,None, W,q),
X86_OP_ENTRY3(CVTTPx2PI, P,q, None,None, W,dq),
X86_OP_ENTRY3(VCVTTSx2SI, G,y, None,None, W,ss, vex3),
X86_OP_ENTRY3(VCVTTSx2SI, G,y, None,None, W,sd, vex3),
};
*entry = *decode_by_prefix(s, opcodes_0F2C);
}
static void decode_0F2D(DisasContext *s, CPUX86State *env, X86OpEntry *entry, uint8_t *b)
{
static const X86OpEntry opcodes_0F2D[4] = {
/* Listed as ps/pd in the manual, but CVTPS2PI only reads 64-bit. */
X86_OP_ENTRY3(CVTPx2PI, P,q, None,None, W,q),
X86_OP_ENTRY3(CVTPx2PI, P,q, None,None, W,dq),
X86_OP_ENTRY3(VCVTSx2SI, G,y, None,None, W,ss, vex3),
X86_OP_ENTRY3(VCVTSx2SI, G,y, None,None, W,sd, vex3),
};
*entry = *decode_by_prefix(s, opcodes_0F2D);
}
target/i386: fix operand size for VCOMI/VUCOMI instructions Compared to other SSE instructions, VUCOMISx and VCOMISx are different: the single and double precision versions are distinguished through a prefix, however they use no-prefix and 0x66 for SS and SD respectively. Scalar values usually are associated with 0xF2 and 0xF3. Because of these, they incorrectly perform a 128-bit memory load instead of a 32- or 64-bit load. Fix this by writing a custom decoding function. I tested that the reproducer is fixed and the test-avx output does not change. Reported-by: Gabriele Svelto <gsvelto@mozilla.com> Resolves: https://gitlab.com/qemu-project/qemu/-/issues/1637 Fixes: f8d19eec0d53 ("target/i386: reimplement 0x0f 0x28-0x2f, add AVX", 2022-10-18) Cc: qemu-stable@nongnu.org Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-05-09 17:17:15 +03:00
static void decode_VxCOMISx(DisasContext *s, CPUX86State *env, X86OpEntry *entry, uint8_t *b)
{
/*
* VUCOMISx and VCOMISx are different and use no-prefix and 0x66 for SS and SD
* respectively. Scalar values usually are associated with 0xF2 and 0xF3, for
* which X86_VEX_REPScalar exists, but here it has to be decoded by hand.
*/
entry->s1 = entry->s2 = (s->prefix & PREFIX_DATA ? X86_SIZE_sd : X86_SIZE_ss);
entry->gen = (*b == 0x2E ? gen_VUCOMI : gen_VCOMI);
}
target/i386: reimplement 0x0f 0x50-0x5f, add AVX These are mostly floating-point SSE operations. The odd ones out are MOVMSK and CVTxx2yy, the others are straightforward. Unary operations are a bit special in AVX because they have 2 operands for PD/PS operands (VEX.vvvv must be 1111b), and 3 operands for SD/SS. They are handled using X86_OP_GROUP3 for compactness. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-01 15:27:55 +03:00
static void decode_sse_unary(DisasContext *s, CPUX86State *env, X86OpEntry *entry, uint8_t *b)
{
if (!(s->prefix & (PREFIX_REPZ | PREFIX_REPNZ))) {
entry->op1 = X86_TYPE_None;
entry->s1 = X86_SIZE_None;
}
switch (*b) {
case 0x51: entry->gen = gen_VSQRT; break;
case 0x52: entry->gen = gen_VRSQRT; break;
case 0x53: entry->gen = gen_VRCP; break;
}
}
target/i386: fix memory operand size for CVTPS2PD CVTPS2PD only loads a half-register for memory, unlike the other operations under 0x0F 0x5A. "Unpack" the group into separate emission functions instead of using gen_unary_fp_sse. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-08-29 19:28:33 +03:00
static void decode_0F5A(DisasContext *s, CPUX86State *env, X86OpEntry *entry, uint8_t *b)
{
static const X86OpEntry opcodes_0F5A[4] = {
X86_OP_ENTRY2(VCVTPS2PD, V,x, W,xh, vex2), /* VCVTPS2PD */
X86_OP_ENTRY2(VCVTPD2PS, V,x, W,x, vex2), /* VCVTPD2PS */
X86_OP_ENTRY3(VCVTSS2SD, V,x, H,x, W,x, vex2_rep3), /* VCVTSS2SD */
X86_OP_ENTRY3(VCVTSD2SS, V,x, H,x, W,x, vex2_rep3), /* VCVTSD2SS */
};
*entry = *decode_by_prefix(s, opcodes_0F5A);
}
target/i386: reimplement 0x0f 0x50-0x5f, add AVX These are mostly floating-point SSE operations. The odd ones out are MOVMSK and CVTxx2yy, the others are straightforward. Unary operations are a bit special in AVX because they have 2 operands for PD/PS operands (VEX.vvvv must be 1111b), and 3 operands for SD/SS. They are handled using X86_OP_GROUP3 for compactness. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-01 15:27:55 +03:00
static void decode_0F5B(DisasContext *s, CPUX86State *env, X86OpEntry *entry, uint8_t *b)
{
static const X86OpEntry opcodes_0F5B[4] = {
X86_OP_ENTRY2(VCVTDQ2PS, V,x, W,x, vex2),
X86_OP_ENTRY2(VCVTPS2DQ, V,x, W,x, vex2),
X86_OP_ENTRY2(VCVTTPS2DQ, V,x, W,x, vex2),
{},
};
*entry = *decode_by_prefix(s, opcodes_0F5B);
}
target/i386: reimplement 0x0f 0xd0-0xd7, 0xe0-0xe7, 0xf0-0xf7, add AVX The more complicated ones here are d6-d7, e6-e7, f7. The others are trivial. For LDDQU, using gen_load_sse directly might corrupt the register if the second part of the load fails. Therefore, add a custom X86_TYPE_WM value; like X86_TYPE_W it does call gen_load(), but it also rejects a value of 11 in the ModRM field like X86_TYPE_M. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-01 15:27:55 +03:00
static void decode_0FE6(DisasContext *s, CPUX86State *env, X86OpEntry *entry, uint8_t *b)
{
static const X86OpEntry opcodes_0FE6[4] = {
{},
X86_OP_ENTRY2(VCVTTPD2DQ, V,x, W,x, vex2),
target/i386: Fix exception classes for SSE/AVX instructions. Fix the exception classes for some SSE/AVX instructions to match what is documented in the Intel manual. These changes are expected to have no functional effect on the behavior that qemu implements (primarily >= 16-byte memory alignment checks). For instance, since qemu does not implement the AC flag, there is no difference in behavior between Exception Classes 4 and 5 for instructions where the SSE version only takes <16 byte memory operands. Message-Id: <20230501111428.95998-2-ricky@rzhou.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-05-01 14:14:27 +03:00
X86_OP_ENTRY2(VCVTDQ2PD, V,x, W,x, vex5),
target/i386: reimplement 0x0f 0xd0-0xd7, 0xe0-0xe7, 0xf0-0xf7, add AVX The more complicated ones here are d6-d7, e6-e7, f7. The others are trivial. For LDDQU, using gen_load_sse directly might corrupt the register if the second part of the load fails. Therefore, add a custom X86_TYPE_WM value; like X86_TYPE_W it does call gen_load(), but it also rejects a value of 11 in the ModRM field like X86_TYPE_M. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-01 15:27:55 +03:00
X86_OP_ENTRY2(VCVTPD2DQ, V,x, W,x, vex2),
};
*entry = *decode_by_prefix(s, opcodes_0FE6);
}
target/i386: convert MOV from/to CR and DR to new decoder Complete implementation of C and D operand types, then the operations are just MOVs. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-29 16:55:22 +03:00
/*
* These ignore the mod bits (assume (modrm&0xc0)==0xc0), so group the
* pre-decode tweak here for all MOVs from/to CR and DR.
*
* AMD documentation (24594.pdf) and testing of Intel 386 and 486
* processors all show that the mod bits are assumed to be 1's,
* regardless of actual values.
*/
static void decode_MOV_CR_DR(DisasContext *s, CPUX86State *env, X86OpEntry *entry, uint8_t *b)
{
/*
*/
get_modrm(s, env);
s->modrm |= 0xC0;
entry->gen = gen_MOV;
}
target/i386: add core of new i386 decoder The new decoder is based on three principles: - use mostly table-driven decoding, using tables derived as much as possible from the Intel manual. Centralizing the decode the operands makes it more homogeneous, for example all immediates are signed. All modrm handling is in one function, and can be shared between SSE and ALU instructions (including XMM<->GPR instructions). The SSE/AVX decoder will also not have duplicated code between the 0F, 0F38 and 0F3A tables. - keep the code as "non-branchy" as possible. Generally, the code for the new decoder is more verbose, but the control flow is simpler. Conditionals are not nested and have small bodies. All instruction groups are resolved even before operands are decoded, and code generation is separated as much as possible within small functions that only handle one instruction each. - keep address generation and (for ALU operands) memory loads and writeback as much in common code as possible. All ALU operations for example are implemented as T0=f(T0,T1). For non-ALU instructions, read-modify-write memory operations are rare, but registers do not have TCGv equivalents: therefore, the common logic sets up pointer temporaries with the operands, while load and writeback are handled by gvec or by helpers. These principles make future code review and extensibility simpler, at the cost of having a relatively large amount of code in the form of this patch. Even EVEX should not be _too_ hard to implement (it's just a crazy large amount of possibilities). This patch introduces the main decoder flow, and integrates the old decoder with the new one. The old decoder takes care of parsing prefixes and then optionally drops to the new one. The changes to the old decoder are minimal and allow it to be replaced incrementally with the new one. There is a debugging mechanism through a "LIMIT" environment variable. In user-mode emulation, the variable is the number of instructions decoded by the new decoder before permanently switching to the old one. In system emulation, the variable is the highest opcode that is decoded by the new decoder (this is less friendly, but it's the best that can be done without requiring deterministic execution). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-23 12:20:55 +03:00
static const X86OpEntry opcodes_0F[256] = {
target/i386: convert non-grouped, helper-based 2-byte opcodes These have very simple generators and no need for complex group decoding. Apart from LAR/LSL which are simplified to use gen_op_deposit_reg_v and movcond, the code is generally lifted from translate.c into the generators. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-25 11:49:26 +03:00
[0x02] = X86_OP_ENTRYwr(LAR, G,v, E,w, chk(prot)),
[0x03] = X86_OP_ENTRYwr(LSL, G,v, E,w, chk(prot)),
[0x05] = X86_OP_ENTRY0(SYSCALL, chk(o64_intel)),
[0x06] = X86_OP_ENTRY0(CLTS, chk(cpl0) svm(WRITE_CR0)),
[0x07] = X86_OP_ENTRY0(SYSRET, chk3(o64_intel, prot, cpl0)),
target/i386: reimplement 0x0f 0x10-0x17, add AVX These are mostly moves, and yet are a total pain. The main issue is that: 1) some instructions are selected by mod==11 (register operand) vs. mod=00/01/10 (memory operand) 2) stores to memory are two-operand operations, while the 3-register and load-from-memory versions operate on the entire contents of the destination; this makes it easier to separate the gen_* function for the store case 3) it's inefficient to load into xmm_T0 only to move the value out again, so the gen_* function for the load case is separated too The manual also has various mistakes in the operands here, for example the store case of MOVHPS operates on a 128-bit source (albeit discarding the bottom 64 bits) and therefore should be Mq,Vdq rather than Mq,Vq. Likewise for the destination and source of MOVHLPS. VUNPCK?PS and VUNPCK?PD are the same as VUNPCK?DQ and VUNPCK?QDQ, but encoded as prefixes rather than separate operands. The helpers can be reused however. For MOVSLDUP, MOVSHDUP and MOVDDUP I chose to reimplement them as helpers. I named the helper for MOVDDUP "movdldup" in preparation for possible future introduction of MOVDHDUP and to clarify the similarity with MOVSLDUP. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-18 00:22:36 +03:00
[0x10] = X86_OP_GROUP0(0F10),
[0x11] = X86_OP_GROUP0(0F11),
[0x12] = X86_OP_GROUP0(0F12),
target/i386: Fix exception classes for SSE/AVX instructions. Fix the exception classes for some SSE/AVX instructions to match what is documented in the Intel manual. These changes are expected to have no functional effect on the behavior that qemu implements (primarily >= 16-byte memory alignment checks). For instance, since qemu does not implement the AC flag, there is no difference in behavior between Exception Classes 4 and 5 for instructions where the SSE version only takes <16 byte memory operands. Message-Id: <20230501111428.95998-2-ricky@rzhou.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-05-01 14:14:27 +03:00
[0x13] = X86_OP_ENTRY3(VMOVLPx_st, M,q, None,None, V,q, vex5 p_00_66),
target/i386: reimplement 0x0f 0x10-0x17, add AVX These are mostly moves, and yet are a total pain. The main issue is that: 1) some instructions are selected by mod==11 (register operand) vs. mod=00/01/10 (memory operand) 2) stores to memory are two-operand operations, while the 3-register and load-from-memory versions operate on the entire contents of the destination; this makes it easier to separate the gen_* function for the store case 3) it's inefficient to load into xmm_T0 only to move the value out again, so the gen_* function for the load case is separated too The manual also has various mistakes in the operands here, for example the store case of MOVHPS operates on a 128-bit source (albeit discarding the bottom 64 bits) and therefore should be Mq,Vdq rather than Mq,Vq. Likewise for the destination and source of MOVHLPS. VUNPCK?PS and VUNPCK?PD are the same as VUNPCK?DQ and VUNPCK?QDQ, but encoded as prefixes rather than separate operands. The helpers can be reused however. For MOVSLDUP, MOVSHDUP and MOVDDUP I chose to reimplement them as helpers. I named the helper for MOVDDUP "movdldup" in preparation for possible future introduction of MOVDHDUP and to clarify the similarity with MOVSLDUP. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-18 00:22:36 +03:00
[0x14] = X86_OP_ENTRY3(VUNPCKLPx, V,x, H,x, W,x, vex4 p_00_66),
[0x15] = X86_OP_ENTRY3(VUNPCKHPx, V,x, H,x, W,x, vex4 p_00_66),
[0x16] = X86_OP_GROUP0(0F16),
/* Incorrectly listed as Mq,Vq in the manual */
target/i386: Fix exception classes for SSE/AVX instructions. Fix the exception classes for some SSE/AVX instructions to match what is documented in the Intel manual. These changes are expected to have no functional effect on the behavior that qemu implements (primarily >= 16-byte memory alignment checks). For instance, since qemu does not implement the AC flag, there is no difference in behavior between Exception Classes 4 and 5 for instructions where the SSE version only takes <16 byte memory operands. Message-Id: <20230501111428.95998-2-ricky@rzhou.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-05-01 14:14:27 +03:00
[0x17] = X86_OP_ENTRY3(VMOVHPx_st, M,q, None,None, V,dq, vex5 p_00_66),
target/i386: reimplement 0x0f 0x10-0x17, add AVX These are mostly moves, and yet are a total pain. The main issue is that: 1) some instructions are selected by mod==11 (register operand) vs. mod=00/01/10 (memory operand) 2) stores to memory are two-operand operations, while the 3-register and load-from-memory versions operate on the entire contents of the destination; this makes it easier to separate the gen_* function for the store case 3) it's inefficient to load into xmm_T0 only to move the value out again, so the gen_* function for the load case is separated too The manual also has various mistakes in the operands here, for example the store case of MOVHPS operates on a 128-bit source (albeit discarding the bottom 64 bits) and therefore should be Mq,Vdq rather than Mq,Vq. Likewise for the destination and source of MOVHLPS. VUNPCK?PS and VUNPCK?PD are the same as VUNPCK?DQ and VUNPCK?QDQ, but encoded as prefixes rather than separate operands. The helpers can be reused however. For MOVSLDUP, MOVSHDUP and MOVDDUP I chose to reimplement them as helpers. I named the helper for MOVDDUP "movdldup" in preparation for possible future introduction of MOVDHDUP and to clarify the similarity with MOVSLDUP. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-18 00:22:36 +03:00
target/i386: convert MOV from/to CR and DR to new decoder Complete implementation of C and D operand types, then the operations are just MOVs. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-29 16:55:22 +03:00
/*
* Incorrectly listed as using "d" operand type in the manual. In reality
* there's no 16-bit version (like y) and it does not use REX.W (like d64).
*/
[0x20] = X86_OP_GROUPwr(MOV_CR_DR, R,y_d64, C,y_d64, chk(cpl0) svm(READ_CR0)),
[0x21] = X86_OP_GROUPwr(MOV_CR_DR, R,y_d64, D,y_d64, chk(cpl0) svm(READ_DR0)),
[0x22] = X86_OP_GROUPwr(MOV_CR_DR, C,y_d64, R,y_d64, zextT0 chk(cpl0) svm(WRITE_CR0)),
[0x23] = X86_OP_GROUPwr(MOV_CR_DR, D,y_d64, R,y_d64, zextT0 chk(cpl0) svm(WRITE_DR0)),
target/i386: convert non-grouped, helper-based 2-byte opcodes These have very simple generators and no need for complex group decoding. Apart from LAR/LSL which are simplified to use gen_op_deposit_reg_v and movcond, the code is generally lifted from translate.c into the generators. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-25 11:49:26 +03:00
[0x30] = X86_OP_ENTRY0(WRMSR, chk(cpl0)),
[0x31] = X86_OP_ENTRY0(RDTSC),
[0x32] = X86_OP_ENTRY0(RDMSR, chk(cpl0)),
[0x33] = X86_OP_ENTRY0(RDPMC),
[0x34] = X86_OP_ENTRY0(SYSENTER, chk2(i64_amd, prot_or_vm86)),
[0x35] = X86_OP_ENTRY0(SYSEXIT, chk3(i64_amd, prot, cpl0)),
target/i386: move remaining conditional operations to new decoder Move long-displacement Jcc, SETcc and CMOVcc to the new decoder. While filling in the tables makes the code seem longer, the new emitters are all just one line of code. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-11 12:51:58 +03:00
[0x40] = X86_OP_ENTRY2(CMOVcc, G,v, E,v, cpuid(CMOV)),
[0x41] = X86_OP_ENTRY2(CMOVcc, G,v, E,v, cpuid(CMOV)),
[0x42] = X86_OP_ENTRY2(CMOVcc, G,v, E,v, cpuid(CMOV)),
[0x43] = X86_OP_ENTRY2(CMOVcc, G,v, E,v, cpuid(CMOV)),
[0x44] = X86_OP_ENTRY2(CMOVcc, G,v, E,v, cpuid(CMOV)),
[0x45] = X86_OP_ENTRY2(CMOVcc, G,v, E,v, cpuid(CMOV)),
[0x46] = X86_OP_ENTRY2(CMOVcc, G,v, E,v, cpuid(CMOV)),
[0x47] = X86_OP_ENTRY2(CMOVcc, G,v, E,v, cpuid(CMOV)),
target/i386: reimplement 0x0f 0x50-0x5f, add AVX These are mostly floating-point SSE operations. The odd ones out are MOVMSK and CVTxx2yy, the others are straightforward. Unary operations are a bit special in AVX because they have 2 operands for PD/PS operands (VEX.vvvv must be 1111b), and 3 operands for SD/SS. They are handled using X86_OP_GROUP3 for compactness. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-01 15:27:55 +03:00
[0x50] = X86_OP_ENTRY3(MOVMSK, G,y, None,None, U,x, vex7 p_00_66),
target/i386: Fix and add some comments next to SSE/AVX instructions. Adds some comments describing what instructions correspond to decoding table entries and fixes some existing comments which named the wrong instruction. Message-Id: <20230501111428.95998-1-ricky@rzhou.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-05-01 14:14:26 +03:00
[0x51] = X86_OP_GROUP3(sse_unary, V,x, H,x, W,x, vex2_rep3 p_00_66_f3_f2), /* sqrtps */
[0x52] = X86_OP_GROUP3(sse_unary, V,x, H,x, W,x, vex4_rep5 p_00_f3), /* rsqrtps */
[0x53] = X86_OP_GROUP3(sse_unary, V,x, H,x, W,x, vex4_rep5 p_00_f3), /* rcpps */
target/i386: reimplement 0x0f 0x50-0x5f, add AVX These are mostly floating-point SSE operations. The odd ones out are MOVMSK and CVTxx2yy, the others are straightforward. Unary operations are a bit special in AVX because they have 2 operands for PD/PS operands (VEX.vvvv must be 1111b), and 3 operands for SD/SS. They are handled using X86_OP_GROUP3 for compactness. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-01 15:27:55 +03:00
[0x54] = X86_OP_ENTRY3(PAND, V,x, H,x, W,x, vex4 p_00_66), /* vand */
[0x55] = X86_OP_ENTRY3(PANDN, V,x, H,x, W,x, vex4 p_00_66), /* vandn */
[0x56] = X86_OP_ENTRY3(POR, V,x, H,x, W,x, vex4 p_00_66), /* vor */
[0x57] = X86_OP_ENTRY3(PXOR, V,x, H,x, W,x, vex4 p_00_66), /* vxor */
target/i386: reimplement 0x0f 0x60-0x6f, add AVX These are both MMX and SSE/AVX instructions, except for vmovdqu. In both cases the inputs and output is in s->ptr{0,1,2}, so the only difference between MMX, SSE, and AVX is which helper to call. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-20 12:42:45 +03:00
[0x60] = X86_OP_ENTRY3(PUNPCKLBW, V,x, H,x, W,x, vex4 mmx avx2_256 p_00_66),
[0x61] = X86_OP_ENTRY3(PUNPCKLWD, V,x, H,x, W,x, vex4 mmx avx2_256 p_00_66),
[0x62] = X86_OP_ENTRY3(PUNPCKLDQ, V,x, H,x, W,x, vex4 mmx avx2_256 p_00_66),
[0x63] = X86_OP_ENTRY3(PACKSSWB, V,x, H,x, W,x, vex4 mmx avx2_256 p_00_66),
[0x64] = X86_OP_ENTRY3(PCMPGTB, V,x, H,x, W,x, vex4 mmx avx2_256 p_00_66),
[0x65] = X86_OP_ENTRY3(PCMPGTW, V,x, H,x, W,x, vex4 mmx avx2_256 p_00_66),
[0x66] = X86_OP_ENTRY3(PCMPGTD, V,x, H,x, W,x, vex4 mmx avx2_256 p_00_66),
[0x67] = X86_OP_ENTRY3(PACKUSWB, V,x, H,x, W,x, vex4 mmx avx2_256 p_00_66),
target/i386: reimplement 0x0f 0x70-0x77, add AVX This includes shifts by immediate, which use bits 3-5 of the ModRM byte as an opcode extension. With the exception of 128-bit shifts, they are implemented using gvec. This also covers VZEROALL and VZEROUPPER, which use the same opcode as EMMS. If we were wanting to optimize out gen_clear_ymmh then this would be one of the starting points. The implementation of the VZEROALL and VZEROUPPER helpers is by Paul Brook. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-02 19:19:06 +03:00
[0x70] = X86_OP_GROUP0(0F70),
[0x71] = X86_OP_GROUP0(group12),
[0x72] = X86_OP_GROUP0(group13),
[0x73] = X86_OP_GROUP0(group14),
[0x74] = X86_OP_ENTRY3(PCMPEQB, V,x, H,x, W,x, vex4 mmx avx2_256 p_00_66),
[0x75] = X86_OP_ENTRY3(PCMPEQW, V,x, H,x, W,x, vex4 mmx avx2_256 p_00_66),
[0x76] = X86_OP_ENTRY3(PCMPEQD, V,x, H,x, W,x, vex4 mmx avx2_256 p_00_66),
[0x77] = X86_OP_GROUP0(0F77),
target/i386: move remaining conditional operations to new decoder Move long-displacement Jcc, SETcc and CMOVcc to the new decoder. While filling in the tables makes the code seem longer, the new emitters are all just one line of code. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-11 12:51:58 +03:00
[0x80] = X86_OP_ENTRYr(Jcc, J,z_f64),
[0x81] = X86_OP_ENTRYr(Jcc, J,z_f64),
[0x82] = X86_OP_ENTRYr(Jcc, J,z_f64),
[0x83] = X86_OP_ENTRYr(Jcc, J,z_f64),
[0x84] = X86_OP_ENTRYr(Jcc, J,z_f64),
[0x85] = X86_OP_ENTRYr(Jcc, J,z_f64),
[0x86] = X86_OP_ENTRYr(Jcc, J,z_f64),
[0x87] = X86_OP_ENTRYr(Jcc, J,z_f64),
[0x90] = X86_OP_ENTRYw(SETcc, E,b),
[0x91] = X86_OP_ENTRYw(SETcc, E,b),
[0x92] = X86_OP_ENTRYw(SETcc, E,b),
[0x93] = X86_OP_ENTRYw(SETcc, E,b),
[0x94] = X86_OP_ENTRYw(SETcc, E,b),
[0x95] = X86_OP_ENTRYw(SETcc, E,b),
[0x96] = X86_OP_ENTRYw(SETcc, E,b),
[0x97] = X86_OP_ENTRYw(SETcc, E,b),
target/i386: port extensions of one-byte opcodes to new decoder A few two-byte opcodes are simple extensions of existing one-byte opcodes; they are easy to decode and need no change to emit.c.inc. Port them to the new decoder. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-21 18:51:23 +03:00
[0xa0] = X86_OP_ENTRYr(PUSH, FS, w),
[0xa1] = X86_OP_ENTRYw(POP, FS, w),
target/i386: convert non-grouped, helper-based 2-byte opcodes These have very simple generators and no need for complex group decoding. Apart from LAR/LSL which are simplified to use gen_op_deposit_reg_v and movcond, the code is generally lifted from translate.c into the generators. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-25 11:49:26 +03:00
[0xa2] = X86_OP_ENTRY0(CPUID),
target/i386: convert SHLD/SHRD to new decoder Use the same flag generation code as SHL and SHR, but use the existing gen_shiftd_rm_T1 function to compute the result as well as CC_SRC. Decoding-wise, SHLD/SHRD by immediate count as a 4 operand instruction because s->T0 and s->T1 actually occupy three op slots. The infrastructure used by opcodes in the 0F 3A table works fine. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-09 12:46:59 +03:00
[0xa4] = X86_OP_ENTRY4(SHLD, E,v, 2op,v, G,v),
[0xa5] = X86_OP_ENTRY3(SHLD, E,v, 2op,v, G,v),
target/i386: port extensions of one-byte opcodes to new decoder A few two-byte opcodes are simple extensions of existing one-byte opcodes; they are easy to decode and need no change to emit.c.inc. Port them to the new decoder. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-21 18:51:23 +03:00
target/i386: convert CMPXCHG to new decoder Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-09 15:41:19 +03:00
[0xb0] = X86_OP_ENTRY2(CMPXCHG,E,b, G,b, lock),
[0xb1] = X86_OP_ENTRY2(CMPXCHG,E,v, G,v, lock),
target/i386: fix bad sorting of entries in the 0F table Aesthetic change only. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-09 15:38:27 +03:00
[0xb2] = X86_OP_ENTRY3(LSS, G,v, EM,p, None, None),
[0xb4] = X86_OP_ENTRY3(LFS, G,v, EM,p, None, None),
[0xb5] = X86_OP_ENTRY3(LGS, G,v, EM,p, None, None),
[0xb6] = X86_OP_ENTRY3(MOV, G,v, E,b, None, None, zextT0), /* MOVZX */
[0xb7] = X86_OP_ENTRY3(MOV, G,v, E,w, None, None, zextT0), /* MOVZX */
target/i386: convert XADD to new decoder Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-14 17:40:32 +03:00
[0xc0] = X86_OP_ENTRY2(XADD, E,b, G,b, lock),
[0xc1] = X86_OP_ENTRY2(XADD, E,v, G,v, lock),
target/i386: fix bad sorting of entries in the 0F table Aesthetic change only. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-09 15:38:27 +03:00
[0xc2] = X86_OP_ENTRY4(VCMP, V,x, H,x, W,x, vex2_rep3 p_00_66_f3_f2),
[0xc3] = X86_OP_ENTRY3(MOV, EM,y,G,y, None,None, cpuid(SSE2)), /* MOVNTI */
[0xc4] = X86_OP_ENTRY4(PINSRW, V,dq,H,dq,E,w, vex5 mmx p_00_66),
[0xc5] = X86_OP_ENTRY3(PEXTRW, G,d, U,dq,I,b, vex5 mmx p_00_66),
[0xc6] = X86_OP_ENTRY4(VSHUF, V,x, H,x, W,x, vex4 p_00_66),
[0xd0] = X86_OP_ENTRY3(VADDSUB, V,x, H,x, W,x, vex2 cpuid(SSE3) p_66_f2),
[0xd1] = X86_OP_ENTRY3(PSRLW_r, V,x, H,x, W,x, vex4 mmx avx2_256 p_00_66),
[0xd2] = X86_OP_ENTRY3(PSRLD_r, V,x, H,x, W,x, vex4 mmx avx2_256 p_00_66),
[0xd3] = X86_OP_ENTRY3(PSRLQ_r, V,x, H,x, W,x, vex4 mmx avx2_256 p_00_66),
[0xd4] = X86_OP_ENTRY3(PADDQ, V,x, H,x, W,x, vex4 mmx avx2_256 p_00_66),
[0xd5] = X86_OP_ENTRY3(PMULLW, V,x, H,x, W,x, vex4 mmx avx2_256 p_00_66),
[0xd6] = X86_OP_GROUP0(0FD6),
[0xd7] = X86_OP_ENTRY3(PMOVMSKB, G,d, None,None, U,x, vex7 mmx avx2_256 p_00_66),
[0xe0] = X86_OP_ENTRY3(PAVGB, V,x, H,x, W,x, vex4 mmx avx2_256 p_00_66),
[0xe1] = X86_OP_ENTRY3(PSRAW_r, V,x, H,x, W,x, vex7 mmx avx2_256 p_00_66),
[0xe2] = X86_OP_ENTRY3(PSRAD_r, V,x, H,x, W,x, vex7 mmx avx2_256 p_00_66),
[0xe3] = X86_OP_ENTRY3(PAVGW, V,x, H,x, W,x, vex4 mmx avx2_256 p_00_66),
[0xe4] = X86_OP_ENTRY3(PMULHUW, V,x, H,x, W,x, vex4 mmx avx2_256 p_00_66),
[0xe5] = X86_OP_ENTRY3(PMULHW, V,x, H,x, W,x, vex4 mmx avx2_256 p_00_66),
[0xe6] = X86_OP_GROUP0(0FE6),
[0xe7] = X86_OP_ENTRY3(MOVDQ, W,x, None,None, V,x, vex1 mmx p_00_66), /* MOVNTQ/MOVNTDQ */
[0xf0] = X86_OP_ENTRY3(MOVDQ, V,x, None,None, WM,x, vex4_unal cpuid(SSE3) p_f2), /* LDDQU */
[0xf1] = X86_OP_ENTRY3(PSLLW_r, V,x, H,x, W,x, vex7 mmx avx2_256 p_00_66),
[0xf2] = X86_OP_ENTRY3(PSLLD_r, V,x, H,x, W,x, vex7 mmx avx2_256 p_00_66),
[0xf3] = X86_OP_ENTRY3(PSLLQ_r, V,x, H,x, W,x, vex7 mmx avx2_256 p_00_66),
[0xf4] = X86_OP_ENTRY3(PMULUDQ, V,x, H,x, W,x, vex4 mmx avx2_256 p_00_66),
[0xf5] = X86_OP_ENTRY3(PMADDWD, V,x, H,x, W,x, vex4 mmx avx2_256 p_00_66),
[0xf6] = X86_OP_ENTRY3(PSADBW, V,x, H,x, W,x, vex4 mmx avx2_256 p_00_66),
[0xf7] = X86_OP_ENTRY3(MASKMOV, None,None, V,dq, U,dq, vex4_unal avx2_256 mmx p_00_66),
target/i386: convert non-grouped, helper-based 2-byte opcodes These have very simple generators and no need for complex group decoding. Apart from LAR/LSL which are simplified to use gen_op_deposit_reg_v and movcond, the code is generally lifted from translate.c into the generators. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-25 11:49:26 +03:00
[0x08] = X86_OP_ENTRY0(NOP, svm(INVD)),
[0x09] = X86_OP_ENTRY0(NOP, svm(WBINVD)),
target/i386: move prefetch and multi-byte UD/NOP to new decoder These are trivial to add, and moving them to the new decoder fixes some corner cases: raising #UD instead of an instruction fetch page fault for the undefined opcodes, and incorrectly rejecting 0F 18 prefetches with register operands (which are treated as reserved NOPs). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Zhao Liu <zhao1.liu@intel.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-06 18:34:55 +03:00
[0x0b] = X86_OP_ENTRY0(UD), /* UD2 */
[0x0d] = X86_OP_ENTRY1(NOP, M,v), /* 3DNow! prefetch */
target/i386: fix bad sorting of entries in the 0F table Aesthetic change only. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-09 15:38:27 +03:00
[0x0e] = X86_OP_ENTRY0(EMMS, cpuid(3DNOW)), /* femms */
/*
* 3DNow!'s opcode byte comes *after* modrm and displacements, making it
* more like an Ib operand. Dispatch to the right helper in a single gen_*
* function.
*/
[0x0f] = X86_OP_ENTRY3(3dnow, P,q, Q,q, I,b, cpuid(3DNOW)),
target/i386: move prefetch and multi-byte UD/NOP to new decoder These are trivial to add, and moving them to the new decoder fixes some corner cases: raising #UD instead of an instruction fetch page fault for the undefined opcodes, and incorrectly rejecting 0F 18 prefetches with register operands (which are treated as reserved NOPs). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Zhao Liu <zhao1.liu@intel.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-06 18:34:55 +03:00
[0x18] = X86_OP_ENTRY1(NOP, nop,v), /* prefetch/reserved NOP */
[0x19] = X86_OP_ENTRY1(NOP, nop,v), /* reserved NOP */
[0x1c] = X86_OP_ENTRY1(NOP, nop,v), /* reserved NOP */
[0x1d] = X86_OP_ENTRY1(NOP, nop,v), /* reserved NOP */
[0x1e] = X86_OP_ENTRY1(NOP, nop,v), /* reserved NOP */
[0x1f] = X86_OP_ENTRY1(NOP, nop,v), /* NOP/reserved NOP */
target/i386: reimplement 0x0f 0x28-0x2f, add AVX Here the code is a bit uglier due to the truncation and extension of registers to and from 32-bit. There is also a mistake in the manual with respect to the size of the memory operand of CVTPS2PI and CVTTPS2PI, reported by Ricky Zhou. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-06 19:44:02 +03:00
[0x28] = X86_OP_ENTRY3(MOVDQ, V,x, None,None, W,x, vex1 p_00_66), /* MOVAPS */
[0x29] = X86_OP_ENTRY3(MOVDQ, W,x, None,None, V,x, vex1 p_00_66), /* MOVAPS */
[0x2A] = X86_OP_GROUP0(0F2A),
[0x2B] = X86_OP_GROUP0(0F2B),
[0x2C] = X86_OP_GROUP0(0F2C),
[0x2D] = X86_OP_GROUP0(0F2D),
target/i386: fix operand size for VCOMI/VUCOMI instructions Compared to other SSE instructions, VUCOMISx and VCOMISx are different: the single and double precision versions are distinguished through a prefix, however they use no-prefix and 0x66 for SS and SD respectively. Scalar values usually are associated with 0xF2 and 0xF3. Because of these, they incorrectly perform a 128-bit memory load instead of a 32- or 64-bit load. Fix this by writing a custom decoding function. I tested that the reproducer is fixed and the test-avx output does not change. Reported-by: Gabriele Svelto <gsvelto@mozilla.com> Resolves: https://gitlab.com/qemu-project/qemu/-/issues/1637 Fixes: f8d19eec0d53 ("target/i386: reimplement 0x0f 0x28-0x2f, add AVX", 2022-10-18) Cc: qemu-stable@nongnu.org Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-05-09 17:17:15 +03:00
[0x2E] = X86_OP_GROUP3(VxCOMISx, None,None, V,x, W,x, vex3 p_00_66), /* VUCOMISS/SD */
[0x2F] = X86_OP_GROUP3(VxCOMISx, None,None, V,x, W,x, vex3 p_00_66), /* VCOMISS/SD */
target/i386: reimplement 0x0f 0x28-0x2f, add AVX Here the code is a bit uglier due to the truncation and extension of registers to and from 32-bit. There is also a mistake in the manual with respect to the size of the memory operand of CVTPS2PI and CVTTPS2PI, reported by Ricky Zhou. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-06 19:44:02 +03:00
target/i386: add core of new i386 decoder The new decoder is based on three principles: - use mostly table-driven decoding, using tables derived as much as possible from the Intel manual. Centralizing the decode the operands makes it more homogeneous, for example all immediates are signed. All modrm handling is in one function, and can be shared between SSE and ALU instructions (including XMM<->GPR instructions). The SSE/AVX decoder will also not have duplicated code between the 0F, 0F38 and 0F3A tables. - keep the code as "non-branchy" as possible. Generally, the code for the new decoder is more verbose, but the control flow is simpler. Conditionals are not nested and have small bodies. All instruction groups are resolved even before operands are decoded, and code generation is separated as much as possible within small functions that only handle one instruction each. - keep address generation and (for ALU operands) memory loads and writeback as much in common code as possible. All ALU operations for example are implemented as T0=f(T0,T1). For non-ALU instructions, read-modify-write memory operations are rare, but registers do not have TCGv equivalents: therefore, the common logic sets up pointer temporaries with the operands, while load and writeback are handled by gvec or by helpers. These principles make future code review and extensibility simpler, at the cost of having a relatively large amount of code in the form of this patch. Even EVEX should not be _too_ hard to implement (it's just a crazy large amount of possibilities). This patch introduces the main decoder flow, and integrates the old decoder with the new one. The old decoder takes care of parsing prefixes and then optionally drops to the new one. The changes to the old decoder are minimal and allow it to be replaced incrementally with the new one. There is a debugging mechanism through a "LIMIT" environment variable. In user-mode emulation, the variable is the number of instructions decoded by the new decoder before permanently switching to the old one. In system emulation, the variable is the highest opcode that is decoded by the new decoder (this is less friendly, but it's the best that can be done without requiring deterministic execution). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-23 12:20:55 +03:00
[0x38] = X86_OP_GROUP0(0F38),
[0x3a] = X86_OP_GROUP0(0F3A),
target/i386: reimplement 0x0f 0x60-0x6f, add AVX These are both MMX and SSE/AVX instructions, except for vmovdqu. In both cases the inputs and output is in s->ptr{0,1,2}, so the only difference between MMX, SSE, and AVX is which helper to call. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-20 12:42:45 +03:00
target/i386: move remaining conditional operations to new decoder Move long-displacement Jcc, SETcc and CMOVcc to the new decoder. While filling in the tables makes the code seem longer, the new emitters are all just one line of code. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-11 12:51:58 +03:00
[0x48] = X86_OP_ENTRY2(CMOVcc, G,v, E,v, cpuid(CMOV)),
[0x49] = X86_OP_ENTRY2(CMOVcc, G,v, E,v, cpuid(CMOV)),
[0x4a] = X86_OP_ENTRY2(CMOVcc, G,v, E,v, cpuid(CMOV)),
[0x4b] = X86_OP_ENTRY2(CMOVcc, G,v, E,v, cpuid(CMOV)),
[0x4c] = X86_OP_ENTRY2(CMOVcc, G,v, E,v, cpuid(CMOV)),
[0x4d] = X86_OP_ENTRY2(CMOVcc, G,v, E,v, cpuid(CMOV)),
[0x4e] = X86_OP_ENTRY2(CMOVcc, G,v, E,v, cpuid(CMOV)),
[0x4f] = X86_OP_ENTRY2(CMOVcc, G,v, E,v, cpuid(CMOV)),
target/i386: reimplement 0x0f 0x50-0x5f, add AVX These are mostly floating-point SSE operations. The odd ones out are MOVMSK and CVTxx2yy, the others are straightforward. Unary operations are a bit special in AVX because they have 2 operands for PD/PS operands (VEX.vvvv must be 1111b), and 3 operands for SD/SS. They are handled using X86_OP_GROUP3 for compactness. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-01 15:27:55 +03:00
[0x58] = X86_OP_ENTRY3(VADD, V,x, H,x, W,x, vex2_rep3 p_00_66_f3_f2),
[0x59] = X86_OP_ENTRY3(VMUL, V,x, H,x, W,x, vex2_rep3 p_00_66_f3_f2),
target/i386: fix memory operand size for CVTPS2PD CVTPS2PD only loads a half-register for memory, unlike the other operations under 0x0F 0x5A. "Unpack" the group into separate emission functions instead of using gen_unary_fp_sse. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-08-29 19:28:33 +03:00
[0x5a] = X86_OP_GROUP0(0F5A),
target/i386: reimplement 0x0f 0x50-0x5f, add AVX These are mostly floating-point SSE operations. The odd ones out are MOVMSK and CVTxx2yy, the others are straightforward. Unary operations are a bit special in AVX because they have 2 operands for PD/PS operands (VEX.vvvv must be 1111b), and 3 operands for SD/SS. They are handled using X86_OP_GROUP3 for compactness. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-01 15:27:55 +03:00
[0x5b] = X86_OP_GROUP0(0F5B),
[0x5c] = X86_OP_ENTRY3(VSUB, V,x, H,x, W,x, vex2_rep3 p_00_66_f3_f2),
[0x5d] = X86_OP_ENTRY3(VMIN, V,x, H,x, W,x, vex2_rep3 p_00_66_f3_f2),
[0x5e] = X86_OP_ENTRY3(VDIV, V,x, H,x, W,x, vex2_rep3 p_00_66_f3_f2),
[0x5f] = X86_OP_ENTRY3(VMAX, V,x, H,x, W,x, vex2_rep3 p_00_66_f3_f2),
target/i386: reimplement 0x0f 0x60-0x6f, add AVX These are both MMX and SSE/AVX instructions, except for vmovdqu. In both cases the inputs and output is in s->ptr{0,1,2}, so the only difference between MMX, SSE, and AVX is which helper to call. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-20 12:42:45 +03:00
[0x68] = X86_OP_ENTRY3(PUNPCKHBW, V,x, H,x, W,x, vex4 mmx avx2_256 p_00_66),
[0x69] = X86_OP_ENTRY3(PUNPCKHWD, V,x, H,x, W,x, vex4 mmx avx2_256 p_00_66),
[0x6a] = X86_OP_ENTRY3(PUNPCKHDQ, V,x, H,x, W,x, vex4 mmx avx2_256 p_00_66),
[0x6b] = X86_OP_ENTRY3(PACKSSDW, V,x, H,x, W,x, vex4 mmx avx2_256 p_00_66),
[0x6c] = X86_OP_ENTRY3(PUNPCKLQDQ, V,x, H,x, W,x, vex4 p_66 avx2_256),
[0x6d] = X86_OP_ENTRY3(PUNPCKHQDQ, V,x, H,x, W,x, vex4 p_66 avx2_256),
[0x6e] = X86_OP_ENTRY3(MOVD_to, V,x, None,None, E,y, vex5 mmx p_00_66), /* wrong dest Vy on SDM! */
[0x6f] = X86_OP_GROUP0(0F6F),
target/i386: reimplement 0x0f 0xd8-0xdf, 0xe8-0xef, 0xf8-0xff, add AVX These are more simple integer instructions present in both MMX and SSE/AVX, with no holes that were later occupied by newer instructions. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-05 16:39:36 +03:00
target/i386: reimplement 0x0f 0x78-0x7f, add AVX These are a mixed batch, including the first two horizontal (66 and F2 only) operations, more moves, and SSE4a extract/insert. Because SSE4a is pretty rare, I chose to leave the helper as they are, but it is possible to unify them by loading index and length from the source XMM register and generating deposit or extract TCG ops. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-01 15:27:55 +03:00
[0x78] = X86_OP_GROUP0(0F78),
[0x79] = X86_OP_GROUP2(0F79, V,x, U,x, cpuid(SSE4A)),
[0x7c] = X86_OP_ENTRY3(VHADD, V,x, H,x, W,x, vex2 cpuid(SSE3) p_66_f2),
[0x7d] = X86_OP_ENTRY3(VHSUB, V,x, H,x, W,x, vex2 cpuid(SSE3) p_66_f2),
[0x7e] = X86_OP_GROUP0(0F7E),
[0x7f] = X86_OP_GROUP0(0F7F),
target/i386: move remaining conditional operations to new decoder Move long-displacement Jcc, SETcc and CMOVcc to the new decoder. While filling in the tables makes the code seem longer, the new emitters are all just one line of code. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-11 12:51:58 +03:00
[0x88] = X86_OP_ENTRYr(Jcc, J,z_f64),
[0x89] = X86_OP_ENTRYr(Jcc, J,z_f64),
[0x8a] = X86_OP_ENTRYr(Jcc, J,z_f64),
[0x8b] = X86_OP_ENTRYr(Jcc, J,z_f64),
[0x8c] = X86_OP_ENTRYr(Jcc, J,z_f64),
[0x8d] = X86_OP_ENTRYr(Jcc, J,z_f64),
[0x8e] = X86_OP_ENTRYr(Jcc, J,z_f64),
[0x8f] = X86_OP_ENTRYr(Jcc, J,z_f64),
[0x98] = X86_OP_ENTRYw(SETcc, E,b),
[0x99] = X86_OP_ENTRYw(SETcc, E,b),
[0x9a] = X86_OP_ENTRYw(SETcc, E,b),
[0x9b] = X86_OP_ENTRYw(SETcc, E,b),
[0x9c] = X86_OP_ENTRYw(SETcc, E,b),
[0x9d] = X86_OP_ENTRYw(SETcc, E,b),
[0x9e] = X86_OP_ENTRYw(SETcc, E,b),
[0x9f] = X86_OP_ENTRYw(SETcc, E,b),
target/i386: port extensions of one-byte opcodes to new decoder A few two-byte opcodes are simple extensions of existing one-byte opcodes; they are easy to decode and need no change to emit.c.inc. Port them to the new decoder. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-21 18:51:23 +03:00
[0xa8] = X86_OP_ENTRYr(PUSH, GS, w),
[0xa9] = X86_OP_ENTRYw(POP, GS, w),
target/i386: convert non-grouped, helper-based 2-byte opcodes These have very simple generators and no need for complex group decoding. Apart from LAR/LSL which are simplified to use gen_op_deposit_reg_v and movcond, the code is generally lifted from translate.c into the generators. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-25 11:49:26 +03:00
[0xaa] = X86_OP_ENTRY0(RSM, chk(smm) svm(RSM)),
target/i386: convert SHLD/SHRD to new decoder Use the same flag generation code as SHL and SHR, but use the existing gen_shiftd_rm_T1 function to compute the result as well as CC_SRC. Decoding-wise, SHLD/SHRD by immediate count as a 4 operand instruction because s->T0 and s->T1 actually occupy three op slots. The infrastructure used by opcodes in the 0F 3A table works fine. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-09 12:46:59 +03:00
[0xac] = X86_OP_ENTRY4(SHRD, E,v, 2op,v, G,v),
[0xad] = X86_OP_ENTRY3(SHRD, E,v, 2op,v, G,v),
target/i386: implement VLDMXCSR/VSTMXCSR These are exactly the same as the non-VEX version, but one has to be careful that only VEX.L=0 is allowed. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-11 14:22:32 +03:00
[0xae] = X86_OP_GROUP0(group15),
target/i386: port extensions of one-byte opcodes to new decoder A few two-byte opcodes are simple extensions of existing one-byte opcodes; they are easy to decode and need no change to emit.c.inc. Port them to the new decoder. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-21 18:51:23 +03:00
/*
* It's slightly more efficient to put Ev operand in T0 and allow gen_IMUL3
* to assume sextT0. Multiplication is commutative anyway.
*/
[0xaf] = X86_OP_ENTRY3(IMUL3, G,v, E,v, 2op,v, sextT0),
target/i386: convert LZCNT/TZCNT/BSF/BSR/POPCNT to new decoder Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-09 16:11:41 +03:00
[0xb8] = X86_OP_GROUP0(0FB8),
target/i386: move prefetch and multi-byte UD/NOP to new decoder These are trivial to add, and moving them to the new decoder fixes some corner cases: raising #UD instead of an instruction fetch page fault for the undefined opcodes, and incorrectly rejecting 0F 18 prefetches with register operands (which are treated as reserved NOPs). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Zhao Liu <zhao1.liu@intel.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-06 18:34:55 +03:00
/* decoded as modrm, which is visible as a difference between page fault and #UD */
[0xb9] = X86_OP_ENTRYr(UD, nop,v), /* UD1 */
target/i386: convert LZCNT/TZCNT/BSF/BSR/POPCNT to new decoder Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-09 16:11:41 +03:00
[0xbc] = X86_OP_GROUP0(0FBC),
[0xbd] = X86_OP_GROUP0(0FBD),
target/i386: port extensions of one-byte opcodes to new decoder A few two-byte opcodes are simple extensions of existing one-byte opcodes; they are easy to decode and need no change to emit.c.inc. Port them to the new decoder. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-21 18:51:23 +03:00
[0xbe] = X86_OP_ENTRY3(MOV, G,v, E,b, None, None, sextT0), /* MOVSX */
[0xbf] = X86_OP_ENTRY3(MOV, G,v, E,w, None, None, sextT0), /* MOVSX */
target/i386: implement VLDMXCSR/VSTMXCSR These are exactly the same as the non-VEX version, but one has to be careful that only VEX.L=0 is allowed. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-11 14:22:32 +03:00
target/i386: move BSWAP to new decoder Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-13 09:49:04 +03:00
[0xc8] = X86_OP_ENTRY1(BSWAP, LoBits,y),
[0xc9] = X86_OP_ENTRY1(BSWAP, LoBits,y),
[0xca] = X86_OP_ENTRY1(BSWAP, LoBits,y),
[0xcb] = X86_OP_ENTRY1(BSWAP, LoBits,y),
[0xcc] = X86_OP_ENTRY1(BSWAP, LoBits,y),
[0xcd] = X86_OP_ENTRY1(BSWAP, LoBits,y),
[0xce] = X86_OP_ENTRY1(BSWAP, LoBits,y),
[0xcf] = X86_OP_ENTRY1(BSWAP, LoBits,y),
target/i386: reimplement 0x0f 0xd8-0xdf, 0xe8-0xef, 0xf8-0xff, add AVX These are more simple integer instructions present in both MMX and SSE/AVX, with no holes that were later occupied by newer instructions. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-05 16:39:36 +03:00
/* Incorrectly missing from 2-17 */
[0xd8] = X86_OP_ENTRY3(PSUBUSB, V,x, H,x, W,x, vex4 mmx avx2_256 p_00_66),
[0xd9] = X86_OP_ENTRY3(PSUBUSW, V,x, H,x, W,x, vex4 mmx avx2_256 p_00_66),
[0xda] = X86_OP_ENTRY3(PMINUB, V,x, H,x, W,x, vex4 mmx avx2_256 p_00_66),
[0xdb] = X86_OP_ENTRY3(PAND, V,x, H,x, W,x, vex4 mmx avx2_256 p_00_66),
[0xdc] = X86_OP_ENTRY3(PADDUSB, V,x, H,x, W,x, vex4 mmx avx2_256 p_00_66),
[0xdd] = X86_OP_ENTRY3(PADDUSW, V,x, H,x, W,x, vex4 mmx avx2_256 p_00_66),
[0xde] = X86_OP_ENTRY3(PMAXUB, V,x, H,x, W,x, vex4 mmx avx2_256 p_00_66),
[0xdf] = X86_OP_ENTRY3(PANDN, V,x, H,x, W,x, vex4 mmx avx2_256 p_00_66),
[0xe8] = X86_OP_ENTRY3(PSUBSB, V,x, H,x, W,x, vex4 mmx avx2_256 p_00_66),
[0xe9] = X86_OP_ENTRY3(PSUBSW, V,x, H,x, W,x, vex4 mmx avx2_256 p_00_66),
[0xea] = X86_OP_ENTRY3(PMINSW, V,x, H,x, W,x, vex4 mmx avx2_256 p_00_66),
[0xeb] = X86_OP_ENTRY3(POR, V,x, H,x, W,x, vex4 mmx avx2_256 p_00_66),
[0xec] = X86_OP_ENTRY3(PADDSB, V,x, H,x, W,x, vex4 mmx avx2_256 p_00_66),
[0xed] = X86_OP_ENTRY3(PADDSW, V,x, H,x, W,x, vex4 mmx avx2_256 p_00_66),
[0xee] = X86_OP_ENTRY3(PMAXSW, V,x, H,x, W,x, vex4 mmx avx2_256 p_00_66),
[0xef] = X86_OP_ENTRY3(PXOR, V,x, H,x, W,x, vex4 mmx avx2_256 p_00_66),
[0xf8] = X86_OP_ENTRY3(PSUBB, V,x, H,x, W,x, vex4 mmx avx2_256 p_00_66),
[0xf9] = X86_OP_ENTRY3(PSUBW, V,x, H,x, W,x, vex4 mmx avx2_256 p_00_66),
[0xfa] = X86_OP_ENTRY3(PSUBD, V,x, H,x, W,x, vex4 mmx avx2_256 p_00_66),
[0xfb] = X86_OP_ENTRY3(PSUBQ, V,x, H,x, W,x, vex4 mmx avx2_256 p_00_66),
[0xfc] = X86_OP_ENTRY3(PADDB, V,x, H,x, W,x, vex4 mmx avx2_256 p_00_66),
[0xfd] = X86_OP_ENTRY3(PADDW, V,x, H,x, W,x, vex4 mmx avx2_256 p_00_66),
[0xfe] = X86_OP_ENTRY3(PADDD, V,x, H,x, W,x, vex4 mmx avx2_256 p_00_66),
target/i386: move prefetch and multi-byte UD/NOP to new decoder These are trivial to add, and moving them to the new decoder fixes some corner cases: raising #UD instead of an instruction fetch page fault for the undefined opcodes, and incorrectly rejecting 0F 18 prefetches with register operands (which are treated as reserved NOPs). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Zhao Liu <zhao1.liu@intel.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-06 18:34:55 +03:00
[0xff] = X86_OP_ENTRYr(UD, nop,v), /* UD0 */
target/i386: add core of new i386 decoder The new decoder is based on three principles: - use mostly table-driven decoding, using tables derived as much as possible from the Intel manual. Centralizing the decode the operands makes it more homogeneous, for example all immediates are signed. All modrm handling is in one function, and can be shared between SSE and ALU instructions (including XMM<->GPR instructions). The SSE/AVX decoder will also not have duplicated code between the 0F, 0F38 and 0F3A tables. - keep the code as "non-branchy" as possible. Generally, the code for the new decoder is more verbose, but the control flow is simpler. Conditionals are not nested and have small bodies. All instruction groups are resolved even before operands are decoded, and code generation is separated as much as possible within small functions that only handle one instruction each. - keep address generation and (for ALU operands) memory loads and writeback as much in common code as possible. All ALU operations for example are implemented as T0=f(T0,T1). For non-ALU instructions, read-modify-write memory operations are rare, but registers do not have TCGv equivalents: therefore, the common logic sets up pointer temporaries with the operands, while load and writeback are handled by gvec or by helpers. These principles make future code review and extensibility simpler, at the cost of having a relatively large amount of code in the form of this patch. Even EVEX should not be _too_ hard to implement (it's just a crazy large amount of possibilities). This patch introduces the main decoder flow, and integrates the old decoder with the new one. The old decoder takes care of parsing prefixes and then optionally drops to the new one. The changes to the old decoder are minimal and allow it to be replaced incrementally with the new one. There is a debugging mechanism through a "LIMIT" environment variable. In user-mode emulation, the variable is the number of instructions decoded by the new decoder before permanently switching to the old one. In system emulation, the variable is the highest opcode that is decoded by the new decoder (this is less friendly, but it's the best that can be done without requiring deterministic execution). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-23 12:20:55 +03:00
};
static void do_decode_0F(DisasContext *s, CPUX86State *env, X86OpEntry *entry, uint8_t *b)
{
*entry = opcodes_0F[*b];
}
static void decode_0F(DisasContext *s, CPUX86State *env, X86OpEntry *entry, uint8_t *b)
{
*b = x86_ldub_code(env, s);
do_decode_0F(s, env, entry, b);
}
target/i386: move 60-BF opcodes to new decoder Compared to the old decoder, the main differences in translation are for the little-used ARPL instruction. IMUL is adjusted a bit to share more code to produce flags, but is otherwise very similar. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-11 11:55:16 +03:00
static void decode_63(DisasContext *s, CPUX86State *env, X86OpEntry *entry, uint8_t *b)
{
static const X86OpEntry arpl = X86_OP_ENTRY2(ARPL, E,w, G,w, chk(prot));
static const X86OpEntry mov = X86_OP_ENTRY3(MOV, G,v, E,v, None, None);
static const X86OpEntry movsxd = X86_OP_ENTRY3(MOV, G,v, E,d, None, None, sextT0);
if (!CODE64(s)) {
*entry = arpl;
} else if (REX_W(s)) {
*entry = movsxd;
} else {
*entry = mov;
}
}
static void decode_group1(DisasContext *s, CPUX86State *env, X86OpEntry *entry, uint8_t *b)
{
static const X86GenFunc group1_gen[8] = {
gen_ADD, gen_OR, gen_ADC, gen_SBB, gen_AND, gen_SUB, gen_XOR, gen_SUB,
};
int op = (get_modrm(s, env) >> 3) & 7;
entry->gen = group1_gen[op];
if (op == 7) {
/* prevent writeback for CMP */
entry->op1 = entry->op0;
entry->op0 = X86_TYPE_None;
entry->s0 = X86_SIZE_None;
} else {
entry->special = X86_SPECIAL_HasLock;
}
}
static void decode_group1A(DisasContext *s, CPUX86State *env, X86OpEntry *entry, uint8_t *b)
{
int op = (get_modrm(s, env) >> 3) & 7;
if (op != 0) {
/* could be XOP prefix too */
*entry = UNKNOWN_OPCODE;
} else {
entry->gen = gen_POP;
/* The address must use the value of ESP after the pop. */
s->popl_esp_hack = 1 << mo_pushpop(s, s->dflag);
}
}
target/i386: move C0-FF opcodes to new decoder (except for x87) The shift instructions are rewritten instead of reusing code from the old decoder. Rotates use CC_OP_ADCOX more extensively and generally rely more on the optimizer, so that the code generators are shared between the immediate-count and variable-count cases. In particular, this makes gen_RCL and gen_RCR pretty efficient for the count == 1 case, which becomes (apart from a few extra movs) something like: (compute_cc_all if needed) // save old value for OF calculation mov cc_src2, T0 // the bulk of RCL is just this! deposit T0, cc_src, T0, 1, TARGET_LONG_BITS - 1 // compute carry shr cc_dst, cc_src2, length - 1 and cc_dst, cc_dst, 1 // compute overflow xor cc_src2, cc_src2, T0 extract cc_src2, cc_src2, length - 1, 1 32-bit MUL and IMUL are also slightly more efficient on 64-bit hosts. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-21 18:36:34 +03:00
static void decode_group2(DisasContext *s, CPUX86State *env, X86OpEntry *entry, uint8_t *b)
{
static const X86GenFunc group2_gen[8] = {
gen_ROL, gen_ROR, gen_RCL, gen_RCR,
gen_SHL, gen_SHR, gen_SHL /* SAL, undocumented */, gen_SAR,
};
int op = (get_modrm(s, env) >> 3) & 7;
entry->gen = group2_gen[op];
if (op == 7) {
entry->special = X86_SPECIAL_SExtT0;
} else {
entry->special = X86_SPECIAL_ZExtT0;
}
}
static void decode_group3(DisasContext *s, CPUX86State *env, X86OpEntry *entry, uint8_t *b)
{
static const X86OpEntry opcodes_grp3[16] = {
/* 0xf6 */
[0x00] = X86_OP_ENTRYrr(AND, E,b, I,b),
[0x02] = X86_OP_ENTRY1(NOT, E,b, lock),
[0x03] = X86_OP_ENTRY1(NEG, E,b, lock),
[0x04] = X86_OP_ENTRYrr(MUL, E,b, 0,b, zextT0),
[0x05] = X86_OP_ENTRYrr(IMUL,E,b, 0,b, sextT0),
[0x06] = X86_OP_ENTRYr(DIV, E,b),
[0x07] = X86_OP_ENTRYr(IDIV, E,b),
/* 0xf7 */
[0x08] = X86_OP_ENTRYrr(AND, E,v, I,z),
[0x0a] = X86_OP_ENTRY1(NOT, E,v, lock),
[0x0b] = X86_OP_ENTRY1(NEG, E,v, lock),
[0x0c] = X86_OP_ENTRYrr(MUL, E,v, 0,v, zextT0),
[0x0d] = X86_OP_ENTRYrr(IMUL,E,v, 0,v, sextT0),
[0x0e] = X86_OP_ENTRYr(DIV, E,v),
[0x0f] = X86_OP_ENTRYr(IDIV, E,v),
};
int w = (*b & 1);
int reg = (get_modrm(s, env) >> 3) & 7;
*entry = opcodes_grp3[(w << 3) | reg];
}
static void decode_group4_5(DisasContext *s, CPUX86State *env, X86OpEntry *entry, uint8_t *b)
{
static const X86OpEntry opcodes_grp4_5[16] = {
/* 0xfe */
[0x00] = X86_OP_ENTRY1(INC, E,b, lock),
[0x01] = X86_OP_ENTRY1(DEC, E,b, lock),
/* 0xff */
[0x08] = X86_OP_ENTRY1(INC, E,v, lock),
[0x09] = X86_OP_ENTRY1(DEC, E,v, lock),
target/i386: change X86_ENTRYr to use T0 I am not sure why I made it use T1. It is a bit more symmetric with respect to X86_ENTRYwr (which uses T0 for the "w"ritten operand and T1 for the "r"ead operand), but it is also less flexible because it does not let you apply zextT0/sextT0. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-08 12:06:50 +03:00
[0x0a] = X86_OP_ENTRYr(CALL_m, E,f64, zextT0),
target/i386: move C0-FF opcodes to new decoder (except for x87) The shift instructions are rewritten instead of reusing code from the old decoder. Rotates use CC_OP_ADCOX more extensively and generally rely more on the optimizer, so that the code generators are shared between the immediate-count and variable-count cases. In particular, this makes gen_RCL and gen_RCR pretty efficient for the count == 1 case, which becomes (apart from a few extra movs) something like: (compute_cc_all if needed) // save old value for OF calculation mov cc_src2, T0 // the bulk of RCL is just this! deposit T0, cc_src, T0, 1, TARGET_LONG_BITS - 1 // compute carry shr cc_dst, cc_src2, length - 1 and cc_dst, cc_dst, 1 // compute overflow xor cc_src2, cc_src2, T0 extract cc_src2, cc_src2, length - 1, 1 32-bit MUL and IMUL are also slightly more efficient on 64-bit hosts. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-21 18:36:34 +03:00
[0x0b] = X86_OP_ENTRYr(CALLF_m, M,p),
target/i386: change X86_ENTRYr to use T0 I am not sure why I made it use T1. It is a bit more symmetric with respect to X86_ENTRYwr (which uses T0 for the "w"ritten operand and T1 for the "r"ead operand), but it is also less flexible because it does not let you apply zextT0/sextT0. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-08 12:06:50 +03:00
[0x0c] = X86_OP_ENTRYr(JMP_m, E,f64, zextT0),
target/i386: move C0-FF opcodes to new decoder (except for x87) The shift instructions are rewritten instead of reusing code from the old decoder. Rotates use CC_OP_ADCOX more extensively and generally rely more on the optimizer, so that the code generators are shared between the immediate-count and variable-count cases. In particular, this makes gen_RCL and gen_RCR pretty efficient for the count == 1 case, which becomes (apart from a few extra movs) something like: (compute_cc_all if needed) // save old value for OF calculation mov cc_src2, T0 // the bulk of RCL is just this! deposit T0, cc_src, T0, 1, TARGET_LONG_BITS - 1 // compute carry shr cc_dst, cc_src2, length - 1 and cc_dst, cc_dst, 1 // compute overflow xor cc_src2, cc_src2, T0 extract cc_src2, cc_src2, length - 1, 1 32-bit MUL and IMUL are also slightly more efficient on 64-bit hosts. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-21 18:36:34 +03:00
[0x0d] = X86_OP_ENTRYr(JMPF_m, M,p),
[0x0e] = X86_OP_ENTRYr(PUSH, E,f64),
};
int w = (*b & 1);
int reg = (get_modrm(s, env) >> 3) & 7;
*entry = opcodes_grp4_5[(w << 3) | reg];
}
static void decode_group11(DisasContext *s, CPUX86State *env, X86OpEntry *entry, uint8_t *b)
{
int op = (get_modrm(s, env) >> 3) & 7;
if (op != 0) {
*entry = UNKNOWN_OPCODE;
} else {
entry->gen = gen_MOV;
}
}
target/i386: cleanup PAUSE helpers Use decode.c's support for intercepts, doing the check in TCG-generated code rather than the helper. This is cleaner because it allows removing the eip_addend argument to helper_pause(), even though it adds a bit of bloat for opcode 0x90's new decoding function. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-29 16:12:22 +03:00
static void decode_90(DisasContext *s, CPUX86State *env, X86OpEntry *entry, uint8_t *b)
{
static X86OpEntry pause = X86_OP_ENTRY0(PAUSE, svm(PAUSE));
static X86OpEntry nop = X86_OP_ENTRY0(NOP);
static X86OpEntry xchg_ax = X86_OP_ENTRY2(XCHG, 0,v, LoBits,v);
if (REX_B(s)) {
*entry = xchg_ax;
} else {
*entry = (s->prefix & PREFIX_REPZ) ? pause : nop;
}
}
target/i386: add core of new i386 decoder The new decoder is based on three principles: - use mostly table-driven decoding, using tables derived as much as possible from the Intel manual. Centralizing the decode the operands makes it more homogeneous, for example all immediates are signed. All modrm handling is in one function, and can be shared between SSE and ALU instructions (including XMM<->GPR instructions). The SSE/AVX decoder will also not have duplicated code between the 0F, 0F38 and 0F3A tables. - keep the code as "non-branchy" as possible. Generally, the code for the new decoder is more verbose, but the control flow is simpler. Conditionals are not nested and have small bodies. All instruction groups are resolved even before operands are decoded, and code generation is separated as much as possible within small functions that only handle one instruction each. - keep address generation and (for ALU operands) memory loads and writeback as much in common code as possible. All ALU operations for example are implemented as T0=f(T0,T1). For non-ALU instructions, read-modify-write memory operations are rare, but registers do not have TCGv equivalents: therefore, the common logic sets up pointer temporaries with the operands, while load and writeback are handled by gvec or by helpers. These principles make future code review and extensibility simpler, at the cost of having a relatively large amount of code in the form of this patch. Even EVEX should not be _too_ hard to implement (it's just a crazy large amount of possibilities). This patch introduces the main decoder flow, and integrates the old decoder with the new one. The old decoder takes care of parsing prefixes and then optionally drops to the new one. The changes to the old decoder are minimal and allow it to be replaced incrementally with the new one. There is a debugging mechanism through a "LIMIT" environment variable. In user-mode emulation, the variable is the number of instructions decoded by the new decoder before permanently switching to the old one. In system emulation, the variable is the highest opcode that is decoded by the new decoder (this is less friendly, but it's the best that can be done without requiring deterministic execution). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-23 12:20:55 +03:00
static const X86OpEntry opcodes_root[256] = {
target/i386: move 00-5F opcodes to new decoder Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-20 01:30:21 +03:00
[0x00] = X86_OP_ENTRY2(ADD, E,b, G,b, lock),
[0x01] = X86_OP_ENTRY2(ADD, E,v, G,v, lock),
[0x02] = X86_OP_ENTRY2(ADD, G,b, E,b, lock),
[0x03] = X86_OP_ENTRY2(ADD, G,v, E,v, lock),
[0x04] = X86_OP_ENTRY2(ADD, 0,b, I,b, lock), /* AL, Ib */
[0x05] = X86_OP_ENTRY2(ADD, 0,v, I,z, lock), /* rAX, Iz */
[0x06] = X86_OP_ENTRYr(PUSH, ES, w, chk(i64)),
[0x07] = X86_OP_ENTRYw(POP, ES, w, chk(i64)),
[0x10] = X86_OP_ENTRY2(ADC, E,b, G,b, lock),
[0x11] = X86_OP_ENTRY2(ADC, E,v, G,v, lock),
[0x12] = X86_OP_ENTRY2(ADC, G,b, E,b, lock),
[0x13] = X86_OP_ENTRY2(ADC, G,v, E,v, lock),
[0x14] = X86_OP_ENTRY2(ADC, 0,b, I,b, lock), /* AL, Ib */
[0x15] = X86_OP_ENTRY2(ADC, 0,v, I,z, lock), /* rAX, Iz */
[0x16] = X86_OP_ENTRYr(PUSH, SS, w, chk(i64)),
[0x17] = X86_OP_ENTRYw(POP, SS, w, chk(i64)),
[0x20] = X86_OP_ENTRY2(AND, E,b, G,b, lock),
[0x21] = X86_OP_ENTRY2(AND, E,v, G,v, lock),
[0x22] = X86_OP_ENTRY2(AND, G,b, E,b, lock),
[0x23] = X86_OP_ENTRY2(AND, G,v, E,v, lock),
[0x24] = X86_OP_ENTRY2(AND, 0,b, I,b, lock), /* AL, Ib */
[0x25] = X86_OP_ENTRY2(AND, 0,v, I,z, lock), /* rAX, Iz */
[0x26] = {},
[0x27] = X86_OP_ENTRY0(DAA, chk(i64)),
[0x30] = X86_OP_ENTRY2(XOR, E,b, G,b, lock),
[0x31] = X86_OP_ENTRY2(XOR, E,v, G,v, lock),
[0x32] = X86_OP_ENTRY2(XOR, G,b, E,b, lock),
[0x33] = X86_OP_ENTRY2(XOR, G,v, E,v, lock),
[0x34] = X86_OP_ENTRY2(XOR, 0,b, I,b, lock), /* AL, Ib */
[0x35] = X86_OP_ENTRY2(XOR, 0,v, I,z, lock), /* rAX, Iz */
[0x36] = {},
[0x37] = X86_OP_ENTRY0(AAA, chk(i64)),
[0x40] = X86_OP_ENTRY1(INC, 0,v, chk(i64)),
[0x41] = X86_OP_ENTRY1(INC, 1,v, chk(i64)),
[0x42] = X86_OP_ENTRY1(INC, 2,v, chk(i64)),
[0x43] = X86_OP_ENTRY1(INC, 3,v, chk(i64)),
[0x44] = X86_OP_ENTRY1(INC, 4,v, chk(i64)),
[0x45] = X86_OP_ENTRY1(INC, 5,v, chk(i64)),
[0x46] = X86_OP_ENTRY1(INC, 6,v, chk(i64)),
[0x47] = X86_OP_ENTRY1(INC, 7,v, chk(i64)),
[0x50] = X86_OP_ENTRYr(PUSH, LoBits,d64),
[0x51] = X86_OP_ENTRYr(PUSH, LoBits,d64),
[0x52] = X86_OP_ENTRYr(PUSH, LoBits,d64),
[0x53] = X86_OP_ENTRYr(PUSH, LoBits,d64),
[0x54] = X86_OP_ENTRYr(PUSH, LoBits,d64),
[0x55] = X86_OP_ENTRYr(PUSH, LoBits,d64),
[0x56] = X86_OP_ENTRYr(PUSH, LoBits,d64),
[0x57] = X86_OP_ENTRYr(PUSH, LoBits,d64),
target/i386: move 60-BF opcodes to new decoder Compared to the old decoder, the main differences in translation are for the little-used ARPL instruction. IMUL is adjusted a bit to share more code to produce flags, but is otherwise very similar. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-11 11:55:16 +03:00
[0x60] = X86_OP_ENTRY0(PUSHA, chk(i64)),
[0x61] = X86_OP_ENTRY0(POPA, chk(i64)),
[0x62] = X86_OP_ENTRYrr(BOUND, G,v, M,a, chk(i64)),
[0x63] = X86_OP_GROUP0(63),
[0x64] = {},
[0x65] = {},
[0x66] = {},
[0x67] = {},
[0x70] = X86_OP_ENTRYr(Jcc, J,b),
[0x71] = X86_OP_ENTRYr(Jcc, J,b),
[0x72] = X86_OP_ENTRYr(Jcc, J,b),
[0x73] = X86_OP_ENTRYr(Jcc, J,b),
[0x74] = X86_OP_ENTRYr(Jcc, J,b),
[0x75] = X86_OP_ENTRYr(Jcc, J,b),
[0x76] = X86_OP_ENTRYr(Jcc, J,b),
[0x77] = X86_OP_ENTRYr(Jcc, J,b),
[0x80] = X86_OP_GROUP2(group1, E,b, I,b),
[0x81] = X86_OP_GROUP2(group1, E,v, I,z),
[0x82] = X86_OP_GROUP2(group1, E,b, I,b, chk(i64)),
[0x83] = X86_OP_GROUP2(group1, E,v, I,b),
[0x84] = X86_OP_ENTRYrr(AND, E,b, G,b),
[0x85] = X86_OP_ENTRYrr(AND, E,v, G,v),
[0x86] = X86_OP_ENTRY2(XCHG, E,b, G,b, xchg),
[0x87] = X86_OP_ENTRY2(XCHG, E,v, G,v, xchg),
target/i386: cleanup PAUSE helpers Use decode.c's support for intercepts, doing the check in TCG-generated code rather than the helper. This is cleaner because it allows removing the eip_addend argument to helper_pause(), even though it adds a bit of bloat for opcode 0x90's new decoding function. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-29 16:12:22 +03:00
[0x90] = X86_OP_GROUP0(90),
target/i386: move 60-BF opcodes to new decoder Compared to the old decoder, the main differences in translation are for the little-used ARPL instruction. IMUL is adjusted a bit to share more code to produce flags, but is otherwise very similar. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-11 11:55:16 +03:00
[0x91] = X86_OP_ENTRY2(XCHG, 0,v, LoBits,v),
[0x92] = X86_OP_ENTRY2(XCHG, 0,v, LoBits,v),
[0x93] = X86_OP_ENTRY2(XCHG, 0,v, LoBits,v),
[0x94] = X86_OP_ENTRY2(XCHG, 0,v, LoBits,v),
[0x95] = X86_OP_ENTRY2(XCHG, 0,v, LoBits,v),
[0x96] = X86_OP_ENTRY2(XCHG, 0,v, LoBits,v),
[0x97] = X86_OP_ENTRY2(XCHG, 0,v, LoBits,v),
[0xA0] = X86_OP_ENTRY3(MOV, 0,b, O,b, None, None), /* AL, Ob */
[0xA1] = X86_OP_ENTRY3(MOV, 0,v, O,v, None, None), /* rAX, Ov */
[0xA2] = X86_OP_ENTRY3(MOV, O,b, 0,b, None, None), /* Ob, AL */
[0xA3] = X86_OP_ENTRY3(MOV, O,v, 0,v, None, None), /* Ov, rAX */
[0xA4] = X86_OP_ENTRYrr(MOVS, Y,b, X,b),
[0xA5] = X86_OP_ENTRYrr(MOVS, Y,v, X,v),
[0xA6] = X86_OP_ENTRYrr(CMPS, Y,b, X,b),
[0xA7] = X86_OP_ENTRYrr(CMPS, Y,v, X,v),
[0xB0] = X86_OP_ENTRY3(MOV, LoBits,b, I,b, None, None),
[0xB1] = X86_OP_ENTRY3(MOV, LoBits,b, I,b, None, None),
[0xB2] = X86_OP_ENTRY3(MOV, LoBits,b, I,b, None, None),
[0xB3] = X86_OP_ENTRY3(MOV, LoBits,b, I,b, None, None),
[0xB4] = X86_OP_ENTRY3(MOV, LoBits,b, I,b, None, None),
[0xB5] = X86_OP_ENTRY3(MOV, LoBits,b, I,b, None, None),
[0xB6] = X86_OP_ENTRY3(MOV, LoBits,b, I,b, None, None),
[0xB7] = X86_OP_ENTRY3(MOV, LoBits,b, I,b, None, None),
target/i386: move C0-FF opcodes to new decoder (except for x87) The shift instructions are rewritten instead of reusing code from the old decoder. Rotates use CC_OP_ADCOX more extensively and generally rely more on the optimizer, so that the code generators are shared between the immediate-count and variable-count cases. In particular, this makes gen_RCL and gen_RCR pretty efficient for the count == 1 case, which becomes (apart from a few extra movs) something like: (compute_cc_all if needed) // save old value for OF calculation mov cc_src2, T0 // the bulk of RCL is just this! deposit T0, cc_src, T0, 1, TARGET_LONG_BITS - 1 // compute carry shr cc_dst, cc_src2, length - 1 and cc_dst, cc_dst, 1 // compute overflow xor cc_src2, cc_src2, T0 extract cc_src2, cc_src2, length - 1, 1 32-bit MUL and IMUL are also slightly more efficient on 64-bit hosts. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-21 18:36:34 +03:00
[0xC0] = X86_OP_GROUP2(group2, E,b, I,b),
[0xC1] = X86_OP_GROUP2(group2, E,v, I,b),
[0xC2] = X86_OP_ENTRYr(RET, I,w),
[0xC3] = X86_OP_ENTRY0(RET),
[0xC4] = X86_OP_ENTRY3(LES, G,z, EM,p, None, None, chk(i64)),
[0xC5] = X86_OP_ENTRY3(LDS, G,z, EM,p, None, None, chk(i64)),
[0xC6] = X86_OP_GROUP3(group11, E,b, I,b, None, None), /* reg=000b */
[0xC7] = X86_OP_GROUP3(group11, E,v, I,z, None, None), /* reg=000b */
[0xD0] = X86_OP_GROUP1(group2, E,b),
[0xD1] = X86_OP_GROUP1(group2, E,v),
[0xD2] = X86_OP_GROUP2(group2, E,b, 1,b), /* CL */
[0xD3] = X86_OP_GROUP2(group2, E,v, 1,b), /* CL */
target/i386: clean up AAM/AAD The 32-bit AAM/AAD opcodes are using helpers that read and write flags and env->regs[R_EAX]. Clean them up so that the table correctly includes AX as a 16-bit input and output. No real reason to do it to be honest, but they are nice one-output helpers and it removes the masking of env->regs[R_EAX] that generic load/writeback code already does. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-ID: <20240522123912.608497-1-pbonzini@redhat.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-22 15:39:12 +03:00
[0xD4] = X86_OP_ENTRY2(AAM, 0,w, I,b),
[0xD5] = X86_OP_ENTRY2(AAD, 0,w, I,b),
target/i386: move C0-FF opcodes to new decoder (except for x87) The shift instructions are rewritten instead of reusing code from the old decoder. Rotates use CC_OP_ADCOX more extensively and generally rely more on the optimizer, so that the code generators are shared between the immediate-count and variable-count cases. In particular, this makes gen_RCL and gen_RCR pretty efficient for the count == 1 case, which becomes (apart from a few extra movs) something like: (compute_cc_all if needed) // save old value for OF calculation mov cc_src2, T0 // the bulk of RCL is just this! deposit T0, cc_src, T0, 1, TARGET_LONG_BITS - 1 // compute carry shr cc_dst, cc_src2, length - 1 and cc_dst, cc_dst, 1 // compute overflow xor cc_src2, cc_src2, T0 extract cc_src2, cc_src2, length - 1, 1 32-bit MUL and IMUL are also slightly more efficient on 64-bit hosts. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-21 18:36:34 +03:00
[0xD6] = X86_OP_ENTRYw(SALC, 0,b),
[0xD7] = X86_OP_ENTRY1(XLAT, 0,b, zextT0), /* AL read/written */
[0xE0] = X86_OP_ENTRYr(LOOPNE, J,b), /* implicit: CX with aflag size */
[0xE1] = X86_OP_ENTRYr(LOOPE, J,b), /* implicit: CX with aflag size */
[0xE2] = X86_OP_ENTRYr(LOOP, J,b), /* implicit: CX with aflag size */
[0xE3] = X86_OP_ENTRYr(JCXZ, J,b), /* implicit: CX with aflag size */
[0xE4] = X86_OP_ENTRYwr(IN, 0,b, I_unsigned,b), /* AL */
[0xE5] = X86_OP_ENTRYwr(IN, 0,v, I_unsigned,b), /* AX/EAX */
[0xE6] = X86_OP_ENTRYrr(OUT, 0,b, I_unsigned,b), /* AL */
[0xE7] = X86_OP_ENTRYrr(OUT, 0,v, I_unsigned,b), /* AX/EAX */
[0xF1] = X86_OP_ENTRY0(INT1, svm(ICEBP)),
target/i386: cleanup HLT helpers Use decode.c's support for intercepts, doing the check in TCG-generated code rather than the helper. This is cleaner because it allows removing the eip_addend argument to helper_hlt(). Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-29 16:12:22 +03:00
[0xF4] = X86_OP_ENTRY0(HLT, chk(cpl0) svm(HLT)),
target/i386: move C0-FF opcodes to new decoder (except for x87) The shift instructions are rewritten instead of reusing code from the old decoder. Rotates use CC_OP_ADCOX more extensively and generally rely more on the optimizer, so that the code generators are shared between the immediate-count and variable-count cases. In particular, this makes gen_RCL and gen_RCR pretty efficient for the count == 1 case, which becomes (apart from a few extra movs) something like: (compute_cc_all if needed) // save old value for OF calculation mov cc_src2, T0 // the bulk of RCL is just this! deposit T0, cc_src, T0, 1, TARGET_LONG_BITS - 1 // compute carry shr cc_dst, cc_src2, length - 1 and cc_dst, cc_dst, 1 // compute overflow xor cc_src2, cc_src2, T0 extract cc_src2, cc_src2, length - 1, 1 32-bit MUL and IMUL are also slightly more efficient on 64-bit hosts. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-21 18:36:34 +03:00
[0xF5] = X86_OP_ENTRY0(CMC),
[0xF6] = X86_OP_GROUP1(group3, E,b),
[0xF7] = X86_OP_GROUP1(group3, E,v),
target/i386: move 00-5F opcodes to new decoder Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-20 01:30:21 +03:00
[0x08] = X86_OP_ENTRY2(OR, E,b, G,b, lock),
[0x09] = X86_OP_ENTRY2(OR, E,v, G,v, lock),
[0x0A] = X86_OP_ENTRY2(OR, G,b, E,b, lock),
[0x0B] = X86_OP_ENTRY2(OR, G,v, E,v, lock),
[0x0C] = X86_OP_ENTRY2(OR, 0,b, I,b, lock), /* AL, Ib */
[0x0D] = X86_OP_ENTRY2(OR, 0,v, I,z, lock), /* rAX, Iz */
[0x0E] = X86_OP_ENTRYr(PUSH, CS, w, chk(i64)),
target/i386: add core of new i386 decoder The new decoder is based on three principles: - use mostly table-driven decoding, using tables derived as much as possible from the Intel manual. Centralizing the decode the operands makes it more homogeneous, for example all immediates are signed. All modrm handling is in one function, and can be shared between SSE and ALU instructions (including XMM<->GPR instructions). The SSE/AVX decoder will also not have duplicated code between the 0F, 0F38 and 0F3A tables. - keep the code as "non-branchy" as possible. Generally, the code for the new decoder is more verbose, but the control flow is simpler. Conditionals are not nested and have small bodies. All instruction groups are resolved even before operands are decoded, and code generation is separated as much as possible within small functions that only handle one instruction each. - keep address generation and (for ALU operands) memory loads and writeback as much in common code as possible. All ALU operations for example are implemented as T0=f(T0,T1). For non-ALU instructions, read-modify-write memory operations are rare, but registers do not have TCGv equivalents: therefore, the common logic sets up pointer temporaries with the operands, while load and writeback are handled by gvec or by helpers. These principles make future code review and extensibility simpler, at the cost of having a relatively large amount of code in the form of this patch. Even EVEX should not be _too_ hard to implement (it's just a crazy large amount of possibilities). This patch introduces the main decoder flow, and integrates the old decoder with the new one. The old decoder takes care of parsing prefixes and then optionally drops to the new one. The changes to the old decoder are minimal and allow it to be replaced incrementally with the new one. There is a debugging mechanism through a "LIMIT" environment variable. In user-mode emulation, the variable is the number of instructions decoded by the new decoder before permanently switching to the old one. In system emulation, the variable is the highest opcode that is decoded by the new decoder (this is less friendly, but it's the best that can be done without requiring deterministic execution). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-23 12:20:55 +03:00
[0x0F] = X86_OP_GROUP0(0F),
target/i386: move 00-5F opcodes to new decoder Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-20 01:30:21 +03:00
[0x18] = X86_OP_ENTRY2(SBB, E,b, G,b, lock),
[0x19] = X86_OP_ENTRY2(SBB, E,v, G,v, lock),
[0x1A] = X86_OP_ENTRY2(SBB, G,b, E,b, lock),
[0x1B] = X86_OP_ENTRY2(SBB, G,v, E,v, lock),
[0x1C] = X86_OP_ENTRY2(SBB, 0,b, I,b, lock), /* AL, Ib */
[0x1D] = X86_OP_ENTRY2(SBB, 0,v, I,z, lock), /* rAX, Iz */
[0x1E] = X86_OP_ENTRYr(PUSH, DS, w, chk(i64)),
[0x1F] = X86_OP_ENTRYw(POP, DS, w, chk(i64)),
[0x28] = X86_OP_ENTRY2(SUB, E,b, G,b, lock),
[0x29] = X86_OP_ENTRY2(SUB, E,v, G,v, lock),
[0x2A] = X86_OP_ENTRY2(SUB, G,b, E,b, lock),
[0x2B] = X86_OP_ENTRY2(SUB, G,v, E,v, lock),
[0x2C] = X86_OP_ENTRY2(SUB, 0,b, I,b, lock), /* AL, Ib */
[0x2D] = X86_OP_ENTRY2(SUB, 0,v, I,z, lock), /* rAX, Iz */
[0x2E] = {},
[0x2F] = X86_OP_ENTRY0(DAS, chk(i64)),
[0x38] = X86_OP_ENTRYrr(SUB, E,b, G,b),
[0x39] = X86_OP_ENTRYrr(SUB, E,v, G,v),
[0x3A] = X86_OP_ENTRYrr(SUB, G,b, E,b),
[0x3B] = X86_OP_ENTRYrr(SUB, G,v, E,v),
[0x3C] = X86_OP_ENTRYrr(SUB, 0,b, I,b), /* AL, Ib */
[0x3D] = X86_OP_ENTRYrr(SUB, 0,v, I,z), /* rAX, Iz */
[0x3E] = {},
[0x3F] = X86_OP_ENTRY0(AAS, chk(i64)),
[0x48] = X86_OP_ENTRY1(DEC, 0,v, chk(i64)),
[0x49] = X86_OP_ENTRY1(DEC, 1,v, chk(i64)),
[0x4A] = X86_OP_ENTRY1(DEC, 2,v, chk(i64)),
[0x4B] = X86_OP_ENTRY1(DEC, 3,v, chk(i64)),
[0x4C] = X86_OP_ENTRY1(DEC, 4,v, chk(i64)),
[0x4D] = X86_OP_ENTRY1(DEC, 5,v, chk(i64)),
[0x4E] = X86_OP_ENTRY1(DEC, 6,v, chk(i64)),
[0x4F] = X86_OP_ENTRY1(DEC, 7,v, chk(i64)),
[0x58] = X86_OP_ENTRYw(POP, LoBits,d64),
[0x59] = X86_OP_ENTRYw(POP, LoBits,d64),
[0x5A] = X86_OP_ENTRYw(POP, LoBits,d64),
[0x5B] = X86_OP_ENTRYw(POP, LoBits,d64),
[0x5C] = X86_OP_ENTRYw(POP, LoBits,d64),
[0x5D] = X86_OP_ENTRYw(POP, LoBits,d64),
[0x5E] = X86_OP_ENTRYw(POP, LoBits,d64),
[0x5F] = X86_OP_ENTRYw(POP, LoBits,d64),
target/i386: move 60-BF opcodes to new decoder Compared to the old decoder, the main differences in translation are for the little-used ARPL instruction. IMUL is adjusted a bit to share more code to produce flags, but is otherwise very similar. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-11 11:55:16 +03:00
[0x68] = X86_OP_ENTRYr(PUSH, I,z),
[0x69] = X86_OP_ENTRY3(IMUL3, G,v, E,v, I,z, sextT0),
[0x6A] = X86_OP_ENTRYr(PUSH, I,b),
[0x6B] = X86_OP_ENTRY3(IMUL3, G,v, E,v, I,b, sextT0),
[0x6C] = X86_OP_ENTRYrr(INS, Y,b, 2,w), /* DX */
[0x6D] = X86_OP_ENTRYrr(INS, Y,z, 2,w), /* DX */
[0x6E] = X86_OP_ENTRYrr(OUTS, X,b, 2,w), /* DX */
[0x6F] = X86_OP_ENTRYrr(OUTS, X,z, 2,w), /* DX */
[0x78] = X86_OP_ENTRYr(Jcc, J,b),
[0x79] = X86_OP_ENTRYr(Jcc, J,b),
[0x7A] = X86_OP_ENTRYr(Jcc, J,b),
[0x7B] = X86_OP_ENTRYr(Jcc, J,b),
[0x7C] = X86_OP_ENTRYr(Jcc, J,b),
[0x7D] = X86_OP_ENTRYr(Jcc, J,b),
[0x7E] = X86_OP_ENTRYr(Jcc, J,b),
[0x7F] = X86_OP_ENTRYr(Jcc, J,b),
target/i386: change X86_ENTRYwr to use T0, use it for moves Just like X86_ENTRYr, X86_ENTRYwr is easily changed to use only T0. In this case, the motivation is to use it for the MOV instruction family. The case when you need to preserve the input value is the odd one, as it is used basically only for BLS* instructions. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-08 12:06:50 +03:00
[0x88] = X86_OP_ENTRYwr(MOV, E,b, G,b),
[0x89] = X86_OP_ENTRYwr(MOV, E,v, G,v),
[0x8A] = X86_OP_ENTRYwr(MOV, G,b, E,b),
[0x8B] = X86_OP_ENTRYwr(MOV, G,v, E,v),
/* Missing in Table A-2: memory destination is always 16-bit. */
[0x8C] = X86_OP_ENTRYwr(MOV, E,v, S,w, op0_Mw),
target/i386: replace NoSeg special with NoLoadEA This is a bit more generic, as it can be applied to MPX as well. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-09 17:56:26 +03:00
[0x8D] = X86_OP_ENTRYwr(LEA, G,v, M,v, nolea),
target/i386: change X86_ENTRYwr to use T0, use it for moves Just like X86_ENTRYr, X86_ENTRYwr is easily changed to use only T0. In this case, the motivation is to use it for the MOV instruction family. The case when you need to preserve the input value is the odd one, as it is used basically only for BLS* instructions. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-08 12:06:50 +03:00
[0x8E] = X86_OP_ENTRYwr(MOV, S,w, E,w),
target/i386: move 60-BF opcodes to new decoder Compared to the old decoder, the main differences in translation are for the little-used ARPL instruction. IMUL is adjusted a bit to share more code to produce flags, but is otherwise very similar. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-11 11:55:16 +03:00
[0x8F] = X86_OP_GROUPw(group1A, E,v),
[0x98] = X86_OP_ENTRY1(CBW, 0,v), /* rAX */
target/i386: change X86_ENTRYwr to use T0, use it for moves Just like X86_ENTRYr, X86_ENTRYwr is easily changed to use only T0. In this case, the motivation is to use it for the MOV instruction family. The case when you need to preserve the input value is the odd one, as it is used basically only for BLS* instructions. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-08 12:06:50 +03:00
[0x99] = X86_OP_ENTRYwr(CWD, 2,v, 0,v), /* rDX, rAX */
target/i386: move 60-BF opcodes to new decoder Compared to the old decoder, the main differences in translation are for the little-used ARPL instruction. IMUL is adjusted a bit to share more code to produce flags, but is otherwise very similar. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-11 11:55:16 +03:00
[0x9A] = X86_OP_ENTRYrr(CALLF, I_unsigned,p, I_unsigned,w, chk(i64)),
[0x9B] = X86_OP_ENTRY0(WAIT),
[0x9C] = X86_OP_ENTRY0(PUSHF, chk(vm86_iopl) svm(PUSHF)),
[0x9D] = X86_OP_ENTRY0(POPF, chk(vm86_iopl) svm(POPF)),
[0x9E] = X86_OP_ENTRY0(SAHF),
[0x9F] = X86_OP_ENTRY0(LAHF),
[0xA8] = X86_OP_ENTRYrr(AND, 0,b, I,b), /* AL, Ib */
[0xA9] = X86_OP_ENTRYrr(AND, 0,v, I,z), /* rAX, Iz */
target/i386: change X86_ENTRYwr to use T0, use it for moves Just like X86_ENTRYr, X86_ENTRYwr is easily changed to use only T0. In this case, the motivation is to use it for the MOV instruction family. The case when you need to preserve the input value is the odd one, as it is used basically only for BLS* instructions. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-08 12:06:50 +03:00
[0xAA] = X86_OP_ENTRYwr(STOS, Y,b, 0,b),
[0xAB] = X86_OP_ENTRYwr(STOS, Y,v, 0,v),
target/i386: move 60-BF opcodes to new decoder Compared to the old decoder, the main differences in translation are for the little-used ARPL instruction. IMUL is adjusted a bit to share more code to produce flags, but is otherwise very similar. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-11 11:55:16 +03:00
/* Manual writeback because REP LODS (!) has to write EAX/RAX after every LODS. */
[0xAC] = X86_OP_ENTRYr(LODS, X,b),
[0xAD] = X86_OP_ENTRYr(LODS, X,v),
[0xAE] = X86_OP_ENTRYrr(SCAS, 0,b, Y,b),
[0xAF] = X86_OP_ENTRYrr(SCAS, 0,v, Y,v),
target/i386: change X86_ENTRYwr to use T0, use it for moves Just like X86_ENTRYr, X86_ENTRYwr is easily changed to use only T0. In this case, the motivation is to use it for the MOV instruction family. The case when you need to preserve the input value is the odd one, as it is used basically only for BLS* instructions. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-08 12:06:50 +03:00
[0xB8] = X86_OP_ENTRYwr(MOV, LoBits,v, I,v),
[0xB9] = X86_OP_ENTRYwr(MOV, LoBits,v, I,v),
[0xBA] = X86_OP_ENTRYwr(MOV, LoBits,v, I,v),
[0xBB] = X86_OP_ENTRYwr(MOV, LoBits,v, I,v),
[0xBC] = X86_OP_ENTRYwr(MOV, LoBits,v, I,v),
[0xBD] = X86_OP_ENTRYwr(MOV, LoBits,v, I,v),
[0xBE] = X86_OP_ENTRYwr(MOV, LoBits,v, I,v),
[0xBF] = X86_OP_ENTRYwr(MOV, LoBits,v, I,v),
target/i386: move C0-FF opcodes to new decoder (except for x87) The shift instructions are rewritten instead of reusing code from the old decoder. Rotates use CC_OP_ADCOX more extensively and generally rely more on the optimizer, so that the code generators are shared between the immediate-count and variable-count cases. In particular, this makes gen_RCL and gen_RCR pretty efficient for the count == 1 case, which becomes (apart from a few extra movs) something like: (compute_cc_all if needed) // save old value for OF calculation mov cc_src2, T0 // the bulk of RCL is just this! deposit T0, cc_src, T0, 1, TARGET_LONG_BITS - 1 // compute carry shr cc_dst, cc_src2, length - 1 and cc_dst, cc_dst, 1 // compute overflow xor cc_src2, cc_src2, T0 extract cc_src2, cc_src2, length - 1, 1 32-bit MUL and IMUL are also slightly more efficient on 64-bit hosts. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-21 18:36:34 +03:00
[0xC8] = X86_OP_ENTRYrr(ENTER, I,w, I,b),
[0xC9] = X86_OP_ENTRY1(LEAVE, A,d64),
[0xCA] = X86_OP_ENTRYr(RETF, I,w),
[0xCB] = X86_OP_ENTRY0(RETF),
[0xCC] = X86_OP_ENTRY0(INT3),
[0xCD] = X86_OP_ENTRYr(INT, I,b, chk(vm86_iopl)),
[0xCE] = X86_OP_ENTRY0(INTO),
[0xCF] = X86_OP_ENTRY0(IRET, chk(vm86_iopl) svm(IRET)),
[0xE8] = X86_OP_ENTRYr(CALL, J,z_f64),
[0xE9] = X86_OP_ENTRYr(JMP, J,z_f64),
[0xEA] = X86_OP_ENTRYrr(JMPF, I_unsigned,p, I_unsigned,w, chk(i64)),
[0xEB] = X86_OP_ENTRYr(JMP, J,b),
[0xEC] = X86_OP_ENTRYwr(IN, 0,b, 2,w), /* AL, DX */
[0xED] = X86_OP_ENTRYwr(IN, 0,v, 2,w), /* AX/EAX, DX */
[0xEE] = X86_OP_ENTRYrr(OUT, 0,b, 2,w), /* DX, AL */
[0xEF] = X86_OP_ENTRYrr(OUT, 0,v, 2,w), /* DX, AX/EAX */
[0xF8] = X86_OP_ENTRY0(CLC),
[0xF9] = X86_OP_ENTRY0(STC),
[0xFA] = X86_OP_ENTRY0(CLI, chk(iopl)),
[0xFB] = X86_OP_ENTRY0(STI, chk(iopl)),
[0xFC] = X86_OP_ENTRY0(CLD),
[0xFD] = X86_OP_ENTRY0(STD),
[0xFE] = X86_OP_GROUP1(group4_5, E,b),
[0xFF] = X86_OP_GROUP1(group4_5, E,v),
target/i386: add core of new i386 decoder The new decoder is based on three principles: - use mostly table-driven decoding, using tables derived as much as possible from the Intel manual. Centralizing the decode the operands makes it more homogeneous, for example all immediates are signed. All modrm handling is in one function, and can be shared between SSE and ALU instructions (including XMM<->GPR instructions). The SSE/AVX decoder will also not have duplicated code between the 0F, 0F38 and 0F3A tables. - keep the code as "non-branchy" as possible. Generally, the code for the new decoder is more verbose, but the control flow is simpler. Conditionals are not nested and have small bodies. All instruction groups are resolved even before operands are decoded, and code generation is separated as much as possible within small functions that only handle one instruction each. - keep address generation and (for ALU operands) memory loads and writeback as much in common code as possible. All ALU operations for example are implemented as T0=f(T0,T1). For non-ALU instructions, read-modify-write memory operations are rare, but registers do not have TCGv equivalents: therefore, the common logic sets up pointer temporaries with the operands, while load and writeback are handled by gvec or by helpers. These principles make future code review and extensibility simpler, at the cost of having a relatively large amount of code in the form of this patch. Even EVEX should not be _too_ hard to implement (it's just a crazy large amount of possibilities). This patch introduces the main decoder flow, and integrates the old decoder with the new one. The old decoder takes care of parsing prefixes and then optionally drops to the new one. The changes to the old decoder are minimal and allow it to be replaced incrementally with the new one. There is a debugging mechanism through a "LIMIT" environment variable. In user-mode emulation, the variable is the number of instructions decoded by the new decoder before permanently switching to the old one. In system emulation, the variable is the highest opcode that is decoded by the new decoder (this is less friendly, but it's the best that can be done without requiring deterministic execution). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-23 12:20:55 +03:00
};
#undef mmx
target/i386: validate VEX prefixes via the instructions' exception classes Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-18 01:43:52 +03:00
#undef vex1
#undef vex2
#undef vex3
#undef vex4
#undef vex4_unal
#undef vex5
#undef vex6
#undef vex7
#undef vex8
#undef vex11
#undef vex12
#undef vex13
target/i386: add core of new i386 decoder The new decoder is based on three principles: - use mostly table-driven decoding, using tables derived as much as possible from the Intel manual. Centralizing the decode the operands makes it more homogeneous, for example all immediates are signed. All modrm handling is in one function, and can be shared between SSE and ALU instructions (including XMM<->GPR instructions). The SSE/AVX decoder will also not have duplicated code between the 0F, 0F38 and 0F3A tables. - keep the code as "non-branchy" as possible. Generally, the code for the new decoder is more verbose, but the control flow is simpler. Conditionals are not nested and have small bodies. All instruction groups are resolved even before operands are decoded, and code generation is separated as much as possible within small functions that only handle one instruction each. - keep address generation and (for ALU operands) memory loads and writeback as much in common code as possible. All ALU operations for example are implemented as T0=f(T0,T1). For non-ALU instructions, read-modify-write memory operations are rare, but registers do not have TCGv equivalents: therefore, the common logic sets up pointer temporaries with the operands, while load and writeback are handled by gvec or by helpers. These principles make future code review and extensibility simpler, at the cost of having a relatively large amount of code in the form of this patch. Even EVEX should not be _too_ hard to implement (it's just a crazy large amount of possibilities). This patch introduces the main decoder flow, and integrates the old decoder with the new one. The old decoder takes care of parsing prefixes and then optionally drops to the new one. The changes to the old decoder are minimal and allow it to be replaced incrementally with the new one. There is a debugging mechanism through a "LIMIT" environment variable. In user-mode emulation, the variable is the number of instructions decoded by the new decoder before permanently switching to the old one. In system emulation, the variable is the highest opcode that is decoded by the new decoder (this is less friendly, but it's the best that can be done without requiring deterministic execution). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-23 12:20:55 +03:00
/*
* Decode the fixed part of the opcode and place the last
* in b.
*/
static void decode_root(DisasContext *s, CPUX86State *env, X86OpEntry *entry, uint8_t *b)
{
*entry = opcodes_root[*b];
}
static int decode_modrm(DisasContext *s, CPUX86State *env, X86DecodedInsn *decode,
X86DecodedOp *op, X86OpType type)
{
int modrm = get_modrm(s, env);
if ((modrm >> 6) == 3) {
op->n = (modrm & 7);
if (type != X86_TYPE_Q && type != X86_TYPE_N) {
op->n |= REX_B(s);
}
} else {
op->has_ea = true;
op->n = -1;
decode->mem = gen_lea_modrm_0(env, s, get_modrm(s, env));
}
return modrm;
}
static bool decode_op_size(DisasContext *s, X86OpEntry *e, X86OpSize size, MemOp *ot)
{
switch (size) {
case X86_SIZE_b: /* byte */
*ot = MO_8;
return true;
case X86_SIZE_d: /* 32-bit */
case X86_SIZE_ss: /* SSE/AVX scalar single precision */
*ot = MO_32;
return true;
case X86_SIZE_p: /* Far pointer, return offset size */
case X86_SIZE_s: /* Descriptor, return offset size */
case X86_SIZE_v: /* 16/32/64-bit, based on operand size */
*ot = s->dflag;
return true;
case X86_SIZE_pi: /* MMX */
case X86_SIZE_q: /* 64-bit */
case X86_SIZE_sd: /* SSE/AVX scalar double precision */
*ot = MO_64;
return true;
case X86_SIZE_w: /* 16-bit */
*ot = MO_16;
return true;
case X86_SIZE_y: /* 32/64-bit, based on operand size */
*ot = s->dflag == MO_16 ? MO_32 : s->dflag;
return true;
target/i386: convert MOV from/to CR and DR to new decoder Complete implementation of C and D operand types, then the operations are just MOVs. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-29 16:55:22 +03:00
case X86_SIZE_y_d64: /* Full (not 16-bit) register access */
*ot = CODE64(s) ? MO_64 : MO_32;
return true;
target/i386: add core of new i386 decoder The new decoder is based on three principles: - use mostly table-driven decoding, using tables derived as much as possible from the Intel manual. Centralizing the decode the operands makes it more homogeneous, for example all immediates are signed. All modrm handling is in one function, and can be shared between SSE and ALU instructions (including XMM<->GPR instructions). The SSE/AVX decoder will also not have duplicated code between the 0F, 0F38 and 0F3A tables. - keep the code as "non-branchy" as possible. Generally, the code for the new decoder is more verbose, but the control flow is simpler. Conditionals are not nested and have small bodies. All instruction groups are resolved even before operands are decoded, and code generation is separated as much as possible within small functions that only handle one instruction each. - keep address generation and (for ALU operands) memory loads and writeback as much in common code as possible. All ALU operations for example are implemented as T0=f(T0,T1). For non-ALU instructions, read-modify-write memory operations are rare, but registers do not have TCGv equivalents: therefore, the common logic sets up pointer temporaries with the operands, while load and writeback are handled by gvec or by helpers. These principles make future code review and extensibility simpler, at the cost of having a relatively large amount of code in the form of this patch. Even EVEX should not be _too_ hard to implement (it's just a crazy large amount of possibilities). This patch introduces the main decoder flow, and integrates the old decoder with the new one. The old decoder takes care of parsing prefixes and then optionally drops to the new one. The changes to the old decoder are minimal and allow it to be replaced incrementally with the new one. There is a debugging mechanism through a "LIMIT" environment variable. In user-mode emulation, the variable is the number of instructions decoded by the new decoder before permanently switching to the old one. In system emulation, the variable is the highest opcode that is decoded by the new decoder (this is less friendly, but it's the best that can be done without requiring deterministic execution). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-23 12:20:55 +03:00
case X86_SIZE_z: /* 16-bit for 16-bit operand size, else 32-bit */
*ot = s->dflag == MO_16 ? MO_16 : MO_32;
return true;
target/i386: move C0-FF opcodes to new decoder (except for x87) The shift instructions are rewritten instead of reusing code from the old decoder. Rotates use CC_OP_ADCOX more extensively and generally rely more on the optimizer, so that the code generators are shared between the immediate-count and variable-count cases. In particular, this makes gen_RCL and gen_RCR pretty efficient for the count == 1 case, which becomes (apart from a few extra movs) something like: (compute_cc_all if needed) // save old value for OF calculation mov cc_src2, T0 // the bulk of RCL is just this! deposit T0, cc_src, T0, 1, TARGET_LONG_BITS - 1 // compute carry shr cc_dst, cc_src2, length - 1 and cc_dst, cc_dst, 1 // compute overflow xor cc_src2, cc_src2, T0 extract cc_src2, cc_src2, length - 1, 1 32-bit MUL and IMUL are also slightly more efficient on 64-bit hosts. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-21 18:36:34 +03:00
case X86_SIZE_z_f64: /* 32-bit for 32-bit operand size or 64-bit mode, else 16-bit */
*ot = !CODE64(s) && s->dflag == MO_16 ? MO_16 : MO_32;
return true;
target/i386: add core of new i386 decoder The new decoder is based on three principles: - use mostly table-driven decoding, using tables derived as much as possible from the Intel manual. Centralizing the decode the operands makes it more homogeneous, for example all immediates are signed. All modrm handling is in one function, and can be shared between SSE and ALU instructions (including XMM<->GPR instructions). The SSE/AVX decoder will also not have duplicated code between the 0F, 0F38 and 0F3A tables. - keep the code as "non-branchy" as possible. Generally, the code for the new decoder is more verbose, but the control flow is simpler. Conditionals are not nested and have small bodies. All instruction groups are resolved even before operands are decoded, and code generation is separated as much as possible within small functions that only handle one instruction each. - keep address generation and (for ALU operands) memory loads and writeback as much in common code as possible. All ALU operations for example are implemented as T0=f(T0,T1). For non-ALU instructions, read-modify-write memory operations are rare, but registers do not have TCGv equivalents: therefore, the common logic sets up pointer temporaries with the operands, while load and writeback are handled by gvec or by helpers. These principles make future code review and extensibility simpler, at the cost of having a relatively large amount of code in the form of this patch. Even EVEX should not be _too_ hard to implement (it's just a crazy large amount of possibilities). This patch introduces the main decoder flow, and integrates the old decoder with the new one. The old decoder takes care of parsing prefixes and then optionally drops to the new one. The changes to the old decoder are minimal and allow it to be replaced incrementally with the new one. There is a debugging mechanism through a "LIMIT" environment variable. In user-mode emulation, the variable is the number of instructions decoded by the new decoder before permanently switching to the old one. In system emulation, the variable is the highest opcode that is decoded by the new decoder (this is less friendly, but it's the best that can be done without requiring deterministic execution). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-23 12:20:55 +03:00
case X86_SIZE_dq: /* SSE/AVX 128-bit */
if (e->special == X86_SPECIAL_MMX &&
!(s->prefix & (PREFIX_DATA | PREFIX_REPZ | PREFIX_REPNZ))) {
*ot = MO_64;
return true;
}
if (s->vex_l && e->s0 != X86_SIZE_qq && e->s1 != X86_SIZE_qq) {
return false;
}
*ot = MO_128;
return true;
case X86_SIZE_qq: /* AVX 256-bit */
if (!s->vex_l) {
return false;
}
*ot = MO_256;
return true;
case X86_SIZE_x: /* 128/256-bit, based on operand size */
if (e->special == X86_SPECIAL_MMX &&
!(s->prefix & (PREFIX_DATA | PREFIX_REPZ | PREFIX_REPNZ))) {
*ot = MO_64;
return true;
}
/* fall through */
case X86_SIZE_ps: /* SSE/AVX packed single precision */
case X86_SIZE_pd: /* SSE/AVX packed double precision */
*ot = s->vex_l ? MO_256 : MO_128;
return true;
target/i386: generalize operand size "ph" for use in CVTPS2PD CVTPS2PD only loads a half-register for memory, like CVTPH2PS. It can reuse the "ph" packed half-precision size to load a half-register, but rename it to "xh" because it is now a variation of "x" (it is not used only for half-precision values). Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-08-29 19:25:46 +03:00
case X86_SIZE_xh: /* SSE/AVX packed half register */
target/i386: implement F16C instructions F16C only consists of two instructions, which are a bit peculiar nevertheless. First, they access only the low half of an YMM or XMM register for the packed-half operand; the exact size still depends on the VEX.L flag. This is similar to the existing avx_movx flag, but not exactly because avx_movx is hardcoded to affect operand 2. To this end I added a "ph" format name; it's possible to reuse this approach for the VPMOVSX and VPMOVZX instructions, though that would also require adding two more formats for the low-quarter and low-eighth of an operand. Second, VCVTPS2PH is somewhat weird because it *stores* the result of the instruction into memory rather than loading it. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-10-19 14:22:06 +03:00
*ot = s->vex_l ? MO_128 : MO_64;
return true;
target/i386: add core of new i386 decoder The new decoder is based on three principles: - use mostly table-driven decoding, using tables derived as much as possible from the Intel manual. Centralizing the decode the operands makes it more homogeneous, for example all immediates are signed. All modrm handling is in one function, and can be shared between SSE and ALU instructions (including XMM<->GPR instructions). The SSE/AVX decoder will also not have duplicated code between the 0F, 0F38 and 0F3A tables. - keep the code as "non-branchy" as possible. Generally, the code for the new decoder is more verbose, but the control flow is simpler. Conditionals are not nested and have small bodies. All instruction groups are resolved even before operands are decoded, and code generation is separated as much as possible within small functions that only handle one instruction each. - keep address generation and (for ALU operands) memory loads and writeback as much in common code as possible. All ALU operations for example are implemented as T0=f(T0,T1). For non-ALU instructions, read-modify-write memory operations are rare, but registers do not have TCGv equivalents: therefore, the common logic sets up pointer temporaries with the operands, while load and writeback are handled by gvec or by helpers. These principles make future code review and extensibility simpler, at the cost of having a relatively large amount of code in the form of this patch. Even EVEX should not be _too_ hard to implement (it's just a crazy large amount of possibilities). This patch introduces the main decoder flow, and integrates the old decoder with the new one. The old decoder takes care of parsing prefixes and then optionally drops to the new one. The changes to the old decoder are minimal and allow it to be replaced incrementally with the new one. There is a debugging mechanism through a "LIMIT" environment variable. In user-mode emulation, the variable is the number of instructions decoded by the new decoder before permanently switching to the old one. In system emulation, the variable is the highest opcode that is decoded by the new decoder (this is less friendly, but it's the best that can be done without requiring deterministic execution). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-23 12:20:55 +03:00
case X86_SIZE_d64: /* Default to 64-bit in 64-bit mode */
*ot = CODE64(s) && s->dflag == MO_32 ? MO_64 : s->dflag;
return true;
case X86_SIZE_f64: /* Ignore size override prefix in 64-bit mode */
*ot = CODE64(s) ? MO_64 : s->dflag;
return true;
default:
*ot = -1;
return true;
}
}
static bool decode_op(DisasContext *s, CPUX86State *env, X86DecodedInsn *decode,
X86DecodedOp *op, X86OpType type, int b)
{
int modrm;
switch (type) {
case X86_TYPE_None: /* Implicit or absent */
case X86_TYPE_A: /* Implicit */
case X86_TYPE_F: /* EFLAGS/RFLAGS */
target/i386: do not decode string source/destination into decode->mem decode->mem is only used if one operand has has_ea == true. String operations will not use decode->mem and will load A0 on their own, because they are the only case of two memory operands in a single instruction. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-19 15:51:16 +03:00
case X86_TYPE_X: /* string source */
case X86_TYPE_Y: /* string destination */
target/i386: add core of new i386 decoder The new decoder is based on three principles: - use mostly table-driven decoding, using tables derived as much as possible from the Intel manual. Centralizing the decode the operands makes it more homogeneous, for example all immediates are signed. All modrm handling is in one function, and can be shared between SSE and ALU instructions (including XMM<->GPR instructions). The SSE/AVX decoder will also not have duplicated code between the 0F, 0F38 and 0F3A tables. - keep the code as "non-branchy" as possible. Generally, the code for the new decoder is more verbose, but the control flow is simpler. Conditionals are not nested and have small bodies. All instruction groups are resolved even before operands are decoded, and code generation is separated as much as possible within small functions that only handle one instruction each. - keep address generation and (for ALU operands) memory loads and writeback as much in common code as possible. All ALU operations for example are implemented as T0=f(T0,T1). For non-ALU instructions, read-modify-write memory operations are rare, but registers do not have TCGv equivalents: therefore, the common logic sets up pointer temporaries with the operands, while load and writeback are handled by gvec or by helpers. These principles make future code review and extensibility simpler, at the cost of having a relatively large amount of code in the form of this patch. Even EVEX should not be _too_ hard to implement (it's just a crazy large amount of possibilities). This patch introduces the main decoder flow, and integrates the old decoder with the new one. The old decoder takes care of parsing prefixes and then optionally drops to the new one. The changes to the old decoder are minimal and allow it to be replaced incrementally with the new one. There is a debugging mechanism through a "LIMIT" environment variable. In user-mode emulation, the variable is the number of instructions decoded by the new decoder before permanently switching to the old one. In system emulation, the variable is the highest opcode that is decoded by the new decoder (this is less friendly, but it's the best that can be done without requiring deterministic execution). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-23 12:20:55 +03:00
break;
case X86_TYPE_B: /* VEX.vvvv selects a GPR */
op->unit = X86_OP_INT;
op->n = s->vex_v;
break;
case X86_TYPE_C: /* REG in the modrm byte selects a control register */
op->unit = X86_OP_CR;
target/i386: convert MOV from/to CR and DR to new decoder Complete implementation of C and D operand types, then the operations are just MOVs. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-29 16:55:22 +03:00
op->n = ((get_modrm(s, env) >> 3) & 7) | REX_R(s);
if (op->n == 0 && (s->prefix & PREFIX_LOCK) &&
(s->cpuid_ext3_features & CPUID_EXT3_CR8LEG)) {
op->n = 8;
s->prefix &= ~PREFIX_LOCK;
}
if (op->n != 0 && op->n != 2 && op->n != 3 && op->n != 4 && op->n != 8) {
return false;
}
if (decode->e.intercept) {
decode->e.intercept += op->n;
}
break;
target/i386: add core of new i386 decoder The new decoder is based on three principles: - use mostly table-driven decoding, using tables derived as much as possible from the Intel manual. Centralizing the decode the operands makes it more homogeneous, for example all immediates are signed. All modrm handling is in one function, and can be shared between SSE and ALU instructions (including XMM<->GPR instructions). The SSE/AVX decoder will also not have duplicated code between the 0F, 0F38 and 0F3A tables. - keep the code as "non-branchy" as possible. Generally, the code for the new decoder is more verbose, but the control flow is simpler. Conditionals are not nested and have small bodies. All instruction groups are resolved even before operands are decoded, and code generation is separated as much as possible within small functions that only handle one instruction each. - keep address generation and (for ALU operands) memory loads and writeback as much in common code as possible. All ALU operations for example are implemented as T0=f(T0,T1). For non-ALU instructions, read-modify-write memory operations are rare, but registers do not have TCGv equivalents: therefore, the common logic sets up pointer temporaries with the operands, while load and writeback are handled by gvec or by helpers. These principles make future code review and extensibility simpler, at the cost of having a relatively large amount of code in the form of this patch. Even EVEX should not be _too_ hard to implement (it's just a crazy large amount of possibilities). This patch introduces the main decoder flow, and integrates the old decoder with the new one. The old decoder takes care of parsing prefixes and then optionally drops to the new one. The changes to the old decoder are minimal and allow it to be replaced incrementally with the new one. There is a debugging mechanism through a "LIMIT" environment variable. In user-mode emulation, the variable is the number of instructions decoded by the new decoder before permanently switching to the old one. In system emulation, the variable is the highest opcode that is decoded by the new decoder (this is less friendly, but it's the best that can be done without requiring deterministic execution). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-23 12:20:55 +03:00
case X86_TYPE_D: /* REG in the modrm byte selects a debug register */
op->unit = X86_OP_DR;
target/i386: convert MOV from/to CR and DR to new decoder Complete implementation of C and D operand types, then the operations are just MOVs. Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-29 16:55:22 +03:00
op->n = ((get_modrm(s, env) >> 3) & 7) | REX_R(s);
if (op->n >= 8) {
/*
* illegal opcode. The DR4 and DR5 case is checked in the generated
* code instead, to save on hflags bits.
*/
return false;
}
if (decode->e.intercept) {
decode->e.intercept += op->n;
}
break;
target/i386: add core of new i386 decoder The new decoder is based on three principles: - use mostly table-driven decoding, using tables derived as much as possible from the Intel manual. Centralizing the decode the operands makes it more homogeneous, for example all immediates are signed. All modrm handling is in one function, and can be shared between SSE and ALU instructions (including XMM<->GPR instructions). The SSE/AVX decoder will also not have duplicated code between the 0F, 0F38 and 0F3A tables. - keep the code as "non-branchy" as possible. Generally, the code for the new decoder is more verbose, but the control flow is simpler. Conditionals are not nested and have small bodies. All instruction groups are resolved even before operands are decoded, and code generation is separated as much as possible within small functions that only handle one instruction each. - keep address generation and (for ALU operands) memory loads and writeback as much in common code as possible. All ALU operations for example are implemented as T0=f(T0,T1). For non-ALU instructions, read-modify-write memory operations are rare, but registers do not have TCGv equivalents: therefore, the common logic sets up pointer temporaries with the operands, while load and writeback are handled by gvec or by helpers. These principles make future code review and extensibility simpler, at the cost of having a relatively large amount of code in the form of this patch. Even EVEX should not be _too_ hard to implement (it's just a crazy large amount of possibilities). This patch introduces the main decoder flow, and integrates the old decoder with the new one. The old decoder takes care of parsing prefixes and then optionally drops to the new one. The changes to the old decoder are minimal and allow it to be replaced incrementally with the new one. There is a debugging mechanism through a "LIMIT" environment variable. In user-mode emulation, the variable is the number of instructions decoded by the new decoder before permanently switching to the old one. In system emulation, the variable is the highest opcode that is decoded by the new decoder (this is less friendly, but it's the best that can be done without requiring deterministic execution). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-23 12:20:55 +03:00
case X86_TYPE_G: /* REG in the modrm byte selects a GPR */
op->unit = X86_OP_INT;
goto get_reg;
case X86_TYPE_S: /* reg selects a segment register */
op->unit = X86_OP_SEG;
goto get_reg;
case X86_TYPE_P:
op->unit = X86_OP_MMX;
goto get_reg;
case X86_TYPE_V: /* reg in the modrm byte selects an XMM/YMM register */
if (decode->e.special == X86_SPECIAL_MMX &&
!(s->prefix & (PREFIX_DATA | PREFIX_REPZ | PREFIX_REPNZ))) {
op->unit = X86_OP_MMX;
} else {
op->unit = X86_OP_SSE;
}
get_reg:
op->n = ((get_modrm(s, env) >> 3) & 7) | REX_R(s);
break;
case X86_TYPE_E: /* ALU modrm operand */
op->unit = X86_OP_INT;
goto get_modrm;
case X86_TYPE_Q: /* MMX modrm operand */
op->unit = X86_OP_MMX;
goto get_modrm;
case X86_TYPE_W: /* XMM/YMM modrm operand */
if (decode->e.special == X86_SPECIAL_MMX &&
!(s->prefix & (PREFIX_DATA | PREFIX_REPZ | PREFIX_REPNZ))) {
op->unit = X86_OP_MMX;
} else {
op->unit = X86_OP_SSE;
}
goto get_modrm;
case X86_TYPE_N: /* R/M in the modrm byte selects an MMX register */
op->unit = X86_OP_MMX;
goto get_modrm_reg;
case X86_TYPE_U: /* R/M in the modrm byte selects an XMM/YMM register */
if (decode->e.special == X86_SPECIAL_MMX &&
!(s->prefix & (PREFIX_DATA | PREFIX_REPZ | PREFIX_REPNZ))) {
op->unit = X86_OP_MMX;
} else {
op->unit = X86_OP_SSE;
}
goto get_modrm_reg;
case X86_TYPE_R: /* R/M in the modrm byte selects a register */
op->unit = X86_OP_INT;
get_modrm_reg:
modrm = get_modrm(s, env);
if ((modrm >> 6) != 3) {
return false;
}
goto get_modrm;
target/i386: reimplement 0x0f 0xd0-0xd7, 0xe0-0xe7, 0xf0-0xf7, add AVX The more complicated ones here are d6-d7, e6-e7, f7. The others are trivial. For LDDQU, using gen_load_sse directly might corrupt the register if the second part of the load fails. Therefore, add a custom X86_TYPE_WM value; like X86_TYPE_W it does call gen_load(), but it also rejects a value of 11 in the ModRM field like X86_TYPE_M. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-01 15:27:55 +03:00
case X86_TYPE_WM: /* modrm byte selects an XMM/YMM memory operand */
op->unit = X86_OP_SSE;
target/i386: move C0-FF opcodes to new decoder (except for x87) The shift instructions are rewritten instead of reusing code from the old decoder. Rotates use CC_OP_ADCOX more extensively and generally rely more on the optimizer, so that the code generators are shared between the immediate-count and variable-count cases. In particular, this makes gen_RCL and gen_RCR pretty efficient for the count == 1 case, which becomes (apart from a few extra movs) something like: (compute_cc_all if needed) // save old value for OF calculation mov cc_src2, T0 // the bulk of RCL is just this! deposit T0, cc_src, T0, 1, TARGET_LONG_BITS - 1 // compute carry shr cc_dst, cc_src2, length - 1 and cc_dst, cc_dst, 1 // compute overflow xor cc_src2, cc_src2, T0 extract cc_src2, cc_src2, length - 1, 1 32-bit MUL and IMUL are also slightly more efficient on 64-bit hosts. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-21 18:36:34 +03:00
goto get_modrm_mem;
case X86_TYPE_EM: /* modrm byte selects an ALU memory operand */
op->unit = X86_OP_INT;
target/i386: reimplement 0x0f 0xd0-0xd7, 0xe0-0xe7, 0xf0-0xf7, add AVX The more complicated ones here are d6-d7, e6-e7, f7. The others are trivial. For LDDQU, using gen_load_sse directly might corrupt the register if the second part of the load fails. Therefore, add a custom X86_TYPE_WM value; like X86_TYPE_W it does call gen_load(), but it also rejects a value of 11 in the ModRM field like X86_TYPE_M. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-01 15:27:55 +03:00
/* fall through */
target/i386: add core of new i386 decoder The new decoder is based on three principles: - use mostly table-driven decoding, using tables derived as much as possible from the Intel manual. Centralizing the decode the operands makes it more homogeneous, for example all immediates are signed. All modrm handling is in one function, and can be shared between SSE and ALU instructions (including XMM<->GPR instructions). The SSE/AVX decoder will also not have duplicated code between the 0F, 0F38 and 0F3A tables. - keep the code as "non-branchy" as possible. Generally, the code for the new decoder is more verbose, but the control flow is simpler. Conditionals are not nested and have small bodies. All instruction groups are resolved even before operands are decoded, and code generation is separated as much as possible within small functions that only handle one instruction each. - keep address generation and (for ALU operands) memory loads and writeback as much in common code as possible. All ALU operations for example are implemented as T0=f(T0,T1). For non-ALU instructions, read-modify-write memory operations are rare, but registers do not have TCGv equivalents: therefore, the common logic sets up pointer temporaries with the operands, while load and writeback are handled by gvec or by helpers. These principles make future code review and extensibility simpler, at the cost of having a relatively large amount of code in the form of this patch. Even EVEX should not be _too_ hard to implement (it's just a crazy large amount of possibilities). This patch introduces the main decoder flow, and integrates the old decoder with the new one. The old decoder takes care of parsing prefixes and then optionally drops to the new one. The changes to the old decoder are minimal and allow it to be replaced incrementally with the new one. There is a debugging mechanism through a "LIMIT" environment variable. In user-mode emulation, the variable is the number of instructions decoded by the new decoder before permanently switching to the old one. In system emulation, the variable is the highest opcode that is decoded by the new decoder (this is less friendly, but it's the best that can be done without requiring deterministic execution). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-23 12:20:55 +03:00
case X86_TYPE_M: /* modrm byte selects a memory operand */
target/i386: move C0-FF opcodes to new decoder (except for x87) The shift instructions are rewritten instead of reusing code from the old decoder. Rotates use CC_OP_ADCOX more extensively and generally rely more on the optimizer, so that the code generators are shared between the immediate-count and variable-count cases. In particular, this makes gen_RCL and gen_RCR pretty efficient for the count == 1 case, which becomes (apart from a few extra movs) something like: (compute_cc_all if needed) // save old value for OF calculation mov cc_src2, T0 // the bulk of RCL is just this! deposit T0, cc_src, T0, 1, TARGET_LONG_BITS - 1 // compute carry shr cc_dst, cc_src2, length - 1 and cc_dst, cc_dst, 1 // compute overflow xor cc_src2, cc_src2, T0 extract cc_src2, cc_src2, length - 1, 1 32-bit MUL and IMUL are also slightly more efficient on 64-bit hosts. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-21 18:36:34 +03:00
get_modrm_mem:
target/i386: add core of new i386 decoder The new decoder is based on three principles: - use mostly table-driven decoding, using tables derived as much as possible from the Intel manual. Centralizing the decode the operands makes it more homogeneous, for example all immediates are signed. All modrm handling is in one function, and can be shared between SSE and ALU instructions (including XMM<->GPR instructions). The SSE/AVX decoder will also not have duplicated code between the 0F, 0F38 and 0F3A tables. - keep the code as "non-branchy" as possible. Generally, the code for the new decoder is more verbose, but the control flow is simpler. Conditionals are not nested and have small bodies. All instruction groups are resolved even before operands are decoded, and code generation is separated as much as possible within small functions that only handle one instruction each. - keep address generation and (for ALU operands) memory loads and writeback as much in common code as possible. All ALU operations for example are implemented as T0=f(T0,T1). For non-ALU instructions, read-modify-write memory operations are rare, but registers do not have TCGv equivalents: therefore, the common logic sets up pointer temporaries with the operands, while load and writeback are handled by gvec or by helpers. These principles make future code review and extensibility simpler, at the cost of having a relatively large amount of code in the form of this patch. Even EVEX should not be _too_ hard to implement (it's just a crazy large amount of possibilities). This patch introduces the main decoder flow, and integrates the old decoder with the new one. The old decoder takes care of parsing prefixes and then optionally drops to the new one. The changes to the old decoder are minimal and allow it to be replaced incrementally with the new one. There is a debugging mechanism through a "LIMIT" environment variable. In user-mode emulation, the variable is the number of instructions decoded by the new decoder before permanently switching to the old one. In system emulation, the variable is the highest opcode that is decoded by the new decoder (this is less friendly, but it's the best that can be done without requiring deterministic execution). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-23 12:20:55 +03:00
modrm = get_modrm(s, env);
if ((modrm >> 6) == 3) {
return false;
}
target/i386: move prefetch and multi-byte UD/NOP to new decoder These are trivial to add, and moving them to the new decoder fixes some corner cases: raising #UD instead of an instruction fetch page fault for the undefined opcodes, and incorrectly rejecting 0F 18 prefetches with register operands (which are treated as reserved NOPs). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Zhao Liu <zhao1.liu@intel.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-06 18:34:55 +03:00
/* fall through */
case X86_TYPE_nop: /* modrm operand decoded but not fetched */
target/i386: add core of new i386 decoder The new decoder is based on three principles: - use mostly table-driven decoding, using tables derived as much as possible from the Intel manual. Centralizing the decode the operands makes it more homogeneous, for example all immediates are signed. All modrm handling is in one function, and can be shared between SSE and ALU instructions (including XMM<->GPR instructions). The SSE/AVX decoder will also not have duplicated code between the 0F, 0F38 and 0F3A tables. - keep the code as "non-branchy" as possible. Generally, the code for the new decoder is more verbose, but the control flow is simpler. Conditionals are not nested and have small bodies. All instruction groups are resolved even before operands are decoded, and code generation is separated as much as possible within small functions that only handle one instruction each. - keep address generation and (for ALU operands) memory loads and writeback as much in common code as possible. All ALU operations for example are implemented as T0=f(T0,T1). For non-ALU instructions, read-modify-write memory operations are rare, but registers do not have TCGv equivalents: therefore, the common logic sets up pointer temporaries with the operands, while load and writeback are handled by gvec or by helpers. These principles make future code review and extensibility simpler, at the cost of having a relatively large amount of code in the form of this patch. Even EVEX should not be _too_ hard to implement (it's just a crazy large amount of possibilities). This patch introduces the main decoder flow, and integrates the old decoder with the new one. The old decoder takes care of parsing prefixes and then optionally drops to the new one. The changes to the old decoder are minimal and allow it to be replaced incrementally with the new one. There is a debugging mechanism through a "LIMIT" environment variable. In user-mode emulation, the variable is the number of instructions decoded by the new decoder before permanently switching to the old one. In system emulation, the variable is the highest opcode that is decoded by the new decoder (this is less friendly, but it's the best that can be done without requiring deterministic execution). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-23 12:20:55 +03:00
get_modrm:
decode_modrm(s, env, decode, op, type);
break;
case X86_TYPE_O: /* Absolute address encoded in the instruction */
op->unit = X86_OP_INT;
op->has_ea = true;
op->n = -1;
decode->mem = (AddressParts) {
.def_seg = R_DS,
.base = -1,
.index = -1,
.disp = insn_get_addr(env, s, s->aflag)
};
break;
case X86_TYPE_H: /* For AVX, VEX.vvvv selects an XMM/YMM register */
if ((s->prefix & PREFIX_VEX)) {
op->unit = X86_OP_SSE;
op->n = s->vex_v;
break;
}
if (op == &decode->op[0]) {
/* shifts place the destination in VEX.vvvv, use modrm */
return decode_op(s, env, decode, op, decode->e.op1, b);
} else {
return decode_op(s, env, decode, op, decode->e.op0, b);
}
case X86_TYPE_I: /* Immediate */
case X86_TYPE_J: /* Relative offset for a jump */
op->unit = X86_OP_IMM;
target/i386: allow instructions with more than one immediate While keeping decode->immediate for convenience and for 4-operand instructions, store the immediate in X86DecodedOp as well. This enables instructions with more than one immediate such as ENTER. It can also be used for far calls and jumps. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-23 09:41:39 +03:00
decode->immediate = op->imm = insn_get_signed(env, s, op->ot);
target/i386: add core of new i386 decoder The new decoder is based on three principles: - use mostly table-driven decoding, using tables derived as much as possible from the Intel manual. Centralizing the decode the operands makes it more homogeneous, for example all immediates are signed. All modrm handling is in one function, and can be shared between SSE and ALU instructions (including XMM<->GPR instructions). The SSE/AVX decoder will also not have duplicated code between the 0F, 0F38 and 0F3A tables. - keep the code as "non-branchy" as possible. Generally, the code for the new decoder is more verbose, but the control flow is simpler. Conditionals are not nested and have small bodies. All instruction groups are resolved even before operands are decoded, and code generation is separated as much as possible within small functions that only handle one instruction each. - keep address generation and (for ALU operands) memory loads and writeback as much in common code as possible. All ALU operations for example are implemented as T0=f(T0,T1). For non-ALU instructions, read-modify-write memory operations are rare, but registers do not have TCGv equivalents: therefore, the common logic sets up pointer temporaries with the operands, while load and writeback are handled by gvec or by helpers. These principles make future code review and extensibility simpler, at the cost of having a relatively large amount of code in the form of this patch. Even EVEX should not be _too_ hard to implement (it's just a crazy large amount of possibilities). This patch introduces the main decoder flow, and integrates the old decoder with the new one. The old decoder takes care of parsing prefixes and then optionally drops to the new one. The changes to the old decoder are minimal and allow it to be replaced incrementally with the new one. There is a debugging mechanism through a "LIMIT" environment variable. In user-mode emulation, the variable is the number of instructions decoded by the new decoder before permanently switching to the old one. In system emulation, the variable is the highest opcode that is decoded by the new decoder (this is less friendly, but it's the best that can be done without requiring deterministic execution). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-23 12:20:55 +03:00
break;
target/i386: move 60-BF opcodes to new decoder Compared to the old decoder, the main differences in translation are for the little-used ARPL instruction. IMUL is adjusted a bit to share more code to produce flags, but is otherwise very similar. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-11 11:55:16 +03:00
case X86_TYPE_I_unsigned: /* Immediate */
op->unit = X86_OP_IMM;
decode->immediate = op->imm = insn_get(env, s, op->ot);
break;
target/i386: add core of new i386 decoder The new decoder is based on three principles: - use mostly table-driven decoding, using tables derived as much as possible from the Intel manual. Centralizing the decode the operands makes it more homogeneous, for example all immediates are signed. All modrm handling is in one function, and can be shared between SSE and ALU instructions (including XMM<->GPR instructions). The SSE/AVX decoder will also not have duplicated code between the 0F, 0F38 and 0F3A tables. - keep the code as "non-branchy" as possible. Generally, the code for the new decoder is more verbose, but the control flow is simpler. Conditionals are not nested and have small bodies. All instruction groups are resolved even before operands are decoded, and code generation is separated as much as possible within small functions that only handle one instruction each. - keep address generation and (for ALU operands) memory loads and writeback as much in common code as possible. All ALU operations for example are implemented as T0=f(T0,T1). For non-ALU instructions, read-modify-write memory operations are rare, but registers do not have TCGv equivalents: therefore, the common logic sets up pointer temporaries with the operands, while load and writeback are handled by gvec or by helpers. These principles make future code review and extensibility simpler, at the cost of having a relatively large amount of code in the form of this patch. Even EVEX should not be _too_ hard to implement (it's just a crazy large amount of possibilities). This patch introduces the main decoder flow, and integrates the old decoder with the new one. The old decoder takes care of parsing prefixes and then optionally drops to the new one. The changes to the old decoder are minimal and allow it to be replaced incrementally with the new one. There is a debugging mechanism through a "LIMIT" environment variable. In user-mode emulation, the variable is the number of instructions decoded by the new decoder before permanently switching to the old one. In system emulation, the variable is the highest opcode that is decoded by the new decoder (this is less friendly, but it's the best that can be done without requiring deterministic execution). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-23 12:20:55 +03:00
case X86_TYPE_L: /* The upper 4 bits of the immediate select a 128-bit register */
op->n = insn_get(env, s, op->ot) >> 4;
break;
case X86_TYPE_2op:
*op = decode->op[0];
break;
case X86_TYPE_LoBits:
op->n = (b & 7) | REX_B(s);
op->unit = X86_OP_INT;
break;
case X86_TYPE_0 ... X86_TYPE_7:
op->n = type - X86_TYPE_0;
op->unit = X86_OP_INT;
break;
case X86_TYPE_ES ... X86_TYPE_GS:
op->n = type - X86_TYPE_ES;
op->unit = X86_OP_SEG;
break;
}
return true;
}
target/i386: validate SSE prefixes directly in the decoding table Many SSE and AVX instructions are only valid with specific prefixes (none, 66, F3, F2). Introduce a direct way to encode this in the decoding table to avoid using decode groups too much. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-05 16:30:02 +03:00
static bool validate_sse_prefix(DisasContext *s, X86OpEntry *e)
{
uint16_t sse_prefixes;
if (!e->valid_prefix) {
return true;
}
if (s->prefix & (PREFIX_REPZ | PREFIX_REPNZ)) {
/* In SSE instructions, 0xF3 and 0xF2 cancel 0x66. */
s->prefix &= ~PREFIX_DATA;
}
/* Now, either zero or one bit is set in sse_prefixes. */
sse_prefixes = s->prefix & (PREFIX_REPZ | PREFIX_REPNZ | PREFIX_DATA);
return e->valid_prefix & (1 << sse_prefixes);
}
target/i386: add core of new i386 decoder The new decoder is based on three principles: - use mostly table-driven decoding, using tables derived as much as possible from the Intel manual. Centralizing the decode the operands makes it more homogeneous, for example all immediates are signed. All modrm handling is in one function, and can be shared between SSE and ALU instructions (including XMM<->GPR instructions). The SSE/AVX decoder will also not have duplicated code between the 0F, 0F38 and 0F3A tables. - keep the code as "non-branchy" as possible. Generally, the code for the new decoder is more verbose, but the control flow is simpler. Conditionals are not nested and have small bodies. All instruction groups are resolved even before operands are decoded, and code generation is separated as much as possible within small functions that only handle one instruction each. - keep address generation and (for ALU operands) memory loads and writeback as much in common code as possible. All ALU operations for example are implemented as T0=f(T0,T1). For non-ALU instructions, read-modify-write memory operations are rare, but registers do not have TCGv equivalents: therefore, the common logic sets up pointer temporaries with the operands, while load and writeback are handled by gvec or by helpers. These principles make future code review and extensibility simpler, at the cost of having a relatively large amount of code in the form of this patch. Even EVEX should not be _too_ hard to implement (it's just a crazy large amount of possibilities). This patch introduces the main decoder flow, and integrates the old decoder with the new one. The old decoder takes care of parsing prefixes and then optionally drops to the new one. The changes to the old decoder are minimal and allow it to be replaced incrementally with the new one. There is a debugging mechanism through a "LIMIT" environment variable. In user-mode emulation, the variable is the number of instructions decoded by the new decoder before permanently switching to the old one. In system emulation, the variable is the highest opcode that is decoded by the new decoder (this is less friendly, but it's the best that can be done without requiring deterministic execution). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-23 12:20:55 +03:00
static bool decode_insn(DisasContext *s, CPUX86State *env, X86DecodeFunc decode_func,
X86DecodedInsn *decode)
{
X86OpEntry *e = &decode->e;
decode_func(s, env, e, &decode->b);
while (e->is_decode) {
e->is_decode = false;
e->decode(s, env, e, &decode->b);
}
target/i386: validate SSE prefixes directly in the decoding table Many SSE and AVX instructions are only valid with specific prefixes (none, 66, F3, F2). Introduce a direct way to encode this in the decoding table to avoid using decode groups too much. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-05 16:30:02 +03:00
if (!validate_sse_prefix(s, e)) {
return false;
}
target/i386: add core of new i386 decoder The new decoder is based on three principles: - use mostly table-driven decoding, using tables derived as much as possible from the Intel manual. Centralizing the decode the operands makes it more homogeneous, for example all immediates are signed. All modrm handling is in one function, and can be shared between SSE and ALU instructions (including XMM<->GPR instructions). The SSE/AVX decoder will also not have duplicated code between the 0F, 0F38 and 0F3A tables. - keep the code as "non-branchy" as possible. Generally, the code for the new decoder is more verbose, but the control flow is simpler. Conditionals are not nested and have small bodies. All instruction groups are resolved even before operands are decoded, and code generation is separated as much as possible within small functions that only handle one instruction each. - keep address generation and (for ALU operands) memory loads and writeback as much in common code as possible. All ALU operations for example are implemented as T0=f(T0,T1). For non-ALU instructions, read-modify-write memory operations are rare, but registers do not have TCGv equivalents: therefore, the common logic sets up pointer temporaries with the operands, while load and writeback are handled by gvec or by helpers. These principles make future code review and extensibility simpler, at the cost of having a relatively large amount of code in the form of this patch. Even EVEX should not be _too_ hard to implement (it's just a crazy large amount of possibilities). This patch introduces the main decoder flow, and integrates the old decoder with the new one. The old decoder takes care of parsing prefixes and then optionally drops to the new one. The changes to the old decoder are minimal and allow it to be replaced incrementally with the new one. There is a debugging mechanism through a "LIMIT" environment variable. In user-mode emulation, the variable is the number of instructions decoded by the new decoder before permanently switching to the old one. In system emulation, the variable is the highest opcode that is decoded by the new decoder (this is less friendly, but it's the best that can be done without requiring deterministic execution). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-23 12:20:55 +03:00
/* First compute size of operands in order to initialize s->rip_offset. */
if (e->op0 != X86_TYPE_None) {
if (!decode_op_size(s, e, e->s0, &decode->op[0].ot)) {
return false;
}
if (e->op0 == X86_TYPE_I) {
s->rip_offset += 1 << decode->op[0].ot;
}
}
if (e->op1 != X86_TYPE_None) {
if (!decode_op_size(s, e, e->s1, &decode->op[1].ot)) {
return false;
}
if (e->op1 == X86_TYPE_I) {
s->rip_offset += 1 << decode->op[1].ot;
}
}
if (e->op2 != X86_TYPE_None) {
if (!decode_op_size(s, e, e->s2, &decode->op[2].ot)) {
return false;
}
if (e->op2 == X86_TYPE_I) {
s->rip_offset += 1 << decode->op[2].ot;
}
}
if (e->op3 != X86_TYPE_None) {
target/i386: reimplement 0x0f 0x3a, add AVX The more complicated operations here are insertions and extractions. Otherwise, there are just more entries than usual because the PS/PD/SS/SD variations are encoded in the opcode rater than in the prefixes. These three-byte opcodes also include AVX new instructions, whose implementation in the helpers was originally done by Paul Brook <paul@nowt.org>. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-06 11:34:11 +03:00
/*
* A couple instructions actually use the extra immediate byte for an Lx
* register operand; those are handled in the gen_* functions as one off.
*/
target/i386: add core of new i386 decoder The new decoder is based on three principles: - use mostly table-driven decoding, using tables derived as much as possible from the Intel manual. Centralizing the decode the operands makes it more homogeneous, for example all immediates are signed. All modrm handling is in one function, and can be shared between SSE and ALU instructions (including XMM<->GPR instructions). The SSE/AVX decoder will also not have duplicated code between the 0F, 0F38 and 0F3A tables. - keep the code as "non-branchy" as possible. Generally, the code for the new decoder is more verbose, but the control flow is simpler. Conditionals are not nested and have small bodies. All instruction groups are resolved even before operands are decoded, and code generation is separated as much as possible within small functions that only handle one instruction each. - keep address generation and (for ALU operands) memory loads and writeback as much in common code as possible. All ALU operations for example are implemented as T0=f(T0,T1). For non-ALU instructions, read-modify-write memory operations are rare, but registers do not have TCGv equivalents: therefore, the common logic sets up pointer temporaries with the operands, while load and writeback are handled by gvec or by helpers. These principles make future code review and extensibility simpler, at the cost of having a relatively large amount of code in the form of this patch. Even EVEX should not be _too_ hard to implement (it's just a crazy large amount of possibilities). This patch introduces the main decoder flow, and integrates the old decoder with the new one. The old decoder takes care of parsing prefixes and then optionally drops to the new one. The changes to the old decoder are minimal and allow it to be replaced incrementally with the new one. There is a debugging mechanism through a "LIMIT" environment variable. In user-mode emulation, the variable is the number of instructions decoded by the new decoder before permanently switching to the old one. In system emulation, the variable is the highest opcode that is decoded by the new decoder (this is less friendly, but it's the best that can be done without requiring deterministic execution). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-23 12:20:55 +03:00
assert(e->op3 == X86_TYPE_I && e->s3 == X86_SIZE_b);
s->rip_offset += 1;
}
if (e->op0 != X86_TYPE_None &&
!decode_op(s, env, decode, &decode->op[0], e->op0, decode->b)) {
return false;
}
if (e->op1 != X86_TYPE_None &&
!decode_op(s, env, decode, &decode->op[1], e->op1, decode->b)) {
return false;
}
if (e->op2 != X86_TYPE_None &&
!decode_op(s, env, decode, &decode->op[2], e->op2, decode->b)) {
return false;
}
if (e->op3 != X86_TYPE_None) {
decode->immediate = insn_get_signed(env, s, MO_8);
}
return true;
}
target/i386: add CPUID feature checks to new decoder Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-01 15:51:35 +03:00
static bool has_cpuid_feature(DisasContext *s, X86CPUIDFeature cpuid)
{
switch (cpuid) {
case X86_FEAT_None:
return true;
target/i386: move remaining conditional operations to new decoder Move long-displacement Jcc, SETcc and CMOVcc to the new decoder. While filling in the tables makes the code seem longer, the new emitters are all just one line of code. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-11 12:51:58 +03:00
case X86_FEAT_CMOV:
return (s->cpuid_features & CPUID_CMOV);
target/i386: finish converting 0F AE to the new decoder This is already partly implemented due to VLDMXCSR and VSTMXCSR; finish the job. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-08 18:45:53 +03:00
case X86_FEAT_CLFLUSH:
return (s->cpuid_features & CPUID_CLFLUSH);
case X86_FEAT_FXSR:
return (s->cpuid_features & CPUID_FXSR);
target/i386: implement F16C instructions F16C only consists of two instructions, which are a bit peculiar nevertheless. First, they access only the low half of an YMM or XMM register for the packed-half operand; the exact size still depends on the VEX.L flag. This is similar to the existing avx_movx flag, but not exactly because avx_movx is hardcoded to affect operand 2. To this end I added a "ph" format name; it's possible to reuse this approach for the VPMOVSX and VPMOVZX instructions, though that would also require adding two more formats for the low-quarter and low-eighth of an operand. Second, VCVTPS2PH is somewhat weird because it *stores* the result of the instruction into memory rather than loading it. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-10-19 14:22:06 +03:00
case X86_FEAT_F16C:
return (s->cpuid_ext_features & CPUID_EXT_F16C);
target/i386: implement FMA instructions The only issue with FMA instructions is that there are _a lot_ of them (30 opcodes, each of which comes in up to 4 versions depending on VEX.W and VEX.L; a total of 96 possibilities). However, they can be implement with only 6 helpers, two for scalar operations and four for packed operations. (Scalar versions do not do any merging; they only affect the bottom 32 or 64 bits of the output operand. Therefore, there is no separate XMM and YMM of the scalar helpers). First, we can reduce the number of helpers to one third by passing four operands (one output and three inputs); the reordering of which operands go to the multiply and which go to the add is done in emit.c. Second, the different instructions also dispatch to the same softfloat function, so the flags for float32_muladd and float64_muladd are passed in the helper as int arguments, with a little extra complication to handle FMADDSUB and FMSUBADD. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-10-19 14:22:06 +03:00
case X86_FEAT_FMA:
return (s->cpuid_ext_features & CPUID_EXT_FMA);
target/i386: add CPUID feature checks to new decoder Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-01 15:51:35 +03:00
case X86_FEAT_MOVBE:
return (s->cpuid_ext_features & CPUID_EXT_MOVBE);
case X86_FEAT_PCLMULQDQ:
return (s->cpuid_ext_features & CPUID_EXT_PCLMULQDQ);
target/i386: convert LZCNT/TZCNT/BSF/BSR/POPCNT to new decoder Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-09 16:11:41 +03:00
case X86_FEAT_POPCNT:
return (s->cpuid_ext_features & CPUID_EXT_POPCNT);
target/i386: add CPUID feature checks to new decoder Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-01 15:51:35 +03:00
case X86_FEAT_SSE:
target/i386: fix SSE and SSE2 feature check Features check of CPUID_SSE and CPUID_SSE2 should use cpuid_features, rather than cpuid_ext_features. Signed-off-by: Xinyu Li <lixinyu20s@ict.ac.cn> Reviewed-by: Zhao Liu <zhao1.liu@intel.com> Message-ID: <20240602100904.2137939-1-lixinyu20s@ict.ac.cn> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-06-02 13:09:04 +03:00
return (s->cpuid_features & CPUID_SSE);
target/i386: add CPUID feature checks to new decoder Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-01 15:51:35 +03:00
case X86_FEAT_SSE2:
target/i386: fix SSE and SSE2 feature check Features check of CPUID_SSE and CPUID_SSE2 should use cpuid_features, rather than cpuid_ext_features. Signed-off-by: Xinyu Li <lixinyu20s@ict.ac.cn> Reviewed-by: Zhao Liu <zhao1.liu@intel.com> Message-ID: <20240602100904.2137939-1-lixinyu20s@ict.ac.cn> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-06-02 13:09:04 +03:00
return (s->cpuid_features & CPUID_SSE2);
target/i386: add CPUID feature checks to new decoder Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-01 15:51:35 +03:00
case X86_FEAT_SSE3:
return (s->cpuid_ext_features & CPUID_EXT_SSE3);
case X86_FEAT_SSSE3:
return (s->cpuid_ext_features & CPUID_EXT_SSSE3);
case X86_FEAT_SSE41:
return (s->cpuid_ext_features & CPUID_EXT_SSE41);
case X86_FEAT_SSE42:
return (s->cpuid_ext_features & CPUID_EXT_SSE42);
case X86_FEAT_AES:
if (!(s->cpuid_ext_features & CPUID_EXT_AES)) {
return false;
} else if (!(s->prefix & PREFIX_VEX)) {
return true;
} else if (!(s->cpuid_ext_features & CPUID_EXT_AVX)) {
return false;
} else {
return !s->vex_l || (s->cpuid_7_0_ecx_features & CPUID_7_0_ECX_VAES);
}
case X86_FEAT_AVX:
return (s->cpuid_ext_features & CPUID_EXT_AVX);
target/i386: finish converting 0F AE to the new decoder This is already partly implemented due to VLDMXCSR and VSTMXCSR; finish the job. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-08 18:45:53 +03:00
case X86_FEAT_XSAVE:
return (s->cpuid_ext_features & CPUID_EXT_XSAVE);
target/i386: add CPUID feature checks to new decoder Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-01 15:51:35 +03:00
target/i386: move 3DNow to the new decoder This adds another kind of weirdness when you thought you had seen it all: an opcode byte that comes _after_ the address, not before. It's not worth adding a new X86_SPECIAL_* constant for it, but it's actually not unlike VCMP; so, forgive me for exploiting the similarity and just deciding to dispatch to the right gen_helper_* call in a single code generation function. In fact, the old decoder had a bug where s->rip_offset should have been set to 1 for 3DNow! instructions, and it's fixed now. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-06 00:27:53 +03:00
case X86_FEAT_3DNOW:
return (s->cpuid_ext2_features & CPUID_EXT2_3DNOW);
target/i386: add CPUID feature checks to new decoder Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-01 15:51:35 +03:00
case X86_FEAT_SSE4A:
return (s->cpuid_ext3_features & CPUID_EXT3_SSE4A);
case X86_FEAT_ADX:
return (s->cpuid_7_0_ebx_features & CPUID_7_0_EBX_ADX);
case X86_FEAT_BMI1:
return (s->cpuid_7_0_ebx_features & CPUID_7_0_EBX_BMI1);
case X86_FEAT_BMI2:
return (s->cpuid_7_0_ebx_features & CPUID_7_0_EBX_BMI2);
case X86_FEAT_AVX2:
return (s->cpuid_7_0_ebx_features & CPUID_7_0_EBX_AVX2);
target/i386: finish converting 0F AE to the new decoder This is already partly implemented due to VLDMXCSR and VSTMXCSR; finish the job. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-08 18:45:53 +03:00
case X86_FEAT_CLFLUSHOPT:
return (s->cpuid_7_0_ebx_features & CPUID_7_0_EBX_CLFLUSHOPT);
case X86_FEAT_CLWB:
return (s->cpuid_7_0_ebx_features & CPUID_7_0_EBX_CLWB);
case X86_FEAT_FSGSBASE:
return (s->cpuid_7_0_ebx_features & CPUID_7_0_EBX_FSGSBASE);
target/i386: implement SHA instructions The implementation was validated with OpenSSL and with the test vectors in https://github.com/rust-lang/stdarch/blob/master/crates/core_arch/src/x86/sha.rs. The instructions provide a ~25% improvement on hashing a 64 MiB file: runtime goes down from 1.8 seconds to 1.4 seconds; instruction count on the host goes down from 5.8 billion to 4.8 billion with slightly better IPC too. Good job Intel. ;) Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-10 11:31:17 +03:00
case X86_FEAT_SHA_NI:
return (s->cpuid_7_0_ebx_features & CPUID_7_0_EBX_SHA_NI);
target/i386: implement CMPccXADD The main difficulty here is that a page fault when writing to the destination must not overwrite the flags. Therefore, the flags computation must be inlined instead of using gen_jcc1*. For simplicity, I am using an unconditional cmpxchg operation, that becomes a NOP if the comparison fails. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-10 11:31:39 +03:00
case X86_FEAT_CMPCCXADD:
return (s->cpuid_7_1_eax_features & CPUID_7_1_EAX_CMPCCXADD);
target/i386: finish converting 0F AE to the new decoder This is already partly implemented due to VLDMXCSR and VSTMXCSR; finish the job. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-08 18:45:53 +03:00
case X86_FEAT_XSAVEOPT:
return (s->cpuid_xsave_features & CPUID_XSAVE_XSAVEOPT);
target/i386: add CPUID feature checks to new decoder Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-01 15:51:35 +03:00
}
g_assert_not_reached();
}
target/i386: validate VEX prefixes via the instructions' exception classes Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-18 01:43:52 +03:00
static bool validate_vex(DisasContext *s, X86DecodedInsn *decode)
{
X86OpEntry *e = &decode->e;
switch (e->vex_special) {
case X86_VEX_REPScalar:
/*
* Instructions which differ between 00/66 and F2/F3 in the
* exception classification and the size of the memory operand.
*/
target/i386: fix operand size of unary SSE operations VRCPSS, VRSQRTSS and VCVTSx2Sx have a 32-bit or 64-bit memory operand, which is represented in the decoding tables by X86_VEX_REPScalar. Add it to the tables, and make validate_vex() handle the case of an instruction that is in exception type 4 without the REP prefix and exception type 5 with it; this is the cas of VRCP and VRSQRT. Reported-by: yongwoo <https://gitlab.com/yongwoo36> Resolves: https://gitlab.com/qemu-project/qemu/-/issues/1377 Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-01-07 20:14:20 +03:00
assert(e->vex_class == 1 || e->vex_class == 2 || e->vex_class == 4);
target/i386: validate VEX prefixes via the instructions' exception classes Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-18 01:43:52 +03:00
if (s->prefix & (PREFIX_REPZ | PREFIX_REPNZ)) {
target/i386: fix operand size of unary SSE operations VRCPSS, VRSQRTSS and VCVTSx2Sx have a 32-bit or 64-bit memory operand, which is represented in the decoding tables by X86_VEX_REPScalar. Add it to the tables, and make validate_vex() handle the case of an instruction that is in exception type 4 without the REP prefix and exception type 5 with it; this is the cas of VRCP and VRSQRT. Reported-by: yongwoo <https://gitlab.com/yongwoo36> Resolves: https://gitlab.com/qemu-project/qemu/-/issues/1377 Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-01-07 20:14:20 +03:00
e->vex_class = e->vex_class < 4 ? 3 : 5;
target/i386: validate VEX prefixes via the instructions' exception classes Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-18 01:43:52 +03:00
if (s->vex_l) {
goto illegal;
}
assert(decode->e.s2 == X86_SIZE_x);
if (decode->op[2].has_ea) {
decode->op[2].ot = s->prefix & PREFIX_REPZ ? MO_32 : MO_64;
}
}
break;
case X86_VEX_SSEUnaligned:
/* handled in sse_needs_alignment. */
break;
case X86_VEX_AVX2_256:
if ((s->prefix & PREFIX_VEX) && s->vex_l && !has_cpuid_feature(s, X86_FEAT_AVX2)) {
goto illegal;
}
}
switch (e->vex_class) {
case 0:
if (s->prefix & PREFIX_VEX) {
goto illegal;
}
return true;
case 1:
case 2:
case 3:
case 4:
case 5:
case 7:
if (s->prefix & PREFIX_VEX) {
if (!(s->flags & HF_AVX_EN_MASK)) {
goto illegal;
}
target/i386: allow MMX instructions with CR4.OSFXSR=0 MMX state is saved/restored by FSAVE/FRSTOR so the instructions are not illegal opcodes even if CR4.OSFXSR=0. Make sure that validate_vex takes into account the prefix and only checks HF_OSFXSR_MASK in the presence of an SSE instruction. Fixes: 20581aadec5e ("target/i386: validate VEX prefixes via the instructions' exception classes", 2022-10-18) Resolves: https://gitlab.com/qemu-project/qemu/-/issues/1350 Reported-by: Helge Konetzka (@hejko on gitlab.com) Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-11-30 17:16:57 +03:00
} else if (e->special != X86_SPECIAL_MMX ||
(s->prefix & (PREFIX_REPZ | PREFIX_REPNZ | PREFIX_DATA))) {
target/i386: validate VEX prefixes via the instructions' exception classes Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-18 01:43:52 +03:00
if (!(s->flags & HF_OSFXSR_MASK)) {
goto illegal;
}
}
break;
case 12:
/* Must have a VSIB byte and no address prefix. */
assert(s->has_modrm);
if ((s->modrm & 7) != 4 || s->aflag == MO_16) {
goto illegal;
}
/* Check no overlap between registers. */
if (!decode->op[0].has_ea &&
(decode->op[0].n == decode->mem.index || decode->op[0].n == decode->op[1].n)) {
goto illegal;
}
assert(!decode->op[1].has_ea);
if (decode->op[1].n == decode->mem.index) {
goto illegal;
}
if (!decode->op[2].has_ea &&
(decode->op[2].n == decode->mem.index || decode->op[2].n == decode->op[1].n)) {
goto illegal;
}
/* fall through */
case 6:
case 11:
if (!(s->prefix & PREFIX_VEX)) {
goto illegal;
}
if (!(s->flags & HF_AVX_EN_MASK)) {
goto illegal;
}
break;
case 8:
target/i386: reimplement 0x0f 0x70-0x77, add AVX This includes shifts by immediate, which use bits 3-5 of the ModRM byte as an opcode extension. With the exception of 128-bit shifts, they are implemented using gvec. This also covers VZEROALL and VZEROUPPER, which use the same opcode as EMMS. If we were wanting to optimize out gen_clear_ymmh then this would be one of the starting points. The implementation of the VZEROALL and VZEROUPPER helpers is by Paul Brook. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-02 19:19:06 +03:00
/* Non-VEX case handled in decode_0F77. */
assert(s->prefix & PREFIX_VEX);
target/i386: validate VEX prefixes via the instructions' exception classes Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-18 01:43:52 +03:00
if (!(s->flags & HF_AVX_EN_MASK)) {
goto illegal;
}
break;
case 13:
if (!(s->prefix & PREFIX_VEX)) {
goto illegal;
}
if (s->vex_l) {
goto illegal;
}
/* All integer instructions use VEX.vvvv, so exit. */
return true;
}
if (s->vex_v != 0 &&
e->op0 != X86_TYPE_H && e->op0 != X86_TYPE_B &&
e->op1 != X86_TYPE_H && e->op1 != X86_TYPE_B &&
e->op2 != X86_TYPE_H && e->op2 != X86_TYPE_B) {
goto illegal;
}
if (s->flags & HF_TS_MASK) {
goto nm_exception;
}
if (s->flags & HF_EM_MASK) {
goto illegal;
}
target/i386: group common checks in the decoding phase In preparation for adding more similar checks, move the VEX.L=0 check and several X86_SPECIAL_* checks to a new field, where each bit represent a common check on unused bits, or a restriction on the processor mode. Likewise, many SVM intercepts can be checked during the decoding phase, the main exception being the selective CR0 write, MSR and IOIO intercepts. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-09 18:43:12 +03:00
target/i386: validate VEX.W for AVX instructions Instructions in VEX exception class 6 generally look at the value of VEX.W. Note that the manual places some instructions incorrectly in class 4, for example VPERMQ which has no non-VEX encoding and no legacy SSE analogue. AMD does a mess of its own, as documented in the comment that this patch adds. Most of them are checked for VEX.W=0, and are listed in the manual (though with an omission) in table 2-16; VPERMQ and VPERMPD check for VEX.W=1, which is only listed in the instruction description. Others, such as VPSRLV, VPSLLV and the FMA3 instructions, use VEX.W to switch between a 32-bit and 64-bit operation. Fix more of the class 4/class 6 mismatches, and implement the check for VEX.W in TCG. Acked-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-09 19:16:27 +03:00
if (e->check) {
if (e->check & X86_CHECK_VEX128) {
if (s->vex_l) {
goto illegal;
}
}
if (e->check & X86_CHECK_W0) {
if (s->vex_w) {
goto illegal;
}
}
if (e->check & X86_CHECK_W1) {
if (!s->vex_w) {
goto illegal;
}
target/i386: group common checks in the decoding phase In preparation for adding more similar checks, move the VEX.L=0 check and several X86_SPECIAL_* checks to a new field, where each bit represent a common check on unused bits, or a restriction on the processor mode. Likewise, many SVM intercepts can be checked during the decoding phase, the main exception being the selective CR0 write, MSR and IOIO intercepts. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-09 18:43:12 +03:00
}
}
target/i386: validate VEX prefixes via the instructions' exception classes Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-18 01:43:52 +03:00
return true;
nm_exception:
gen_NM_exception(s);
return false;
illegal:
gen_illegal_opcode(s);
return false;
}
target/i386: add core of new i386 decoder The new decoder is based on three principles: - use mostly table-driven decoding, using tables derived as much as possible from the Intel manual. Centralizing the decode the operands makes it more homogeneous, for example all immediates are signed. All modrm handling is in one function, and can be shared between SSE and ALU instructions (including XMM<->GPR instructions). The SSE/AVX decoder will also not have duplicated code between the 0F, 0F38 and 0F3A tables. - keep the code as "non-branchy" as possible. Generally, the code for the new decoder is more verbose, but the control flow is simpler. Conditionals are not nested and have small bodies. All instruction groups are resolved even before operands are decoded, and code generation is separated as much as possible within small functions that only handle one instruction each. - keep address generation and (for ALU operands) memory loads and writeback as much in common code as possible. All ALU operations for example are implemented as T0=f(T0,T1). For non-ALU instructions, read-modify-write memory operations are rare, but registers do not have TCGv equivalents: therefore, the common logic sets up pointer temporaries with the operands, while load and writeback are handled by gvec or by helpers. These principles make future code review and extensibility simpler, at the cost of having a relatively large amount of code in the form of this patch. Even EVEX should not be _too_ hard to implement (it's just a crazy large amount of possibilities). This patch introduces the main decoder flow, and integrates the old decoder with the new one. The old decoder takes care of parsing prefixes and then optionally drops to the new one. The changes to the old decoder are minimal and allow it to be replaced incrementally with the new one. There is a debugging mechanism through a "LIMIT" environment variable. In user-mode emulation, the variable is the number of instructions decoded by the new decoder before permanently switching to the old one. In system emulation, the variable is the highest opcode that is decoded by the new decoder (this is less friendly, but it's the best that can be done without requiring deterministic execution). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-23 12:20:55 +03:00
/*
* Convert one instruction. s->base.is_jmp is set if the translation must
* be stopped.
*/
target/i386: remove duplicate prefix decoding Now that a bulk of opcodes go through the new decoder, it is sensible to do some cleanup. Go immediately through disas_insn_new and only jump back after parsing the prefixes. disas_insn() now only contains the three sigsetjmp cases, and they are more easily managed if they are inlined into i386_tr_translate_insn. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-04-09 18:31:23 +03:00
static void disas_insn(DisasContext *s, CPUState *cpu)
target/i386: add core of new i386 decoder The new decoder is based on three principles: - use mostly table-driven decoding, using tables derived as much as possible from the Intel manual. Centralizing the decode the operands makes it more homogeneous, for example all immediates are signed. All modrm handling is in one function, and can be shared between SSE and ALU instructions (including XMM<->GPR instructions). The SSE/AVX decoder will also not have duplicated code between the 0F, 0F38 and 0F3A tables. - keep the code as "non-branchy" as possible. Generally, the code for the new decoder is more verbose, but the control flow is simpler. Conditionals are not nested and have small bodies. All instruction groups are resolved even before operands are decoded, and code generation is separated as much as possible within small functions that only handle one instruction each. - keep address generation and (for ALU operands) memory loads and writeback as much in common code as possible. All ALU operations for example are implemented as T0=f(T0,T1). For non-ALU instructions, read-modify-write memory operations are rare, but registers do not have TCGv equivalents: therefore, the common logic sets up pointer temporaries with the operands, while load and writeback are handled by gvec or by helpers. These principles make future code review and extensibility simpler, at the cost of having a relatively large amount of code in the form of this patch. Even EVEX should not be _too_ hard to implement (it's just a crazy large amount of possibilities). This patch introduces the main decoder flow, and integrates the old decoder with the new one. The old decoder takes care of parsing prefixes and then optionally drops to the new one. The changes to the old decoder are minimal and allow it to be replaced incrementally with the new one. There is a debugging mechanism through a "LIMIT" environment variable. In user-mode emulation, the variable is the number of instructions decoded by the new decoder before permanently switching to the old one. In system emulation, the variable is the highest opcode that is decoded by the new decoder (this is less friendly, but it's the best that can be done without requiring deterministic execution). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-23 12:20:55 +03:00
{
accel/tcg: Replace CPUState.env_ptr with cpu_env() Reviewed-by: Anton Johansson <anjo@rev.ng> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-09-14 03:22:49 +03:00
CPUX86State *env = cpu_env(cpu);
target/i386: add core of new i386 decoder The new decoder is based on three principles: - use mostly table-driven decoding, using tables derived as much as possible from the Intel manual. Centralizing the decode the operands makes it more homogeneous, for example all immediates are signed. All modrm handling is in one function, and can be shared between SSE and ALU instructions (including XMM<->GPR instructions). The SSE/AVX decoder will also not have duplicated code between the 0F, 0F38 and 0F3A tables. - keep the code as "non-branchy" as possible. Generally, the code for the new decoder is more verbose, but the control flow is simpler. Conditionals are not nested and have small bodies. All instruction groups are resolved even before operands are decoded, and code generation is separated as much as possible within small functions that only handle one instruction each. - keep address generation and (for ALU operands) memory loads and writeback as much in common code as possible. All ALU operations for example are implemented as T0=f(T0,T1). For non-ALU instructions, read-modify-write memory operations are rare, but registers do not have TCGv equivalents: therefore, the common logic sets up pointer temporaries with the operands, while load and writeback are handled by gvec or by helpers. These principles make future code review and extensibility simpler, at the cost of having a relatively large amount of code in the form of this patch. Even EVEX should not be _too_ hard to implement (it's just a crazy large amount of possibilities). This patch introduces the main decoder flow, and integrates the old decoder with the new one. The old decoder takes care of parsing prefixes and then optionally drops to the new one. The changes to the old decoder are minimal and allow it to be replaced incrementally with the new one. There is a debugging mechanism through a "LIMIT" environment variable. In user-mode emulation, the variable is the number of instructions decoded by the new decoder before permanently switching to the old one. In system emulation, the variable is the highest opcode that is decoded by the new decoder (this is less friendly, but it's the best that can be done without requiring deterministic execution). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-23 12:20:55 +03:00
X86DecodedInsn decode;
X86DecodeFunc decode_func = decode_root;
target/i386: remove duplicate prefix decoding Now that a bulk of opcodes go through the new decoder, it is sensible to do some cleanup. Go immediately through disas_insn_new and only jump back after parsing the prefixes. disas_insn() now only contains the three sigsetjmp cases, and they are more easily managed if they are inlined into i386_tr_translate_insn. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-04-09 18:31:23 +03:00
uint8_t cc_live, b;
target/i386: add core of new i386 decoder The new decoder is based on three principles: - use mostly table-driven decoding, using tables derived as much as possible from the Intel manual. Centralizing the decode the operands makes it more homogeneous, for example all immediates are signed. All modrm handling is in one function, and can be shared between SSE and ALU instructions (including XMM<->GPR instructions). The SSE/AVX decoder will also not have duplicated code between the 0F, 0F38 and 0F3A tables. - keep the code as "non-branchy" as possible. Generally, the code for the new decoder is more verbose, but the control flow is simpler. Conditionals are not nested and have small bodies. All instruction groups are resolved even before operands are decoded, and code generation is separated as much as possible within small functions that only handle one instruction each. - keep address generation and (for ALU operands) memory loads and writeback as much in common code as possible. All ALU operations for example are implemented as T0=f(T0,T1). For non-ALU instructions, read-modify-write memory operations are rare, but registers do not have TCGv equivalents: therefore, the common logic sets up pointer temporaries with the operands, while load and writeback are handled by gvec or by helpers. These principles make future code review and extensibility simpler, at the cost of having a relatively large amount of code in the form of this patch. Even EVEX should not be _too_ hard to implement (it's just a crazy large amount of possibilities). This patch introduces the main decoder flow, and integrates the old decoder with the new one. The old decoder takes care of parsing prefixes and then optionally drops to the new one. The changes to the old decoder are minimal and allow it to be replaced incrementally with the new one. There is a debugging mechanism through a "LIMIT" environment variable. In user-mode emulation, the variable is the number of instructions decoded by the new decoder before permanently switching to the old one. In system emulation, the variable is the highest opcode that is decoded by the new decoder (this is less friendly, but it's the best that can be done without requiring deterministic execution). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-23 12:20:55 +03:00
target/i386: remove duplicate prefix decoding Now that a bulk of opcodes go through the new decoder, it is sensible to do some cleanup. Go immediately through disas_insn_new and only jump back after parsing the prefixes. disas_insn() now only contains the three sigsetjmp cases, and they are more easily managed if they are inlined into i386_tr_translate_insn. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-04-09 18:31:23 +03:00
s->pc = s->base.pc_next;
s->override = -1;
s->popl_esp_hack = 0;
#ifdef TARGET_X86_64
s->rex_r = 0;
s->rex_x = 0;
s->rex_b = 0;
#endif
s->rip_offset = 0; /* for relative ip address */
s->vex_l = 0;
s->vex_v = 0;
s->vex_w = false;
target/i386: add core of new i386 decoder The new decoder is based on three principles: - use mostly table-driven decoding, using tables derived as much as possible from the Intel manual. Centralizing the decode the operands makes it more homogeneous, for example all immediates are signed. All modrm handling is in one function, and can be shared between SSE and ALU instructions (including XMM<->GPR instructions). The SSE/AVX decoder will also not have duplicated code between the 0F, 0F38 and 0F3A tables. - keep the code as "non-branchy" as possible. Generally, the code for the new decoder is more verbose, but the control flow is simpler. Conditionals are not nested and have small bodies. All instruction groups are resolved even before operands are decoded, and code generation is separated as much as possible within small functions that only handle one instruction each. - keep address generation and (for ALU operands) memory loads and writeback as much in common code as possible. All ALU operations for example are implemented as T0=f(T0,T1). For non-ALU instructions, read-modify-write memory operations are rare, but registers do not have TCGv equivalents: therefore, the common logic sets up pointer temporaries with the operands, while load and writeback are handled by gvec or by helpers. These principles make future code review and extensibility simpler, at the cost of having a relatively large amount of code in the form of this patch. Even EVEX should not be _too_ hard to implement (it's just a crazy large amount of possibilities). This patch introduces the main decoder flow, and integrates the old decoder with the new one. The old decoder takes care of parsing prefixes and then optionally drops to the new one. The changes to the old decoder are minimal and allow it to be replaced incrementally with the new one. There is a debugging mechanism through a "LIMIT" environment variable. In user-mode emulation, the variable is the number of instructions decoded by the new decoder before permanently switching to the old one. In system emulation, the variable is the highest opcode that is decoded by the new decoder (this is less friendly, but it's the best that can be done without requiring deterministic execution). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-23 12:20:55 +03:00
s->has_modrm = false;
target/i386: remove duplicate prefix decoding Now that a bulk of opcodes go through the new decoder, it is sensible to do some cleanup. Go immediately through disas_insn_new and only jump back after parsing the prefixes. disas_insn() now only contains the three sigsetjmp cases, and they are more easily managed if they are inlined into i386_tr_translate_insn. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-04-09 18:31:23 +03:00
s->prefix = 0;
target/i386: add core of new i386 decoder The new decoder is based on three principles: - use mostly table-driven decoding, using tables derived as much as possible from the Intel manual. Centralizing the decode the operands makes it more homogeneous, for example all immediates are signed. All modrm handling is in one function, and can be shared between SSE and ALU instructions (including XMM<->GPR instructions). The SSE/AVX decoder will also not have duplicated code between the 0F, 0F38 and 0F3A tables. - keep the code as "non-branchy" as possible. Generally, the code for the new decoder is more verbose, but the control flow is simpler. Conditionals are not nested and have small bodies. All instruction groups are resolved even before operands are decoded, and code generation is separated as much as possible within small functions that only handle one instruction each. - keep address generation and (for ALU operands) memory loads and writeback as much in common code as possible. All ALU operations for example are implemented as T0=f(T0,T1). For non-ALU instructions, read-modify-write memory operations are rare, but registers do not have TCGv equivalents: therefore, the common logic sets up pointer temporaries with the operands, while load and writeback are handled by gvec or by helpers. These principles make future code review and extensibility simpler, at the cost of having a relatively large amount of code in the form of this patch. Even EVEX should not be _too_ hard to implement (it's just a crazy large amount of possibilities). This patch introduces the main decoder flow, and integrates the old decoder with the new one. The old decoder takes care of parsing prefixes and then optionally drops to the new one. The changes to the old decoder are minimal and allow it to be replaced incrementally with the new one. There is a debugging mechanism through a "LIMIT" environment variable. In user-mode emulation, the variable is the number of instructions decoded by the new decoder before permanently switching to the old one. In system emulation, the variable is the highest opcode that is decoded by the new decoder (this is less friendly, but it's the best that can be done without requiring deterministic execution). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-23 12:20:55 +03:00
next_byte:
target/i386: remove duplicate prefix decoding Now that a bulk of opcodes go through the new decoder, it is sensible to do some cleanup. Go immediately through disas_insn_new and only jump back after parsing the prefixes. disas_insn() now only contains the three sigsetjmp cases, and they are more easily managed if they are inlined into i386_tr_translate_insn. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-04-09 18:31:23 +03:00
b = x86_ldub_code(env, s);
target/i386: add core of new i386 decoder The new decoder is based on three principles: - use mostly table-driven decoding, using tables derived as much as possible from the Intel manual. Centralizing the decode the operands makes it more homogeneous, for example all immediates are signed. All modrm handling is in one function, and can be shared between SSE and ALU instructions (including XMM<->GPR instructions). The SSE/AVX decoder will also not have duplicated code between the 0F, 0F38 and 0F3A tables. - keep the code as "non-branchy" as possible. Generally, the code for the new decoder is more verbose, but the control flow is simpler. Conditionals are not nested and have small bodies. All instruction groups are resolved even before operands are decoded, and code generation is separated as much as possible within small functions that only handle one instruction each. - keep address generation and (for ALU operands) memory loads and writeback as much in common code as possible. All ALU operations for example are implemented as T0=f(T0,T1). For non-ALU instructions, read-modify-write memory operations are rare, but registers do not have TCGv equivalents: therefore, the common logic sets up pointer temporaries with the operands, while load and writeback are handled by gvec or by helpers. These principles make future code review and extensibility simpler, at the cost of having a relatively large amount of code in the form of this patch. Even EVEX should not be _too_ hard to implement (it's just a crazy large amount of possibilities). This patch introduces the main decoder flow, and integrates the old decoder with the new one. The old decoder takes care of parsing prefixes and then optionally drops to the new one. The changes to the old decoder are minimal and allow it to be replaced incrementally with the new one. There is a debugging mechanism through a "LIMIT" environment variable. In user-mode emulation, the variable is the number of instructions decoded by the new decoder before permanently switching to the old one. In system emulation, the variable is the highest opcode that is decoded by the new decoder (this is less friendly, but it's the best that can be done without requiring deterministic execution). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-23 12:20:55 +03:00
/* Collect prefixes. */
switch (b) {
case 0xf3:
s->prefix |= PREFIX_REPZ;
s->prefix &= ~PREFIX_REPNZ;
goto next_byte;
case 0xf2:
s->prefix |= PREFIX_REPNZ;
s->prefix &= ~PREFIX_REPZ;
goto next_byte;
case 0xf0:
s->prefix |= PREFIX_LOCK;
goto next_byte;
case 0x2e:
s->override = R_CS;
goto next_byte;
case 0x36:
s->override = R_SS;
goto next_byte;
case 0x3e:
s->override = R_DS;
goto next_byte;
case 0x26:
s->override = R_ES;
goto next_byte;
case 0x64:
s->override = R_FS;
goto next_byte;
case 0x65:
s->override = R_GS;
goto next_byte;
case 0x66:
s->prefix |= PREFIX_DATA;
goto next_byte;
case 0x67:
s->prefix |= PREFIX_ADR;
goto next_byte;
#ifdef TARGET_X86_64
case 0x40 ... 0x4f:
if (CODE64(s)) {
/* REX prefix */
s->prefix |= PREFIX_REX;
s->vex_w = (b >> 3) & 1;
s->rex_r = (b & 0x4) << 1;
s->rex_x = (b & 0x2) << 2;
s->rex_b = (b & 0x1) << 3;
goto next_byte;
}
break;
#endif
case 0xc5: /* 2-byte VEX */
case 0xc4: /* 3-byte VEX */
/*
* VEX prefixes cannot be used except in 32-bit mode.
* Otherwise the instruction is LES or LDS.
*/
if (CODE32(s) && !VM86(s)) {
static const int pp_prefix[4] = {
0, PREFIX_DATA, PREFIX_REPZ, PREFIX_REPNZ
};
int vex3, vex2 = x86_ldub_code(env, s);
if (!CODE64(s) && (vex2 & 0xc0) != 0xc0) {
/*
* 4.1.4.6: In 32-bit mode, bits [7:6] must be 11b,
* otherwise the instruction is LES or LDS.
*/
s->pc--; /* rewind the advance_pc() x86_ldub_code() did */
break;
}
/* 4.1.1-4.1.3: No preceding lock, 66, f2, f3, or rex prefixes. */
if (s->prefix & (PREFIX_REPZ | PREFIX_REPNZ
| PREFIX_LOCK | PREFIX_DATA | PREFIX_REX)) {
goto illegal_op;
}
#ifdef TARGET_X86_64
s->rex_r = (~vex2 >> 4) & 8;
#endif
if (b == 0xc5) {
/* 2-byte VEX prefix: RVVVVlpp, implied 0f leading opcode byte */
vex3 = vex2;
decode_func = decode_0F;
} else {
/* 3-byte VEX prefix: RXBmmmmm wVVVVlpp */
vex3 = x86_ldub_code(env, s);
#ifdef TARGET_X86_64
s->rex_x = (~vex2 >> 3) & 8;
s->rex_b = (~vex2 >> 2) & 8;
#endif
s->vex_w = (vex3 >> 7) & 1;
switch (vex2 & 0x1f) {
case 0x01: /* Implied 0f leading opcode bytes. */
decode_func = decode_0F;
break;
case 0x02: /* Implied 0f 38 leading opcode bytes. */
decode_func = decode_0F38;
break;
case 0x03: /* Implied 0f 3a leading opcode bytes. */
decode_func = decode_0F3A;
break;
default: /* Reserved for future use. */
goto unknown_op;
}
}
s->vex_v = (~vex3 >> 3) & 0xf;
s->vex_l = (vex3 >> 2) & 1;
s->prefix |= pp_prefix[vex3 & 3] | PREFIX_VEX;
}
break;
default:
break;
}
/* Post-process prefixes. */
if (CODE64(s)) {
/*
* In 64-bit mode, the default data size is 32-bit. Select 64-bit
* data with rex_w, and 16-bit data with 0x66; rex_w takes precedence
* over 0x66 if both are present.
*/
s->dflag = (REX_W(s) ? MO_64 : s->prefix & PREFIX_DATA ? MO_16 : MO_32);
/* In 64-bit mode, 0x67 selects 32-bit addressing. */
s->aflag = (s->prefix & PREFIX_ADR ? MO_32 : MO_64);
} else {
/* In 16/32-bit mode, 0x66 selects the opposite data size. */
if (CODE32(s) ^ ((s->prefix & PREFIX_DATA) != 0)) {
s->dflag = MO_32;
} else {
s->dflag = MO_16;
}
/* In 16/32-bit mode, 0x67 selects the opposite addressing. */
if (CODE32(s) ^ ((s->prefix & PREFIX_ADR) != 0)) {
s->aflag = MO_32;
} else {
s->aflag = MO_16;
}
}
target/i386: remove duplicate prefix decoding Now that a bulk of opcodes go through the new decoder, it is sensible to do some cleanup. Go immediately through disas_insn_new and only jump back after parsing the prefixes. disas_insn() now only contains the three sigsetjmp cases, and they are more easily managed if they are inlined into i386_tr_translate_insn. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-04-09 18:31:23 +03:00
/* Go back to old decoder for unconverted opcodes. */
if (!(s->prefix & PREFIX_VEX)) {
if ((b & ~7) == 0xd8) {
if (!disas_insn_x87(s, cpu, b)) {
goto unknown_op;
}
return;
}
if (b == 0x0f) {
b = x86_ldub_code(env, s);
switch (b) {
target/i386: convert non-grouped, helper-based 2-byte opcodes These have very simple generators and no need for complex group decoding. Apart from LAR/LSL which are simplified to use gen_op_deposit_reg_v and movcond, the code is generally lifted from translate.c into the generators. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-25 11:49:26 +03:00
case 0x00 ... 0x01: /* mostly privileged instructions */
target/i386: move prefetch and multi-byte UD/NOP to new decoder These are trivial to add, and moving them to the new decoder fixes some corner cases: raising #UD instead of an instruction fetch page fault for the undefined opcodes, and incorrectly rejecting 0F 18 prefetches with register operands (which are treated as reserved NOPs). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Zhao Liu <zhao1.liu@intel.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-06 18:34:55 +03:00
case 0x1a ... 0x1b: /* MPX */
target/i386: convert SHLD/SHRD to new decoder Use the same flag generation code as SHL and SHR, but use the existing gen_shiftd_rm_T1 function to compute the result as well as CC_SRC. Decoding-wise, SHLD/SHRD by immediate count as a 4 operand instruction because s->T0 and s->T1 actually occupy three op slots. The infrastructure used by opcodes in the 0F 3A table works fine. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-09 12:46:59 +03:00
case 0xa3: /* bt */
case 0xab: /* bts */
target/i386: remove duplicate prefix decoding Now that a bulk of opcodes go through the new decoder, it is sensible to do some cleanup. Go immediately through disas_insn_new and only jump back after parsing the prefixes. disas_insn() now only contains the three sigsetjmp cases, and they are more easily managed if they are inlined into i386_tr_translate_insn. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-04-09 18:31:23 +03:00
case 0xb3: /* btr */
target/i386: convert LZCNT/TZCNT/BSF/BSR/POPCNT to new decoder Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-09 16:11:41 +03:00
case 0xba ... 0xbb: /* grp8, btc */
target/i386: remove duplicate prefix decoding Now that a bulk of opcodes go through the new decoder, it is sensible to do some cleanup. Go immediately through disas_insn_new and only jump back after parsing the prefixes. disas_insn() now only contains the three sigsetjmp cases, and they are more easily managed if they are inlined into i386_tr_translate_insn. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-04-09 18:31:23 +03:00
case 0xc7: /* grp9 */
disas_insn_old(s, cpu, b + 0x100);
return;
default:
decode_func = do_decode_0F;
break;
}
}
}
target/i386: add core of new i386 decoder The new decoder is based on three principles: - use mostly table-driven decoding, using tables derived as much as possible from the Intel manual. Centralizing the decode the operands makes it more homogeneous, for example all immediates are signed. All modrm handling is in one function, and can be shared between SSE and ALU instructions (including XMM<->GPR instructions). The SSE/AVX decoder will also not have duplicated code between the 0F, 0F38 and 0F3A tables. - keep the code as "non-branchy" as possible. Generally, the code for the new decoder is more verbose, but the control flow is simpler. Conditionals are not nested and have small bodies. All instruction groups are resolved even before operands are decoded, and code generation is separated as much as possible within small functions that only handle one instruction each. - keep address generation and (for ALU operands) memory loads and writeback as much in common code as possible. All ALU operations for example are implemented as T0=f(T0,T1). For non-ALU instructions, read-modify-write memory operations are rare, but registers do not have TCGv equivalents: therefore, the common logic sets up pointer temporaries with the operands, while load and writeback are handled by gvec or by helpers. These principles make future code review and extensibility simpler, at the cost of having a relatively large amount of code in the form of this patch. Even EVEX should not be _too_ hard to implement (it's just a crazy large amount of possibilities). This patch introduces the main decoder flow, and integrates the old decoder with the new one. The old decoder takes care of parsing prefixes and then optionally drops to the new one. The changes to the old decoder are minimal and allow it to be replaced incrementally with the new one. There is a debugging mechanism through a "LIMIT" environment variable. In user-mode emulation, the variable is the number of instructions decoded by the new decoder before permanently switching to the old one. In system emulation, the variable is the highest opcode that is decoded by the new decoder (this is less friendly, but it's the best that can be done without requiring deterministic execution). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-23 12:20:55 +03:00
memset(&decode, 0, sizeof(decode));
target/i386: introduce flags writeback mechanism ALU instructions can write to both memory and flags. If the CC_SRC* and CC_DST locations have been written already when a memory access causes a fault, the value in CC_SRC* and CC_DST might be interpreted with the wrong CC_OP (the one that is in effect before the instruction. Besides just using the wrong result for the flags, something like subtracting -1 can have disastrous effects if the current CC_OP is CC_OP_EFLAGS: this is because QEMU does not expect bits outside the ALU flags to be set in CC_SRC, and env->eflags can end up set to all-ones. In the case of the attached testcase, this sets IOPL to 3 and would cause an assertion failure if SUB is moved to the new decoder. This mechanism is not really needed for BMI instructions, which can only write to a register, but put it to use anyway for cleanliness. In the case of BZHI, the code has to be modified slightly to ensure that decode->cc_src is written, otherwise the new assertions trigger. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-11 16:26:40 +03:00
decode.cc_op = -1;
target/i386: add core of new i386 decoder The new decoder is based on three principles: - use mostly table-driven decoding, using tables derived as much as possible from the Intel manual. Centralizing the decode the operands makes it more homogeneous, for example all immediates are signed. All modrm handling is in one function, and can be shared between SSE and ALU instructions (including XMM<->GPR instructions). The SSE/AVX decoder will also not have duplicated code between the 0F, 0F38 and 0F3A tables. - keep the code as "non-branchy" as possible. Generally, the code for the new decoder is more verbose, but the control flow is simpler. Conditionals are not nested and have small bodies. All instruction groups are resolved even before operands are decoded, and code generation is separated as much as possible within small functions that only handle one instruction each. - keep address generation and (for ALU operands) memory loads and writeback as much in common code as possible. All ALU operations for example are implemented as T0=f(T0,T1). For non-ALU instructions, read-modify-write memory operations are rare, but registers do not have TCGv equivalents: therefore, the common logic sets up pointer temporaries with the operands, while load and writeback are handled by gvec or by helpers. These principles make future code review and extensibility simpler, at the cost of having a relatively large amount of code in the form of this patch. Even EVEX should not be _too_ hard to implement (it's just a crazy large amount of possibilities). This patch introduces the main decoder flow, and integrates the old decoder with the new one. The old decoder takes care of parsing prefixes and then optionally drops to the new one. The changes to the old decoder are minimal and allow it to be replaced incrementally with the new one. There is a debugging mechanism through a "LIMIT" environment variable. In user-mode emulation, the variable is the number of instructions decoded by the new decoder before permanently switching to the old one. In system emulation, the variable is the highest opcode that is decoded by the new decoder (this is less friendly, but it's the best that can be done without requiring deterministic execution). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-23 12:20:55 +03:00
decode.b = b;
if (!decode_insn(s, env, decode_func, &decode)) {
goto illegal_op;
}
if (!decode.e.gen) {
goto unknown_op;
}
target/i386: add CPUID feature checks to new decoder Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-01 15:51:35 +03:00
if (!has_cpuid_feature(s, decode.e.cpuid)) {
goto illegal_op;
}
target/i386: group common checks in the decoding phase In preparation for adding more similar checks, move the VEX.L=0 check and several X86_SPECIAL_* checks to a new field, where each bit represent a common check on unused bits, or a restriction on the processor mode. Likewise, many SVM intercepts can be checked during the decoding phase, the main exception being the selective CR0 write, MSR and IOIO intercepts. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-09 18:43:12 +03:00
/* Checks that result in #UD come first. */
if (decode.e.check) {
target/i386: convert non-grouped, helper-based 2-byte opcodes These have very simple generators and no need for complex group decoding. Apart from LAR/LSL which are simplified to use gen_op_deposit_reg_v and movcond, the code is generally lifted from translate.c into the generators. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-25 11:49:26 +03:00
if (CODE64(s)) {
if (decode.e.check & X86_CHECK_i64) {
target/i386: group common checks in the decoding phase In preparation for adding more similar checks, move the VEX.L=0 check and several X86_SPECIAL_* checks to a new field, where each bit represent a common check on unused bits, or a restriction on the processor mode. Likewise, many SVM intercepts can be checked during the decoding phase, the main exception being the selective CR0 write, MSR and IOIO intercepts. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-09 18:43:12 +03:00
goto illegal_op;
}
target/i386: convert non-grouped, helper-based 2-byte opcodes These have very simple generators and no need for complex group decoding. Apart from LAR/LSL which are simplified to use gen_op_deposit_reg_v and movcond, the code is generally lifted from translate.c into the generators. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-25 11:49:26 +03:00
if ((decode.e.check & X86_CHECK_i64_amd) && env->cpuid_vendor1 != CPUID_VENDOR_INTEL_1) {
goto illegal_op;
}
} else {
if (decode.e.check & X86_CHECK_o64) {
goto illegal_op;
}
if ((decode.e.check & X86_CHECK_o64_intel) && env->cpuid_vendor1 == CPUID_VENDOR_INTEL_1) {
target/i386: group common checks in the decoding phase In preparation for adding more similar checks, move the VEX.L=0 check and several X86_SPECIAL_* checks to a new field, where each bit represent a common check on unused bits, or a restriction on the processor mode. Likewise, many SVM intercepts can be checked during the decoding phase, the main exception being the selective CR0 write, MSR and IOIO intercepts. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-09 18:43:12 +03:00
goto illegal_op;
}
}
target/i386: split X86_CHECK_prot into PE and VM86 checks SYSENTER is allowed in VM86 mode, but not in real mode. Split the check so that PE and !VM86 are covered by separate bits. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-09 10:52:30 +03:00
if (decode.e.check & X86_CHECK_prot_or_vm86) {
if (!PE(s)) {
goto illegal_op;
}
}
if (decode.e.check & X86_CHECK_no_vm86) {
if (VM86(s)) {
target/i386: group common checks in the decoding phase In preparation for adding more similar checks, move the VEX.L=0 check and several X86_SPECIAL_* checks to a new field, where each bit represent a common check on unused bits, or a restriction on the processor mode. Likewise, many SVM intercepts can be checked during the decoding phase, the main exception being the selective CR0 write, MSR and IOIO intercepts. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-09 18:43:12 +03:00
goto illegal_op;
}
}
}
target/i386: add core of new i386 decoder The new decoder is based on three principles: - use mostly table-driven decoding, using tables derived as much as possible from the Intel manual. Centralizing the decode the operands makes it more homogeneous, for example all immediates are signed. All modrm handling is in one function, and can be shared between SSE and ALU instructions (including XMM<->GPR instructions). The SSE/AVX decoder will also not have duplicated code between the 0F, 0F38 and 0F3A tables. - keep the code as "non-branchy" as possible. Generally, the code for the new decoder is more verbose, but the control flow is simpler. Conditionals are not nested and have small bodies. All instruction groups are resolved even before operands are decoded, and code generation is separated as much as possible within small functions that only handle one instruction each. - keep address generation and (for ALU operands) memory loads and writeback as much in common code as possible. All ALU operations for example are implemented as T0=f(T0,T1). For non-ALU instructions, read-modify-write memory operations are rare, but registers do not have TCGv equivalents: therefore, the common logic sets up pointer temporaries with the operands, while load and writeback are handled by gvec or by helpers. These principles make future code review and extensibility simpler, at the cost of having a relatively large amount of code in the form of this patch. Even EVEX should not be _too_ hard to implement (it's just a crazy large amount of possibilities). This patch introduces the main decoder flow, and integrates the old decoder with the new one. The old decoder takes care of parsing prefixes and then optionally drops to the new one. The changes to the old decoder are minimal and allow it to be replaced incrementally with the new one. There is a debugging mechanism through a "LIMIT" environment variable. In user-mode emulation, the variable is the number of instructions decoded by the new decoder before permanently switching to the old one. In system emulation, the variable is the highest opcode that is decoded by the new decoder (this is less friendly, but it's the best that can be done without requiring deterministic execution). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-23 12:20:55 +03:00
switch (decode.e.special) {
case X86_SPECIAL_None:
break;
case X86_SPECIAL_Locked:
if (decode.op[0].has_ea) {
s->prefix |= PREFIX_LOCK;
}
target/i386: reimplement check for validity of LOCK prefix The previous check erroneously allowed CMP to be modified with LOCK. Instead, tag explicitly the instructions that do support LOCK. Acked-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-19 15:22:53 +03:00
decode.e.special = X86_SPECIAL_HasLock;
/* fallthrough */
case X86_SPECIAL_HasLock:
target/i386: add core of new i386 decoder The new decoder is based on three principles: - use mostly table-driven decoding, using tables derived as much as possible from the Intel manual. Centralizing the decode the operands makes it more homogeneous, for example all immediates are signed. All modrm handling is in one function, and can be shared between SSE and ALU instructions (including XMM<->GPR instructions). The SSE/AVX decoder will also not have duplicated code between the 0F, 0F38 and 0F3A tables. - keep the code as "non-branchy" as possible. Generally, the code for the new decoder is more verbose, but the control flow is simpler. Conditionals are not nested and have small bodies. All instruction groups are resolved even before operands are decoded, and code generation is separated as much as possible within small functions that only handle one instruction each. - keep address generation and (for ALU operands) memory loads and writeback as much in common code as possible. All ALU operations for example are implemented as T0=f(T0,T1). For non-ALU instructions, read-modify-write memory operations are rare, but registers do not have TCGv equivalents: therefore, the common logic sets up pointer temporaries with the operands, while load and writeback are handled by gvec or by helpers. These principles make future code review and extensibility simpler, at the cost of having a relatively large amount of code in the form of this patch. Even EVEX should not be _too_ hard to implement (it's just a crazy large amount of possibilities). This patch introduces the main decoder flow, and integrates the old decoder with the new one. The old decoder takes care of parsing prefixes and then optionally drops to the new one. The changes to the old decoder are minimal and allow it to be replaced incrementally with the new one. There is a debugging mechanism through a "LIMIT" environment variable. In user-mode emulation, the variable is the number of instructions decoded by the new decoder before permanently switching to the old one. In system emulation, the variable is the highest opcode that is decoded by the new decoder (this is less friendly, but it's the best that can be done without requiring deterministic execution). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-23 12:20:55 +03:00
break;
target/i386: rename zext0/zext2 and make them closer to the manual X86_SPECIAL_ZExtOp0 and X86_SPECIAL_ZExtOp2 are poorly named; they are a hack that is needed by scalar insertion and extraction instructions, and not really related to zero extension: for PEXTR the zero extension is done by the generation functions, for PINSR the high bits are not used at all and in fact are *not* filled with zeroes when loaded into s->T1. Rename the values to match the effect described in the manual, and explain better in the comments. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-20 00:14:34 +03:00
case X86_SPECIAL_Op0_Rd:
target/i386: add core of new i386 decoder The new decoder is based on three principles: - use mostly table-driven decoding, using tables derived as much as possible from the Intel manual. Centralizing the decode the operands makes it more homogeneous, for example all immediates are signed. All modrm handling is in one function, and can be shared between SSE and ALU instructions (including XMM<->GPR instructions). The SSE/AVX decoder will also not have duplicated code between the 0F, 0F38 and 0F3A tables. - keep the code as "non-branchy" as possible. Generally, the code for the new decoder is more verbose, but the control flow is simpler. Conditionals are not nested and have small bodies. All instruction groups are resolved even before operands are decoded, and code generation is separated as much as possible within small functions that only handle one instruction each. - keep address generation and (for ALU operands) memory loads and writeback as much in common code as possible. All ALU operations for example are implemented as T0=f(T0,T1). For non-ALU instructions, read-modify-write memory operations are rare, but registers do not have TCGv equivalents: therefore, the common logic sets up pointer temporaries with the operands, while load and writeback are handled by gvec or by helpers. These principles make future code review and extensibility simpler, at the cost of having a relatively large amount of code in the form of this patch. Even EVEX should not be _too_ hard to implement (it's just a crazy large amount of possibilities). This patch introduces the main decoder flow, and integrates the old decoder with the new one. The old decoder takes care of parsing prefixes and then optionally drops to the new one. The changes to the old decoder are minimal and allow it to be replaced incrementally with the new one. There is a debugging mechanism through a "LIMIT" environment variable. In user-mode emulation, the variable is the number of instructions decoded by the new decoder before permanently switching to the old one. In system emulation, the variable is the highest opcode that is decoded by the new decoder (this is less friendly, but it's the best that can be done without requiring deterministic execution). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-23 12:20:55 +03:00
assert(decode.op[0].unit == X86_OP_INT);
if (!decode.op[0].has_ea) {
decode.op[0].ot = MO_32;
}
break;
target/i386: rename zext0/zext2 and make them closer to the manual X86_SPECIAL_ZExtOp0 and X86_SPECIAL_ZExtOp2 are poorly named; they are a hack that is needed by scalar insertion and extraction instructions, and not really related to zero extension: for PEXTR the zero extension is done by the generation functions, for PINSR the high bits are not used at all and in fact are *not* filled with zeroes when loaded into s->T1. Rename the values to match the effect described in the manual, and explain better in the comments. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-20 00:14:34 +03:00
case X86_SPECIAL_Op2_Ry:
target/i386: add core of new i386 decoder The new decoder is based on three principles: - use mostly table-driven decoding, using tables derived as much as possible from the Intel manual. Centralizing the decode the operands makes it more homogeneous, for example all immediates are signed. All modrm handling is in one function, and can be shared between SSE and ALU instructions (including XMM<->GPR instructions). The SSE/AVX decoder will also not have duplicated code between the 0F, 0F38 and 0F3A tables. - keep the code as "non-branchy" as possible. Generally, the code for the new decoder is more verbose, but the control flow is simpler. Conditionals are not nested and have small bodies. All instruction groups are resolved even before operands are decoded, and code generation is separated as much as possible within small functions that only handle one instruction each. - keep address generation and (for ALU operands) memory loads and writeback as much in common code as possible. All ALU operations for example are implemented as T0=f(T0,T1). For non-ALU instructions, read-modify-write memory operations are rare, but registers do not have TCGv equivalents: therefore, the common logic sets up pointer temporaries with the operands, while load and writeback are handled by gvec or by helpers. These principles make future code review and extensibility simpler, at the cost of having a relatively large amount of code in the form of this patch. Even EVEX should not be _too_ hard to implement (it's just a crazy large amount of possibilities). This patch introduces the main decoder flow, and integrates the old decoder with the new one. The old decoder takes care of parsing prefixes and then optionally drops to the new one. The changes to the old decoder are minimal and allow it to be replaced incrementally with the new one. There is a debugging mechanism through a "LIMIT" environment variable. In user-mode emulation, the variable is the number of instructions decoded by the new decoder before permanently switching to the old one. In system emulation, the variable is the highest opcode that is decoded by the new decoder (this is less friendly, but it's the best that can be done without requiring deterministic execution). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-23 12:20:55 +03:00
assert(decode.op[2].unit == X86_OP_INT);
if (!decode.op[2].has_ea) {
target/i386: rename zext0/zext2 and make them closer to the manual X86_SPECIAL_ZExtOp0 and X86_SPECIAL_ZExtOp2 are poorly named; they are a hack that is needed by scalar insertion and extraction instructions, and not really related to zero extension: for PEXTR the zero extension is done by the generation functions, for PINSR the high bits are not used at all and in fact are *not* filled with zeroes when loaded into s->T1. Rename the values to match the effect described in the manual, and explain better in the comments. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-20 00:14:34 +03:00
decode.op[2].ot = s->dflag == MO_16 ? MO_32 : s->dflag;
target/i386: add core of new i386 decoder The new decoder is based on three principles: - use mostly table-driven decoding, using tables derived as much as possible from the Intel manual. Centralizing the decode the operands makes it more homogeneous, for example all immediates are signed. All modrm handling is in one function, and can be shared between SSE and ALU instructions (including XMM<->GPR instructions). The SSE/AVX decoder will also not have duplicated code between the 0F, 0F38 and 0F3A tables. - keep the code as "non-branchy" as possible. Generally, the code for the new decoder is more verbose, but the control flow is simpler. Conditionals are not nested and have small bodies. All instruction groups are resolved even before operands are decoded, and code generation is separated as much as possible within small functions that only handle one instruction each. - keep address generation and (for ALU operands) memory loads and writeback as much in common code as possible. All ALU operations for example are implemented as T0=f(T0,T1). For non-ALU instructions, read-modify-write memory operations are rare, but registers do not have TCGv equivalents: therefore, the common logic sets up pointer temporaries with the operands, while load and writeback are handled by gvec or by helpers. These principles make future code review and extensibility simpler, at the cost of having a relatively large amount of code in the form of this patch. Even EVEX should not be _too_ hard to implement (it's just a crazy large amount of possibilities). This patch introduces the main decoder flow, and integrates the old decoder with the new one. The old decoder takes care of parsing prefixes and then optionally drops to the new one. The changes to the old decoder are minimal and allow it to be replaced incrementally with the new one. There is a debugging mechanism through a "LIMIT" environment variable. In user-mode emulation, the variable is the number of instructions decoded by the new decoder before permanently switching to the old one. In system emulation, the variable is the highest opcode that is decoded by the new decoder (this is less friendly, but it's the best that can be done without requiring deterministic execution). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-23 12:20:55 +03:00
}
break;
target/i386: reimplement 0x0f 0x38, add AVX There are several special cases here: 1) extending moves have different widths for the helpers vs. for the memory loads, and the width for memory loads depends on VEX.L too. This is represented by X86_SPECIAL_AVXExtMov. 2) some instructions, such as variable-width shifts, select the vector element size via REX.W. 3) VSIB instructions (VGATHERxPy, VPGATHERxy) are also part of this group, and they have (among other things) two output operands. 3) the macros for 4-operand blends (which are under 0x0f 0x3a) have to be extended to support 2-operand blends. The 2-operand variant actually came a few years earlier, but it is clearer to implement them in the opposite order. X86_TYPE_WM, introduced earlier for unaligned loads, is reused for helpers that accept a Reg* but have a M argument. These three-byte opcodes also include AVX new instructions, for which the helpers were originally implemented by Paul Brook <paul@nowt.org>. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-14 19:52:44 +03:00
case X86_SPECIAL_AVXExtMov:
if (!decode.op[2].has_ea) {
decode.op[2].ot = s->vex_l ? MO_256 : MO_128;
} else if (s->vex_l) {
decode.op[2].ot++;
}
break;
target/i386: add X86_SPECIALs for MOVSX and MOVZX Usually the registers are just moved into s->T0 without much care for their operand size. However, in some cases we can get more efficient code if the operand fetching logic syncs with the emission function on what is nicer. All the current uses are mostly demonstrative and only reduce the code in the emission functions, because the instructions do not support memory operands. However the logic is generic and applies to several more instructions such as MOVSXD (aka movslq), one-byte shift instructions, multiplications, XLAT, and indirect calls/jumps. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-19 16:40:54 +03:00
case X86_SPECIAL_SExtT0:
case X86_SPECIAL_ZExtT0:
/* Handled in gen_load. */
assert(decode.op[1].unit == X86_OP_INT);
break;
target/i386: fix memory opsize for Mov to/from Seg This commit fixes an issue with MOV instructions (0x8C and 0x8E) involving segment registers; MOV to segment register's source is 16-bit, while MOV from segment register has to explicitly set the memory operand size to 16 bits. Introduce a new flag X86_SPECIAL_Op0_Mw to handle this specification correctly. Signed-off-by: Xinyu Li <lixinyu20s@ict.ac.cn> Message-ID: <20240602100528.2135717-1-lixinyu20s@ict.ac.cn> Fixes: 5e9e21bcc4d ("target/i386: move 60-BF opcodes to new decoder", 2024-05-07) Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-06-02 13:05:28 +03:00
case X86_SPECIAL_Op0_Mw:
assert(decode.op[0].unit == X86_OP_INT);
if (decode.op[0].has_ea) {
decode.op[0].ot = MO_16;
}
break;
target/i386: Check CR0.TS before enter_mmx When CR0.TS=1, execution of x87 FPU, MMX, and some SSE instructions will cause a Device Not Available (DNA) exception (#NM). System software uses this exception event to lazily context switch FPU state. Before this patch, enter_mmx helpers may be generated just before #NM generation, prematurely resetting FPU state before the guest has a chance to save it. Signed-off-by: Matt Borgerson <contact@mborgerson.com> Message-ID: <CADc=-s5F10muEhLs4f3mxqsEPAHWj0XFfOC2sfFMVHrk9fcpMg@mail.gmail.com> Cc: qemu-stable@nongnu.org Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-07-13 10:29:01 +03:00
default:
target/i386: add core of new i386 decoder The new decoder is based on three principles: - use mostly table-driven decoding, using tables derived as much as possible from the Intel manual. Centralizing the decode the operands makes it more homogeneous, for example all immediates are signed. All modrm handling is in one function, and can be shared between SSE and ALU instructions (including XMM<->GPR instructions). The SSE/AVX decoder will also not have duplicated code between the 0F, 0F38 and 0F3A tables. - keep the code as "non-branchy" as possible. Generally, the code for the new decoder is more verbose, but the control flow is simpler. Conditionals are not nested and have small bodies. All instruction groups are resolved even before operands are decoded, and code generation is separated as much as possible within small functions that only handle one instruction each. - keep address generation and (for ALU operands) memory loads and writeback as much in common code as possible. All ALU operations for example are implemented as T0=f(T0,T1). For non-ALU instructions, read-modify-write memory operations are rare, but registers do not have TCGv equivalents: therefore, the common logic sets up pointer temporaries with the operands, while load and writeback are handled by gvec or by helpers. These principles make future code review and extensibility simpler, at the cost of having a relatively large amount of code in the form of this patch. Even EVEX should not be _too_ hard to implement (it's just a crazy large amount of possibilities). This patch introduces the main decoder flow, and integrates the old decoder with the new one. The old decoder takes care of parsing prefixes and then optionally drops to the new one. The changes to the old decoder are minimal and allow it to be replaced incrementally with the new one. There is a debugging mechanism through a "LIMIT" environment variable. In user-mode emulation, the variable is the number of instructions decoded by the new decoder before permanently switching to the old one. In system emulation, the variable is the highest opcode that is decoded by the new decoder (this is less friendly, but it's the best that can be done without requiring deterministic execution). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-23 12:20:55 +03:00
break;
}
target/i386: reimplement check for validity of LOCK prefix The previous check erroneously allowed CMP to be modified with LOCK. Instead, tag explicitly the instructions that do support LOCK. Acked-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-19 15:22:53 +03:00
if (s->prefix & PREFIX_LOCK) {
if (decode.e.special != X86_SPECIAL_HasLock || !decode.op[0].has_ea) {
goto illegal_op;
}
}
target/i386: validate VEX prefixes via the instructions' exception classes Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-18 01:43:52 +03:00
if (!validate_vex(s, &decode)) {
return;
}
target/i386: group common checks in the decoding phase In preparation for adding more similar checks, move the VEX.L=0 check and several X86_SPECIAL_* checks to a new field, where each bit represent a common check on unused bits, or a restriction on the processor mode. Likewise, many SVM intercepts can be checked during the decoding phase, the main exception being the selective CR0 write, MSR and IOIO intercepts. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-09 18:43:12 +03:00
/*
* Checks that result in #GP or VMEXIT come second. Intercepts are
target/i386: cleanup HLT helpers Use decode.c's support for intercepts, doing the check in TCG-generated code rather than the helper. This is cleaner because it allows removing the eip_addend argument to helper_hlt(). Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-29 16:12:22 +03:00
* generally checked after non-memory exceptions (i.e. after all
target/i386: group common checks in the decoding phase In preparation for adding more similar checks, move the VEX.L=0 check and several X86_SPECIAL_* checks to a new field, where each bit represent a common check on unused bits, or a restriction on the processor mode. Likewise, many SVM intercepts can be checked during the decoding phase, the main exception being the selective CR0 write, MSR and IOIO intercepts. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-09 18:43:12 +03:00
* exceptions if there is no memory operand). Exceptions are
* vm86 checks (INTn, IRET, PUSHF/POPF), RSM and XSETBV (!).
*
target/i386: convert non-grouped, helper-based 2-byte opcodes These have very simple generators and no need for complex group decoding. Apart from LAR/LSL which are simplified to use gen_op_deposit_reg_v and movcond, the code is generally lifted from translate.c into the generators. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-25 11:49:26 +03:00
* XSETBV will check for CPL0 in the gen_* function instead of using chk().
target/i386: group common checks in the decoding phase In preparation for adding more similar checks, move the VEX.L=0 check and several X86_SPECIAL_* checks to a new field, where each bit represent a common check on unused bits, or a restriction on the processor mode. Likewise, many SVM intercepts can be checked during the decoding phase, the main exception being the selective CR0 write, MSR and IOIO intercepts. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-09 18:43:12 +03:00
*/
if (decode.e.check & X86_CHECK_cpl0) {
if (CPL(s) != 0) {
goto gp_fault;
}
}
target/i386: fix processing of intercept 0 (read CR0) Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-09 15:34:24 +03:00
if (decode.e.has_intercept && unlikely(GUEST(s))) {
target/i386: group common checks in the decoding phase In preparation for adding more similar checks, move the VEX.L=0 check and several X86_SPECIAL_* checks to a new field, where each bit represent a common check on unused bits, or a restriction on the processor mode. Likewise, many SVM intercepts can be checked during the decoding phase, the main exception being the selective CR0 write, MSR and IOIO intercepts. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-09 18:43:12 +03:00
gen_helper_svm_check_intercept(tcg_env,
tcg_constant_i32(decode.e.intercept));
}
if (decode.e.check) {
target/i386: convert non-grouped, helper-based 2-byte opcodes These have very simple generators and no need for complex group decoding. Apart from LAR/LSL which are simplified to use gen_op_deposit_reg_v and movcond, the code is generally lifted from translate.c into the generators. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-25 11:49:26 +03:00
if ((decode.e.check & X86_CHECK_smm) && !(s->flags & HF_SMM_MASK)) {
goto illegal_op;
}
target/i386: group common checks in the decoding phase In preparation for adding more similar checks, move the VEX.L=0 check and several X86_SPECIAL_* checks to a new field, where each bit represent a common check on unused bits, or a restriction on the processor mode. Likewise, many SVM intercepts can be checked during the decoding phase, the main exception being the selective CR0 write, MSR and IOIO intercepts. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-09 18:43:12 +03:00
if ((decode.e.check & X86_CHECK_vm86_iopl) && VM86(s)) {
if (IOPL(s) < 3) {
goto gp_fault;
}
} else if (decode.e.check & X86_CHECK_cpl_iopl) {
if (IOPL(s) < CPL(s)) {
goto gp_fault;
}
}
}
target/i386: Check CR0.TS before enter_mmx When CR0.TS=1, execution of x87 FPU, MMX, and some SSE instructions will cause a Device Not Available (DNA) exception (#NM). System software uses this exception event to lazily context switch FPU state. Before this patch, enter_mmx helpers may be generated just before #NM generation, prematurely resetting FPU state before the guest has a chance to save it. Signed-off-by: Matt Borgerson <contact@mborgerson.com> Message-ID: <CADc=-s5F10muEhLs4f3mxqsEPAHWj0XFfOC2sfFMVHrk9fcpMg@mail.gmail.com> Cc: qemu-stable@nongnu.org Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-07-13 10:29:01 +03:00
if (decode.e.special == X86_SPECIAL_MMX &&
!(s->prefix & (PREFIX_REPZ | PREFIX_REPNZ | PREFIX_DATA))) {
tcg: Rename cpu_env to tcg_env Allow the name 'cpu_env' to be used for something else. Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
2023-09-14 02:37:36 +03:00
gen_helper_enter_mmx(tcg_env);
target/i386: Check CR0.TS before enter_mmx When CR0.TS=1, execution of x87 FPU, MMX, and some SSE instructions will cause a Device Not Available (DNA) exception (#NM). System software uses this exception event to lazily context switch FPU state. Before this patch, enter_mmx helpers may be generated just before #NM generation, prematurely resetting FPU state before the guest has a chance to save it. Signed-off-by: Matt Borgerson <contact@mborgerson.com> Message-ID: <CADc=-s5F10muEhLs4f3mxqsEPAHWj0XFfOC2sfFMVHrk9fcpMg@mail.gmail.com> Cc: qemu-stable@nongnu.org Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-07-13 10:29:01 +03:00
}
target/i386: replace NoSeg special with NoLoadEA This is a bit more generic, as it can be applied to MPX as well. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-09 17:56:26 +03:00
if (decode.e.special != X86_SPECIAL_NoLoadEA &&
(decode.op[0].has_ea || decode.op[1].has_ea || decode.op[2].has_ea)) {
target/i386: validate VEX prefixes via the instructions' exception classes Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-09-18 01:43:52 +03:00
gen_load_ea(s, &decode.mem, decode.e.vex_class == 12);
target/i386: add core of new i386 decoder The new decoder is based on three principles: - use mostly table-driven decoding, using tables derived as much as possible from the Intel manual. Centralizing the decode the operands makes it more homogeneous, for example all immediates are signed. All modrm handling is in one function, and can be shared between SSE and ALU instructions (including XMM<->GPR instructions). The SSE/AVX decoder will also not have duplicated code between the 0F, 0F38 and 0F3A tables. - keep the code as "non-branchy" as possible. Generally, the code for the new decoder is more verbose, but the control flow is simpler. Conditionals are not nested and have small bodies. All instruction groups are resolved even before operands are decoded, and code generation is separated as much as possible within small functions that only handle one instruction each. - keep address generation and (for ALU operands) memory loads and writeback as much in common code as possible. All ALU operations for example are implemented as T0=f(T0,T1). For non-ALU instructions, read-modify-write memory operations are rare, but registers do not have TCGv equivalents: therefore, the common logic sets up pointer temporaries with the operands, while load and writeback are handled by gvec or by helpers. These principles make future code review and extensibility simpler, at the cost of having a relatively large amount of code in the form of this patch. Even EVEX should not be _too_ hard to implement (it's just a crazy large amount of possibilities). This patch introduces the main decoder flow, and integrates the old decoder with the new one. The old decoder takes care of parsing prefixes and then optionally drops to the new one. The changes to the old decoder are minimal and allow it to be replaced incrementally with the new one. There is a debugging mechanism through a "LIMIT" environment variable. In user-mode emulation, the variable is the number of instructions decoded by the new decoder before permanently switching to the old one. In system emulation, the variable is the highest opcode that is decoded by the new decoder (this is less friendly, but it's the best that can be done without requiring deterministic execution). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-23 12:20:55 +03:00
}
target/i386: add ALU load/writeback core Add generic code generation that takes care of preparing operands around calls to decode.e.gen in a table-driven manner, so that ALU operations need not take care of that. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-23 15:55:56 +03:00
if (s->prefix & PREFIX_LOCK) {
gen_load(s, &decode, 2, s->T1);
target/i386: remove CPUX86State argument from generator functions CPUX86State argument would only be used to fetch bytes, but that has to be done before the generator function is called. So remove it, and all temptation together with it. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-09 17:04:46 +03:00
decode.e.gen(s, &decode);
target/i386: add ALU load/writeback core Add generic code generation that takes care of preparing operands around calls to decode.e.gen in a table-driven manner, so that ALU operations need not take care of that. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-23 15:55:56 +03:00
} else {
if (decode.op[0].unit == X86_OP_MMX) {
compute_mmx_offset(&decode.op[0]);
} else if (decode.op[0].unit == X86_OP_SSE) {
compute_xmm_offset(&decode.op[0]);
}
gen_load(s, &decode, 1, s->T0);
gen_load(s, &decode, 2, s->T1);
target/i386: remove CPUX86State argument from generator functions CPUX86State argument would only be used to fetch bytes, but that has to be done before the generator function is called. So remove it, and all temptation together with it. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-09 17:04:46 +03:00
decode.e.gen(s, &decode);
target/i386: add ALU load/writeback core Add generic code generation that takes care of preparing operands around calls to decode.e.gen in a table-driven manner, so that ALU operations need not take care of that. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-23 15:55:56 +03:00
gen_writeback(s, &decode, 0, s->T0);
}
target/i386: introduce flags writeback mechanism ALU instructions can write to both memory and flags. If the CC_SRC* and CC_DST locations have been written already when a memory access causes a fault, the value in CC_SRC* and CC_DST might be interpreted with the wrong CC_OP (the one that is in effect before the instruction. Besides just using the wrong result for the flags, something like subtracting -1 can have disastrous effects if the current CC_OP is CC_OP_EFLAGS: this is because QEMU does not expect bits outside the ALU flags to be set in CC_SRC, and env->eflags can end up set to all-ones. In the case of the attached testcase, this sets IOPL to 3 and would cause an assertion failure if SUB is moved to the new decoder. This mechanism is not really needed for BMI instructions, which can only write to a register, but put it to use anyway for cleanliness. In the case of BZHI, the code has to be modified slightly to ensure that decode->cc_src is written, otherwise the new assertions trigger. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-11 16:26:40 +03:00
/*
target/i386: put BLS* input in T1, use generic flag writeback This makes for easier cpu_cc_* setup, and not using set_cc_op() should come in handy if QEMU ever implements APX. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2024-05-23 10:33:22 +03:00
* Write back flags after last memory access. Some older ALU instructions, as
target/i386: introduce flags writeback mechanism ALU instructions can write to both memory and flags. If the CC_SRC* and CC_DST locations have been written already when a memory access causes a fault, the value in CC_SRC* and CC_DST might be interpreted with the wrong CC_OP (the one that is in effect before the instruction. Besides just using the wrong result for the flags, something like subtracting -1 can have disastrous effects if the current CC_OP is CC_OP_EFLAGS: this is because QEMU does not expect bits outside the ALU flags to be set in CC_SRC, and env->eflags can end up set to all-ones. In the case of the attached testcase, this sets IOPL to 3 and would cause an assertion failure if SUB is moved to the new decoder. This mechanism is not really needed for BMI instructions, which can only write to a register, but put it to use anyway for cleanliness. In the case of BZHI, the code has to be modified slightly to ensure that decode->cc_src is written, otherwise the new assertions trigger. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-11 16:26:40 +03:00
* well as SSE instructions, write flags in the gen_* function, but that can
* cause incorrect tracking of CC_OP for instructions that write to both memory
* and flags.
*/
if (decode.cc_op != -1) {
if (decode.cc_dst) {
tcg_gen_mov_tl(cpu_cc_dst, decode.cc_dst);
}
if (decode.cc_src) {
tcg_gen_mov_tl(cpu_cc_src, decode.cc_src);
}
if (decode.cc_src2) {
tcg_gen_mov_tl(cpu_cc_src2, decode.cc_src2);
}
if (decode.cc_op == CC_OP_DYNAMIC) {
tcg_gen_mov_i32(cpu_cc_op, decode.cc_op_dynamic);
}
set_cc_op(s, decode.cc_op);
cc_live = cc_op_live[decode.cc_op];
} else {
cc_live = 0;
}
if (decode.cc_op != CC_OP_DYNAMIC) {
assert(!decode.cc_op_dynamic);
assert(!!decode.cc_dst == !!(cc_live & USES_CC_DST));
assert(!!decode.cc_src == !!(cc_live & USES_CC_SRC));
assert(!!decode.cc_src2 == !!(cc_live & USES_CC_SRC2));
}
target/i386: add core of new i386 decoder The new decoder is based on three principles: - use mostly table-driven decoding, using tables derived as much as possible from the Intel manual. Centralizing the decode the operands makes it more homogeneous, for example all immediates are signed. All modrm handling is in one function, and can be shared between SSE and ALU instructions (including XMM<->GPR instructions). The SSE/AVX decoder will also not have duplicated code between the 0F, 0F38 and 0F3A tables. - keep the code as "non-branchy" as possible. Generally, the code for the new decoder is more verbose, but the control flow is simpler. Conditionals are not nested and have small bodies. All instruction groups are resolved even before operands are decoded, and code generation is separated as much as possible within small functions that only handle one instruction each. - keep address generation and (for ALU operands) memory loads and writeback as much in common code as possible. All ALU operations for example are implemented as T0=f(T0,T1). For non-ALU instructions, read-modify-write memory operations are rare, but registers do not have TCGv equivalents: therefore, the common logic sets up pointer temporaries with the operands, while load and writeback are handled by gvec or by helpers. These principles make future code review and extensibility simpler, at the cost of having a relatively large amount of code in the form of this patch. Even EVEX should not be _too_ hard to implement (it's just a crazy large amount of possibilities). This patch introduces the main decoder flow, and integrates the old decoder with the new one. The old decoder takes care of parsing prefixes and then optionally drops to the new one. The changes to the old decoder are minimal and allow it to be replaced incrementally with the new one. There is a debugging mechanism through a "LIMIT" environment variable. In user-mode emulation, the variable is the number of instructions decoded by the new decoder before permanently switching to the old one. In system emulation, the variable is the highest opcode that is decoded by the new decoder (this is less friendly, but it's the best that can be done without requiring deterministic execution). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-23 12:20:55 +03:00
return;
target/i386: group common checks in the decoding phase In preparation for adding more similar checks, move the VEX.L=0 check and several X86_SPECIAL_* checks to a new field, where each bit represent a common check on unused bits, or a restriction on the processor mode. Likewise, many SVM intercepts can be checked during the decoding phase, the main exception being the selective CR0 write, MSR and IOIO intercepts. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2023-10-09 18:43:12 +03:00
gp_fault:
gen_exception_gpf(s);
return;
target/i386: add core of new i386 decoder The new decoder is based on three principles: - use mostly table-driven decoding, using tables derived as much as possible from the Intel manual. Centralizing the decode the operands makes it more homogeneous, for example all immediates are signed. All modrm handling is in one function, and can be shared between SSE and ALU instructions (including XMM<->GPR instructions). The SSE/AVX decoder will also not have duplicated code between the 0F, 0F38 and 0F3A tables. - keep the code as "non-branchy" as possible. Generally, the code for the new decoder is more verbose, but the control flow is simpler. Conditionals are not nested and have small bodies. All instruction groups are resolved even before operands are decoded, and code generation is separated as much as possible within small functions that only handle one instruction each. - keep address generation and (for ALU operands) memory loads and writeback as much in common code as possible. All ALU operations for example are implemented as T0=f(T0,T1). For non-ALU instructions, read-modify-write memory operations are rare, but registers do not have TCGv equivalents: therefore, the common logic sets up pointer temporaries with the operands, while load and writeback are handled by gvec or by helpers. These principles make future code review and extensibility simpler, at the cost of having a relatively large amount of code in the form of this patch. Even EVEX should not be _too_ hard to implement (it's just a crazy large amount of possibilities). This patch introduces the main decoder flow, and integrates the old decoder with the new one. The old decoder takes care of parsing prefixes and then optionally drops to the new one. The changes to the old decoder are minimal and allow it to be replaced incrementally with the new one. There is a debugging mechanism through a "LIMIT" environment variable. In user-mode emulation, the variable is the number of instructions decoded by the new decoder before permanently switching to the old one. In system emulation, the variable is the highest opcode that is decoded by the new decoder (this is less friendly, but it's the best that can be done without requiring deterministic execution). Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2022-08-23 12:20:55 +03:00
illegal_op:
gen_illegal_opcode(s);
return;
unknown_op:
gen_unknown_opcode(env, s);
}
Reference in New Issue Copy Permalink