x86: Fix RETBleed Vulnerabilities
Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=2090231
MR: !1127 (merged)
CVE: CVE-2022-23816
CVE: CVE-2022-23825
CVE: CVE-2022-29900
CVE: CVE-2022-29901
Depends: https://bugzilla.redhat.com/show_bug.cgi?id=2090254
Omitted-fix: fe83f5eae432 ("Fix SETcc emulation function offsets with SLS")
This patch had already been merged as part of MR #762 to
rebase KVM x86 to upstream v5.18.
Omitted-fix: aa8e73eed7d3 ("crypto: x86/sm3 - Fixup SLS")
Centos-stream-9 does have commit 930ab34d906d ("crypto:
x86/sm3 - add AVX assembly implementation") and so this fix
isn't needed.
Omitted-fix: 291073a566b2 ("kvm: fix objtool relocation warning")
This patch has been merged as RHEL commit 4d5cccc1
("kvm: fix objtool relocation warning").
Omitted-fix: 262448f3d189 ("x86/Kconfig: Only enable CONFIG_CC_HAS_IBT for clang >= 14.0.0")
RHEL doesn't use clang. So this patch is not relevant.
Omitted-fix: 35cb8c713a49 ("tools arch: Update arch/x86/lib/mem{cpy,set}_64.S copies used in 'perf bench mem memcpy'")
I will leave this for the perf team to backport, if needed.
This is a backport of the upstream patches that fix the RETBleed (Arbitrary Speculative Code Execution with Return Instructions) vulnerability. RETBleed is a speculative execution attack on x86-64 processors, including some recent Intel and AMD chips. It is a varient of the Spectre vulnerability which exploits retpoline, which was intended as a mitigation for speculative execution attacks. The affect processors include Intel Skylake generation processors as AMD Zen1/Zen2 and Bulldozer families.
For Intel Skylake generation processors, the mitigation is to enable IBRS on those processors.
For AMD processors, untrained return thunks have to be used. To be more effective, either SMT or STIBP has to be enabled. For the paranoid, there is an option to issue IBPB on kernel entry and exit with the most performance impact.
The upstream RETBleed patch series starts from patch 79. The first 78 patches are some additional patches that ease backporting including SLS (Straight Line Speculation) commits that are needed for fixing CVE-2021-26341.
Even though the current compiler can support SLS, it is not turned on for now as the return thunk enabled by this MR will mitigate SLS for the return instruction.
This MR also includes a partial set of X86_KERNEL_IBT patches to reduce merge conflicts and make it easier to backport the remaining patches in the series in the future if we need to. Therefore, we can't enable CONFIG_X86_KERNEL_IBT at the moment until the full set is merged.
There are also 3 redhat/configs patches right after the commits that introduce new kernel configs as "make dist-configs" will fail without the right config setting under redhat/configs.
There are 32 patches that have some kind of merge conflicts that require manual merging or context diff that can be handled by the patch command.
Signed-off-by: Waiman Long longman@redhat.com
Waiman Long (144):
xen: assume XENFEAT_mmu_pt_update_preserve_ad being set for pv guests
x86/build: Remove stale cc-option checks
Makefile: remove stale cc-option checks
kbuild: Fix TRIM_UNUSED_KSYMS with LTO_CLANG
kbuild: remove unused quiet_cmd_update_lto_symversions
kbuild: remove stale *.symversions
kbuild: clean up objtool_args slightly
objtool: Introduce CFI hash
objtool: Handle __sanitize_cov*() tail calls
x86/xen: Rework the xen_{cpu,irq,mmu}_opsarrays
objtool: Support pv_opsindirect calls for noinstr
x86/mce: Drop copyin special case for #MC
kbuild: move objtool_args back to scripts/Makefile.build
kbuild: rename __objtool_obj and reuse it for cmd_cc_lto_link_modules
kbuild: store the objtool command in *.cmd files
kbuild: factor out OBJECT_FILES_NON_STANDARD check into a macro
kbuild: detect objtool update without using .SECONDEXPANSION
kbuild: reuse $(cmd_objtool) for cmd_cc_lto_link_modules
objtool: Teach get_alt_entry() about more relocation types
objtool: print out the symbol type when complaining about it
objtool: Remove reloc symbol type checks in get_alt_entry()
objtool: Classify symbols
objtool: Explicitly avoid self modifying code in .altinstr_replacement
objtool: Shrink struct instruction
objtool,x86: Replace alternatives with .retpoline_sites
x86/retpoline: Remove unused replacement symbols
x86/asm: Fix register order
x86/asm: Fixup odd GEN-for-each-reg.h usage
x86/retpoline: Move the retpoline thunk declarations to
nospec-branch.h
x86/retpoline: Create a retpoline thunk array
x86/alternative: Implement .retpoline_sites support
x86/alternative: Handle Jcc __x86_indirect_thunk_\reg
x86/alternative: Try inline spectre_v2=retpoline,amd
x86/alternative: Add debug prints to apply_retpolines()
bpf,x86: Respect X86_FEATURE_RETPOLINE*
x86/xen: switch initial pvops IRQ functions to dummy ones
static_call,x86: Robustify trampoline patching
objtool: Fix pv_ops noinstr validation
x86/entry: Add a fence for kernel entry SWAPGS in paranoid_entry()
x86/entry: Use the correct fence macro after swapgs in kernel CR3
x86/xen: Add xenpv_restore_regs_and_return_to_usermode()
x86: Move RETPOLINE*_CFLAGS to arch Makefile
x86: Use -mindirect-branch-cs-prefix for RETPOLINE builds
x86/lib/atomic64_386_32: Rename things
x86/mce: Reduce number of machine checks taken during recovery
x86: Prepare asm files for straight-line-speculation
x86: Prepare inline-asm for straight-line-speculation
objtool: Add straight-line-speculation validation
x86/alternative: Relax text_poke_bp() constraint
x86: Add straight-line-speculation mitigation
redhat/configs: Disable CONFIG_SLS
objtool: Fix truncated string warning
objtool,efi: Update __efi64_thunk annotation
x86/ibt: Base IBT bits
redhat/configs: Disable CONFIG_X86_KERNEL_IBT
x86/ibt: Add ANNOTATE_NOENDBR
x86/entry: Cleanup PARAVIRT
x86/ibt,xen: Sprinkle the ENDBR
x86/ibt,entry: Sprinkle ENDBR dust
x86/ibt,kvm: Add ENDBR to fastops
x86/alternative: Simplify int3_selftest_ip
x86/ibt: Annotate text references
kvm/emulate: Fix SETcc emulation for ENDBR
crypto: x86/poly1305 - Fixup SLS
x86,static_call: Fix __static_call_return0 for i386
objtool: Fix SLS validation for kcov tail-call replacement
x86/static_call: Add ANNOTATE_NOENDBR to static call trampoline
x86/retpoline: Add ANNOTATE_NOENDBR for retpolines
x86/traps: Use pt_regs directly in fixup_bad_iret()
x86/entry: Switch the stack after error_entry() returns
x86/entry: Move PUSH_AND_CLEAR_REGS out of error_entry()
x86/entry: Move CLD to the start of the idtentry macro
x86/entry: Don't call error_entry() for XENPV
x86/entry: Simplify entry_INT80_compat()
x86/entry: Use PUSH_AND_CLEAR_REGS for compat
x86/entry: Remove skip_r11rcx
x86/entry: Fix register corruption in compat syscall
x86/kvm/vmx: Make noinstr clean
x86/cpufeatures: Move RETPOLINE flags to word 11
x86/retpoline: Cleanup some #ifdefery
x86/retpoline: Swizzle retpoline thunk
x86/retpoline: Use -mfunction-return
x86: Undo return-thunk damage
x86,objtool: Create .return_sites
x86,static_call: Use alternative RET encoding
x86/ftrace: Use alternative RET encoding
x86/bpf: Use alternative RET encoding
x86/kvm: Fix SETcc emulation for return thunks
x86/vsyscall_emu/64: Don't use RET in vsyscall emulation
x86/sev: Avoid using __x86_return_thunk
x86: Use return-thunk in asm code
x86/entry: Avoid very early RET
objtool: Treat .text.__x86.* as noinstr
x86: Add magic AMD return-thunk
objtool: skip non-text sections when adding return-thunk sites
x86/bugs: Report AMD retbleed vulnerability
x86/bugs: Add AMD retbleed= boot parameter
x86/bugs: Enable STIBP for JMP2RET
x86/bugs: Keep a per-CPU IA32_SPEC_CTRL value
x86/entry: Add kernel IBRS implementation
x86/bugs: Optimize SPEC_CTRL MSR writes
x86/speculation: Add spectre_v2=ibrs option to support Kernel IBRS
x86/bugs: Split spectre_v2_select_mitigation() and
spectre_v2_user_select_mitigation()
x86/bugs: Report Intel retbleed vulnerability
intel_idle: Disable IBRS during long idle
objtool: Update Retpoline validation
x86/xen: Rename SYS* entry points
x86/xen: Add UNTRAIN_RET
x86/bugs: Add retbleed=ibpb
x86/bugs: Do IBPB fallback check only once
x86/cpu/amd: Add Spectral Chicken
x86/speculation: Fix RSB filling with CONFIG_RETPOLINE=n
x86/speculation: Fix firmware entry SPEC_CTRL handling
x86/speculation: Fix SPEC_CTRL write on SMT state change
x86/speculation: Use cached host SPEC_CTRL value for guest entry/exit
x86/speculation: Remove x86_spec_ctrl_mask
objtool: Re-add UNWIND_HINT_{SAVE_RESTORE}
KVM: VMX: Flatten __vmx_vcpu_run()
KVM: VMX: Convert launched argument to flags
KVM: VMX: Prevent guest RSB poisoning attacks with eIBRS
KVM: VMX: Fix IBRS handling after vmexit
x86/speculation: Fill RSB on vmexit for IBRS
KVM: VMX: Prevent RSB underflow before vmenter
x86/common: Stamp out the stepping madness
x86/cpu/amd: Enumerate BTC_NO
x86/retbleed: Add fine grained Kconfig knobs
redhat/configs: Add new mitigation configs for RetBleed CVEs
x86/bugs: Add Cannon lake to RETBleed affected CPU list
x86/entry: Move PUSH_AND_CLEAR_REGS() back into error_entry
x86/bugs: Do not enable IBPB-on-entry when IBPB is not supported
x86/kexec: Disable RET on kexec
x86/speculation: Disable RRSBA behavior
x86/static_call: Serialize __static_call_fixup() properly
x86/asm/32: Fix ANNOTATE_UNRET_SAFE use on 32-bit
x86/bugs: Mark retbleed_strings static
x86/entry: Remove UNTRAIN_RET from native_irq_return_ldt
x86/speculation: Use DECLARE_PER_CPU for x86_spec_ctrl_current
x86/bugs: Remove apostrophe typo
efi/x86: use naked RET on mixed mode call wrapper
x86/kvm: fix FASTOP_SIZE when return thunks are enabled
KVM: emulate: do not adjust size of fastop and setcc subroutines
um: Add missing apply_returns()
tools headers cpufeatures: Sync with the kernel sources
intel_idle: Fix false positive RCU splats due to incorrect hardirqs
state