linux/arch/riscv/mm
Charlie Jenkins 7c1e5b9690
riscv: Disable preemption while handling PR_RISCV_CTX_SW_FENCEI_OFF
The icache will be flushed in switch_to() if force_icache_flush is true,
or in flush_icache_deferred() if icache_stale_mask is set. Between
setting force_icache_flush to false and calculating the new
icache_stale_mask, preemption needs to be disabled. There are two
reasons for this:

1. If CPU migration happens between force_icache_flush = false, and the
   icache_stale_mask is set, an icache flush will not be emitted.
2. smp_processor_id() is used in set_icache_stale_mask() to mark the
   current CPU as not needing another flush since a flush will have
   happened either by userspace or by the kernel when performing the
   migration. smp_processor_id() is currently called twice with preemption
   enabled which causes a race condition. It allows
   icache_stale_mask to be populated with inconsistent CPU ids.

Resolve these two issues by setting the icache_stale_mask before setting
force_icache_flush to false, and using get_cpu()/put_cpu() to obtain the
smp_processor_id().

Signed-off-by: Charlie Jenkins <charlie@rivosinc.com>
Fixes: 6b9391b581 ("riscv: Include riscv_set_icache_flush_ctx prctl")
Link: https://lore.kernel.org/r/20240903-fix_fencei_optimization-v2-1-8025f20171fc@rivosinc.com
Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2024-09-10 20:38:46 -07:00
..
cache-ops.c riscv: split cache ops out of dma-noncoherent.c 2023-11-07 09:37:42 -08:00
cacheflush.c riscv: Disable preemption while handling PR_RISCV_CTX_SW_FENCEI_OFF 2024-09-10 20:38:46 -07:00
context.c Merge patch series "riscv: ASID-related and UP-related TLB flush enhancements" 2024-04-30 10:35:48 -07:00
dma-noncoherent.c dma-mapping: Simplify arch_setup_dma_ops() 2024-04-26 12:07:28 +02:00
extable.c riscv: select DCACHE_WORD_ACCESS for efficient unaligned access HW 2024-01-09 20:18:20 -08:00
fault.c riscv/mm: Add handling for VM_FAULT_SIGSEGV in mm_fault_error() 2024-08-01 07:15:27 -07:00
hugetlbpage.c mm: provide mm_struct and address to huge_ptep_get() 2024-07-12 15:52:15 -07:00
init.c riscv: Do not restrict memory size because of linear mapping on nommu 2024-09-03 07:57:27 -07:00
kasan_init.c percpu: 2024-01-18 15:01:28 -08:00
Makefile riscv: mm: Combine the SMP and UP TLB flush code 2024-04-29 10:49:29 -07:00
pageattr.c riscv: rewrite __kernel_map_pages() to fix sleeping in invalid context 2024-05-22 09:15:09 -07:00
pgtable.c riscv: Only flush the mm icache when setting an exec pte 2024-03-20 08:56:08 -07:00
physaddr.c riscv: Use PUD/P4D/PGD pages for the linear mapping 2023-04-18 20:43:04 -07:00
pmem.c RISC-V: capitalise CMO op macros 2023-11-05 09:11:23 -08:00
ptdump.c riscv: mm: Take memory hotplug read-lock during kernel page table dump 2024-06-26 08:42:43 -07:00
tlbflush.c RISC-V Patches for the 6.10 Merge Window, Part 1 2024-05-22 09:56:00 -07:00