riscv: xchg: Prefetch the destination word for sc.w
authorGuo Ren <guoren@linux.alibaba.com>
Mon, 21 Apr 2025 14:24:41 +0000 (16:24 +0200)
committerPalmer Dabbelt <palmer@dabbelt.com>
Thu, 5 Jun 2025 18:09:39 +0000 (11:09 -0700)
commiteb87e56d651d0a72009842bc0d8eeae7b605c97d
treea2b81060b56dd0c2d93eb9c52320157a22d4f79b
parenta5f947c73115efb6fb0d9579e71ce1ee5cf706aa
riscv: xchg: Prefetch the destination word for sc.w

The cost of changing a cacheline from shared to exclusive state can be
significant, especially when this is triggered by an exclusive store,
since it may result in having to retry the transaction.

This patch makes use of prefetch.w to prefetch cachelines for write
prior to lr/sc loops when using the xchg_small atomic routine.

This patch is inspired by commit 0ea366f5e1b6 ("arm64: atomics:
prefetch the destination word for write prior to stxr").

Signed-off-by: Guo Ren <guoren@linux.alibaba.com>
Signed-off-by: Guo Ren <guoren@kernel.org>
Link: https://lore.kernel.org/r/20231231082955.16516-4-guoren@kernel.org
Tested-by: Andrea Parri <parri.andrea@gmail.com>
Link: https://lore.kernel.org/r/20250421142441.395849-5-alexghiti@rivosinc.com
Signed-off-by: Alexandre Ghiti <alexghiti@rivosinc.com>
Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
arch/riscv/include/asm/cmpxchg.h