linux/arch/mips
Huacai Chen e02e07e312
MIPS: Loongson: Introduce and use loongson_llsc_mb()
On the Loongson-2G/2H/3A/3B there is a hardware flaw that ll/sc and
lld/scd is very weak ordering. We should add sync instructions "before
each ll/lld" and "at the branch-target between ll/sc" to workaround.
Otherwise, this flaw will cause deadlock occasionally (e.g. when doing
heavy load test with LTP).

Below is the explaination of CPU designer:

"For Loongson 3 family, when a memory access instruction (load, store,
or prefetch)'s executing occurs between the execution of LL and SC, the
success or failure of SC is not predictable. Although programmer would
not insert memory access instructions between LL and SC, the memory
instructions before LL in program-order, may dynamically executed
between the execution of LL/SC, so a memory fence (SYNC) is needed
before LL/LLD to avoid this situation.

Since Loongson-3A R2 (3A2000), we have improved our hardware design to
handle this case. But we later deduce a rarely circumstance that some
speculatively executed memory instructions due to branch misprediction
between LL/SC still fall into the above case, so a memory fence (SYNC)
at branch-target (if its target is not between LL/SC) is needed for
Loongson 3A1000, 3B1500, 3A2000 and 3A3000.

Our processor is continually evolving and we aim to to remove all these
workaround-SYNCs around LL/SC for new-come processor."

Here is an example:

Both cpu1 and cpu2 simutaneously run atomic_add by 1 on same atomic var,
this bug cause both 'sc' run by two cpus (in atomic_add) succeed at same
time('sc' return 1), and the variable is only *added by 1*, sometimes,
which is wrong and unacceptable(it should be added by 2).

Why disable fix-loongson3-llsc in compiler?
Because compiler fix will cause problems in kernel's __ex_table section.

This patch fix all the cases in kernel, but:

+. the fix at the end of futex_atomic_cmpxchg_inatomic is for branch-target
of 'bne', there other cases which smp_mb__before_llsc() and smp_llsc_mb() fix
the ll and branch-target coincidently such as atomic_sub_if_positive/
cmpxchg/xchg, just like this one.

+. Loongson 3 does support CONFIG_EDAC_ATOMIC_SCRUB, so no need to touch
edac.h

+. local_ops and cmpxchg_local should not be affected by this bug since
only the owner can write.

+. mips_atomic_set for syscall.c is deprecated and rarely used, just let
it go

Signed-off-by: Huacai Chen <chenhc@lemote.com>
Signed-off-by: Huang Pei <huangpei@loongson.cn>
[paul.burton@mips.com:
  - Simplify the addition of -mno-fix-loongson3-llsc to cflags, and add
    a comment describing why it's there.
  - Make loongson_llsc_mb() a no-op when
    CONFIG_CPU_LOONGSON3_WORKAROUNDS=n, rather than a compiler memory
    barrier.
  - Add a comment describing the bug & how loongson_llsc_mb() helps
    in asm/barrier.h.]
Signed-off-by: Paul Burton <paul.burton@mips.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: ambrosehua@gmail.com
Cc: Steven J . Hill <Steven.Hill@cavium.com>
Cc: linux-mips@linux-mips.org
Cc: Fuxin Zhang <zhangfx@lemote.com>
Cc: Zhangjin Wu <wuzhangjin@gmail.com>
Cc: Li Xuefeng <lixuefeng@loongson.cn>
Cc: Xu Chenghua <xuchenghua@loongson.cn>
2019-02-04 10:53:34 -08:00
..
alchemy A few early MIPS fixes for 4.21: 2019-01-05 12:48:25 -08:00
ar7 mm: remove include/linux/bootmem.h 2018-10-31 08:54:16 -07:00
ath25 PCI: consolidate PCI config entry in drivers/pci 2018-11-23 11:45:34 +09:00
ath79 PCI: consolidate PCI config entry in drivers/pci 2018-11-23 11:45:34 +09:00
bcm47xx MIPS: BCM47XX: Setup struct device for the SoC 2019-01-09 13:21:02 -08:00
bcm63xx A few early MIPS fixes for 4.21: 2019-01-05 12:48:25 -08:00
bmips mm: remove include/linux/bootmem.h 2018-10-31 08:54:16 -07:00
boot DTS: CI20: Fix bugs in ci20's device tree. 2019-01-25 11:19:37 -08:00
cavium-octeon MIPS: OCTEON: fix kexec support 2019-01-14 13:51:03 -08:00
cobalt
configs MIPS: ath79: Enable OF serial ports in the default config 2019-01-07 13:39:06 -08:00
crypto MIPS: crypto: Add crc32 and crc32c hw accelerated module 2018-02-19 20:50:36 +00:00
dec mm: remove include/linux/bootmem.h 2018-10-31 08:54:16 -07:00
emma mm: remove include/linux/bootmem.h 2018-10-31 08:54:16 -07:00
fw mm: remove include/linux/bootmem.h 2018-10-31 08:54:16 -07:00
generic MIPS: mscc: add PCB120 to the ocelot fitImage 2018-10-09 10:38:29 -07:00
include MIPS: Loongson: Introduce and use loongson_llsc_mb() 2019-02-04 10:53:34 -08:00
jazz MIPS: jazz: fix 64bit build 2019-01-09 13:14:34 -08:00
jz4740 mmc: jz4740: Use GPIO descriptor for power 2018-12-17 08:26:24 +01:00
kernel jump_label: move 'asm goto' support test to Kconfig 2019-01-06 09:46:51 +09:00
kvm Kconfig updates for v4.21 2018-12-29 13:03:29 -08:00
lantiq A few MIPS fixes for 5.0: 2019-01-20 10:33:18 +12:00
lasat mm: remove include/linux/bootmem.h 2018-10-31 08:54:16 -07:00
lib MIPS: lib: Use kernel_pref & user_pref in memcpy() 2018-10-15 23:11:14 -07:00
loongson32 MIPS: Loongson: Merge load addresses 2018-07-30 18:59:01 -07:00
loongson64 MIPS: Loongson: Introduce and use loongson_llsc_mb() 2019-02-04 10:53:34 -08:00
math-emu Remove 'type' argument from access_ok() function 2019-01-03 18:57:57 -08:00
mm MIPS: Loongson: Introduce and use loongson_llsc_mb() 2019-02-04 10:53:34 -08:00
mti-malta MIPS: malta: Use img-ascii-lcd driver for LCD display 2018-11-20 21:05:39 -08:00
net mips: bpf: implement jitting of BPF_ALU | BPF_ARSH | BPF_X 2018-12-07 13:30:48 -08:00
netlogic mm: remove include/linux/bootmem.h 2018-10-31 08:54:16 -07:00
oprofile Remove 'type' argument from access_ok() function 2019-01-03 18:57:57 -08:00
paravirt mips: unify prom_putchar() declarations 2018-07-17 09:40:17 -07:00
pci MIPS: OCTEON: don't set octeon_dma_bar_type if PCI is disabled 2019-01-29 11:47:02 -08:00
pic32 mips: unify prom_putchar() declarations 2018-07-17 09:40:17 -07:00
pistachio
pmcs-msp71xx PCI: consolidate PCI config entry in drivers/pci 2018-11-23 11:45:34 +09:00
pnx833x mtd: rawnand: Move platform_nand_xxx definitions out of rawnand.h 2018-10-03 11:12:25 +02:00
power
ralink A few early MIPS fixes for 4.21: 2019-01-05 12:48:25 -08:00
rb532 ata: rb532_cf: Convert to use GPIO descriptors 2018-12-04 17:15:25 -07:00
sgi-ip22 MIPS: Remove no-op/identity casts 2018-08-31 11:49:20 -07:00
sgi-ip27 MIPS: Loongson3,SGI-IP27: Simplify max_low_pfn calculation 2018-11-15 15:42:15 -08:00
sgi-ip32 MIPS: IP32: use generic dma noncoherent ops 2018-06-24 09:27:27 -07:00
sibyte Remove 'type' argument from access_ok() function 2019-01-03 18:57:57 -08:00
sni MIPS: sni: Remove the read_persistent_clock() 2018-05-14 23:58:25 +01:00
tools MIPS: Use a custom elf-entry program to find kernel entry point 2018-08-30 09:39:22 -07:00
txx9 PCI: consolidate PCI config entry in drivers/pci 2018-11-23 11:45:34 +09:00
vdso MIPS: VDSO: Include $(ccflags-vdso) in o32,n32 .lds builds 2019-01-29 11:50:58 -08:00
vr41xx PCI: consolidate PCI config entry in drivers/pci 2018-11-23 11:45:34 +09:00
Kbuild
Kbuild.platforms
Kconfig MIPS: Loongson: Introduce and use loongson_llsc_mb() 2019-02-04 10:53:34 -08:00
Kconfig.debug Kconfig: consolidate the "Kernel hacking" menu 2018-08-02 08:06:48 +09:00
Makefile mips: generate uapi header and system call table files 2018-12-14 11:19:02 -08:00
Makefile.postlink