1 From fee960bed5e857eb126c4e56dd9ff85938356579 Mon Sep 17 00:00:00 2001
2 From: Mark Rutland <mark.rutland@arm.com>
3 Date: Wed, 3 May 2017 16:09:33 +0100
4 Subject: arm64: xchg: hazard against entire exchange variable
6 From: Mark Rutland <mark.rutland@arm.com>
8 commit fee960bed5e857eb126c4e56dd9ff85938356579 upstream.
10 The inline assembly in __XCHG_CASE() uses a +Q constraint to hazard
11 against other accesses to the memory location being exchanged. However,
12 the pointer passed to the constraint is a u8 pointer, and thus the
13 hazard only applies to the first byte of the location.
15 GCC can take advantage of this, assuming that other portions of the
16 location are unchanged, as demonstrated with the following test case:
23 unsigned long update_char_hazard(union u *u)
28 asm ("str %1, %0" : "+Q" (*(char *)&u->l) : "r" (0UL));
34 unsigned long update_long_hazard(union u *u)
39 asm ("str %1, %0" : "+Q" (*(long *)&u->l) : "r" (0UL));
45 The linaro 15.08 GCC 5.1.1 toolchain compiles the above as follows when
48 0000000000000000 <update_char_hazard>:
49 0: d2800001 mov x1, #0x0 // #0
50 4: f9000001 str x1, [x0]
51 8: d2800000 mov x0, #0x0 // #0
54 0000000000000010 <update_long_hazard>:
55 10: b9400401 ldr w1, [x0,#4]
56 14: d2800002 mov x2, #0x0 // #0
57 18: f9000002 str x2, [x0]
58 1c: b9400400 ldr w0, [x0,#4]
59 20: 4a000020 eor w0, w1, w0
62 This patch fixes the issue by passing an unsigned long pointer into the
63 +Q constraint, as we do for our cmpxchg code. This may hazard against
64 more than is necessary, but this is better than missing a necessary
67 Fixes: 305d454aaa29 ("arm64: atomics: implement native {relaxed, acquire, release} atomics")
68 Acked-by: Will Deacon <will.deacon@arm.com>
69 Signed-off-by: Mark Rutland <mark.rutland@arm.com>
70 Signed-off-by: Catalin Marinas <catalin.marinas@arm.com>
71 Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
74 arch/arm64/include/asm/cmpxchg.h | 2 +-
75 1 file changed, 1 insertion(+), 1 deletion(-)
77 --- a/arch/arm64/include/asm/cmpxchg.h
78 +++ b/arch/arm64/include/asm/cmpxchg.h
79 @@ -46,7 +46,7 @@ static inline unsigned long __xchg_case_
80 " swp" #acq_lse #rel #sz "\t%" #w "3, %" #w "0, %2\n" \
83 - : "=&r" (ret), "=&r" (tmp), "+Q" (*(u8 *)ptr) \
84 + : "=&r" (ret), "=&r" (tmp), "+Q" (*(unsigned long *)ptr) \