Commit 5a0e9b57 authored by Anton Blanchard's avatar Anton Blanchard Committed by Benjamin Herrenschmidt

powerpc: Use lwsync for acquire barrier if CPU supports it

Nick Piggin discovered that lwsync barriers around locks were faster than isync
on 970. That was a long time ago and I completely dropped the ball in testing
his patches across other ppc64 processors.

Turns out the idea helps on other chips. Using a microbenchmark that
uses a lot of threads to contend on a global pthread mutex (and therefore a
global futex), POWER6 improves 8% and POWER7 improves 2%. I checked POWER5
and while I couldn't measure an improvement, there was no regression.

This patch uses the lwsync patching code to replace the isyncs with lwsyncs
on CPUs that support the instruction. We were marking POWER3 and RS64 as lwsync
capable but in reality they treat it as a full sync (ie slow). Remove the
CPU_FTR_LWSYNC bit from these CPUs so they continue to use the faster isync
method.
Signed-off-by: default avatarAnton Blanchard <anton@samba.org>
Signed-off-by: default avatarBenjamin Herrenschmidt <benh@kernel.crashing.org>
parent 53eae228
...@@ -381,9 +381,9 @@ extern const char *powerpc_base_platform; ...@@ -381,9 +381,9 @@ extern const char *powerpc_base_platform;
#define CPU_FTRS_GENERIC_32 (CPU_FTR_COMMON | CPU_FTR_NODSISRALIGN) #define CPU_FTRS_GENERIC_32 (CPU_FTR_COMMON | CPU_FTR_NODSISRALIGN)
/* 64-bit CPUs */ /* 64-bit CPUs */
#define CPU_FTRS_POWER3 (CPU_FTR_USE_TB | CPU_FTR_LWSYNC | \ #define CPU_FTRS_POWER3 (CPU_FTR_USE_TB | \
CPU_FTR_IABR | CPU_FTR_PPC_LE) CPU_FTR_IABR | CPU_FTR_PPC_LE)
#define CPU_FTRS_RS64 (CPU_FTR_USE_TB | CPU_FTR_LWSYNC | \ #define CPU_FTRS_RS64 (CPU_FTR_USE_TB | \
CPU_FTR_IABR | \ CPU_FTR_IABR | \
CPU_FTR_MMCRA | CPU_FTR_CTRL) CPU_FTR_MMCRA | CPU_FTR_CTRL)
#define CPU_FTRS_POWER4 (CPU_FTR_USE_TB | CPU_FTR_LWSYNC | \ #define CPU_FTRS_POWER4 (CPU_FTR_USE_TB | CPU_FTR_LWSYNC | \
......
...@@ -37,7 +37,11 @@ static inline void isync(void) ...@@ -37,7 +37,11 @@ static inline void isync(void)
#endif #endif
#ifdef CONFIG_SMP #ifdef CONFIG_SMP
#define PPC_ACQUIRE_BARRIER "\n\tisync\n" #define __PPC_ACQUIRE_BARRIER \
START_LWSYNC_SECTION(97); \
isync; \
MAKE_LWSYNC_SECTION_ENTRY(97, __lwsync_fixup);
#define PPC_ACQUIRE_BARRIER "\n" stringify_in_c(__PPC_ACQUIRE_BARRIER)
#define PPC_RELEASE_BARRIER stringify_in_c(LWSYNC) "\n" #define PPC_RELEASE_BARRIER stringify_in_c(LWSYNC) "\n"
#else #else
#define PPC_ACQUIRE_BARRIER #define PPC_ACQUIRE_BARRIER
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment