From: Paul Murphy Date: Thu, 29 Oct 2015 20:48:47 +0000 (-0500) Subject: powerpc: Spinlock optimization and cleanup X-Git-Tag: glibc-2.23~188 X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=9695cb3e6528d9f6f5c7c7f96bd6c94e5ba0e2c4;p=thirdparty%2Fglibc.git powerpc: Spinlock optimization and cleanup This patch optimizes powerpc spinlock implementation by: * Use the correct EH hint bit on the larx for supported ISA. For lock acquisition, the thread that acquired the lock with a successful stcx does not want to give away the write ownership on the cacheline. The idea is to make the load reservation "sticky" about retaining write authority to the line. That way, the store that must inevitably come to release the lock can succeed quickly and not contend with other threads issuing lwarx. If another thread does a store to the line (false sharing), the winning thread must give up write authority to the proper value of EH for the larx for a lock acquisition is 1. * Increase contented lock performance by up to 40%, and no measurable impact on uncontended locks on P8. Thanks to Adhemerval Zanella who did most of the work. I've run some tests, and addressed some minor feedback. * sysdeps/powerpc/nptl/pthread_spin_lock.c (pthread_spin_lock): Add lwarx hint, and use macro for acquire instruction. * sysdeps/powerpc/nptl/pthread_spin_trylock.c (pthread_spin_trylock): Likewise. * sysdep/unix/sysv/linux/powerpc/pthread_spin_unlock.c: Move to ... * sysdeps/powerpc/nptl/pthread_spin_unlock.c: ... here, and update to use new atomic macros. --- diff --git a/ChangeLog b/ChangeLog index a5440e2a74d..d822095ed5a 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,3 +1,14 @@ +2015-11-19 Adhemerval Zanella + Paul E. Murphy + + * sysdeps/powerpc/nptl/pthread_spin_lock.c (pthread_spin_lock): + Add lwarx hint, and use macro for acquire instruction. + * sysdeps/powerpc/nptl/pthread_spin_trylock.c (pthread_spin_trylock): + Likewise. + * sysdep/unix/sysv/linux/powerpc/pthread_spin_unlock.c: Move to ... + * sysdeps/powerpc/nptl/pthread_spin_unlock.c: ... here, and + update to use new atomic macros. + 2015-11-19 Paul E. Murphy * sysdeps/unix/sysv/linux/powerpc/elision-trylock.c diff --git a/sysdeps/powerpc/nptl/pthread_spin_lock.c b/sysdeps/powerpc/nptl/pthread_spin_lock.c index d7d4cae6fa6..fae7f7e0b94 100644 --- a/sysdeps/powerpc/nptl/pthread_spin_lock.c +++ b/sysdeps/powerpc/nptl/pthread_spin_lock.c @@ -24,12 +24,12 @@ pthread_spin_lock (pthread_spinlock_t *lock) unsigned int __tmp; asm volatile ( - "1: lwarx %0,0,%1\n" + "1: lwarx %0,0,%1" MUTEX_HINT_ACQ "\n" " cmpwi 0,%0,0\n" " bne- 2f\n" " stwcx. %2,0,%1\n" " bne- 2f\n" - " isync\n" + __ARCH_ACQ_INSTR "\n" " .subsection 1\n" "2: lwzx %0,0,%1\n" " cmpwi 0,%0,0\n" diff --git a/sysdeps/powerpc/nptl/pthread_spin_trylock.c b/sysdeps/powerpc/nptl/pthread_spin_trylock.c index c485aa4bf93..09791c36a8e 100644 --- a/sysdeps/powerpc/nptl/pthread_spin_trylock.c +++ b/sysdeps/powerpc/nptl/pthread_spin_trylock.c @@ -25,13 +25,13 @@ pthread_spin_trylock (pthread_spinlock_t *lock) unsigned int old; int err = EBUSY; - asm ("1: lwarx %0,0,%2\n" + asm ("1: lwarx %0,0,%2" MUTEX_HINT_ACQ "\n" " cmpwi 0,%0,0\n" " bne 2f\n" " stwcx. %3,0,%2\n" " bne- 1b\n" " li %1,0\n" - " isync\n" + __ARCH_ACQ_INSTR "\n" "2: " : "=&r" (old), "=&r" (err) : "r" (lock), "r" (1), "1" (err) diff --git a/sysdeps/unix/sysv/linux/powerpc/pthread_spin_unlock.c b/sysdeps/powerpc/nptl/pthread_spin_unlock.c similarity index 93% rename from sysdeps/unix/sysv/linux/powerpc/pthread_spin_unlock.c rename to sysdeps/powerpc/nptl/pthread_spin_unlock.c index 7af694f4ab4..f830ad2880a 100644 --- a/sysdeps/unix/sysv/linux/powerpc/pthread_spin_unlock.c +++ b/sysdeps/powerpc/nptl/pthread_spin_unlock.c @@ -22,7 +22,6 @@ int pthread_spin_unlock (pthread_spinlock_t *lock) { - __asm __volatile (__ARCH_REL_INSTR ::: "memory"); - *lock = 0; + atomic_store_release (lock, 0); return 0; }