leandrof/linux
1
0
Fork 0
forked from Minki/linux
Code Issues Pull Requests Packages Projects Releases Wiki Activity

ipc/sem.c: avoid using spin_unlock_wait()

Browse Source 
a) The ACQUIRE in spin_lock() applies to the read, not to the store, at
   least for powerpc.  This forces to add a smp_mb() into the fast path.

b) The memory barrier provided by spin_unlock_wait() is right now arch
   dependent.

Therefore: Use spin_lock()/spin_unlock() instead of spin_unlock_wait().

Advantage: faster single op semop calls(), observed +8.9% on x86.  (the
other solution would be arch dependencies in ipc/sem).

Disadvantage: slower complex op semop calls, if (and only if) there are
no sleeping operations.

The next patch adds hysteresis, this further reduces the probability
that the slow path is used.

Link: http://lkml.kernel.org/r/1476851896-3590-2-git-send-email-manfred@colorfullife.com
Signed-off-by: Manfred Spraul <manfred@colorfullife.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Davidlohr Bueso <dave@stgolabs.net>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: <1vier1@web.de>
Cc: kernel test robot <xiaolong.ye@intel.com>
Cc: <felixh@informatik.uni-bremen.de>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
This commit is contained in:
Manfred Spraul 2017-02-27 14:28:15 -08:00 committed by Linus Torvalds
parent 0886551480
commit 27d7be1801
1 changed files with 3 additions and 22 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

25
ipc/sem.c
View File

@ -278,24 +278,13 @@ static void complexmode_enter(struct sem_array *sma)
return;
}
/* We need a full barrier after seting complex_mode:
* The write to complex_mode must be visible
* before we read the first sem->lock spinlock state.
*/
smp_store_mb(sma->complex_mode, true);
sma->complex_mode = true;
for (i = 0; i < sma->sem_nsems; i++) {
sem = sma->sem_base + i;
spin_unlock_wait(&sem->lock);
spin_lock(&sem->lock);
spin_unlock(&sem->lock);
}
/*
* spin_unlock_wait() is not a memory barriers, it is only a
* control barrier. The code must pair with spin_unlock(&sem->lock),
* thus just the control barrier is insufficient.
*
* smp_rmb() is sufficient, as writes cannot pass the control barrier.
*/
smp_rmb();
}
/*
@ -361,14 +350,6 @@ static inline int sem_lock(struct sem_array *sma, struct sembuf *sops,
*/
spin_lock(&sem->lock);
/*
* See 51d7d5205d33
* ("powerpc: Add smp_mb() to arch_spin_is_locked()"):
* A full barrier is required: the write of sem->lock
* must be visible before the read is executed
*/
smp_mb();
if (!smp_load_acquire(&sma->complex_mode)) {
/* fast path successful! */
return sops->sem_num;