sched: Optimize finish_lock_switch()
authorPeter Zijlstra <peterz@infradead.org>
Thu, 10 Dec 2020 16:14:08 +0000 (17:14 +0100)
committerPeter Zijlstra <peterz@infradead.org>
Tue, 15 Dec 2020 10:27:53 +0000 (11:27 +0100)
commitae7927023243dcc7389b2d59b16c09cbbeaecc36
tree64bdf7a8cddcc7a2c2631a107dee1b5bbc4e7405
parent5b78f2dc315354c05300795064f587366a02c6ff
sched: Optimize finish_lock_switch()

The kernel test robot measured a -1.6% performance regression on
will-it-scale/sched_yield due to commit:

  2558aacff858 ("sched/hotplug: Ensure only per-cpu kthreads run during hotplug")

Even though we were careful to replace a single load with another
single load from the same cacheline.

Restore finish_lock_switch() to the exact state before the offending
patch and solve the problem differently.

Fixes: 2558aacff858 ("sched/hotplug: Ensure only per-cpu kthreads run during hotplug")
Reported-by: kernel test robot <oliver.sang@intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Link: https://lkml.kernel.org/r/20201210161408.GX3021@hirez.programming.kicks-ass.net
kernel/sched/core.c
kernel/sched/sched.h