sched: Prepare to use balance_push in ttwu()
authorPeter Zijlstra <peterz@infradead.org>
Wed, 20 Jan 2021 14:05:41 +0000 (15:05 +0100)
committerPeter Zijlstra <peterz@infradead.org>
Fri, 22 Jan 2021 14:09:43 +0000 (15:09 +0100)
commit975707f227b07a8212060f94447171d15d7a681b
treeb9703358195945550846d0e59259178cf5274d83
parent640f17c82460e9724fd256f0a1f5d99e7ff0bda4
sched: Prepare to use balance_push in ttwu()

In preparation of using the balance_push state in ttwu() we need it to
provide a reliable and consistent state.

The immediate problem is that rq->balance_callback gets cleared every
schedule() and then re-set in the balance_push_callback() itself. This
is not a reliable signal, so add a variable that stays set during the
entire time.

Also move setting it before the synchronize_rcu() in
sched_cpu_deactivate(), such that we get guaranteed visibility to
ttwu(), which is a preempt-disable region.

Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Valentin Schneider <valentin.schneider@arm.com>
Tested-by: Valentin Schneider <valentin.schneider@arm.com>
Link: https://lkml.kernel.org/r/20210121103506.966069627@infradead.org
kernel/sched/core.c
kernel/sched/sched.h