mm/munlock: protect the per-CPU pagevec by a local_lock_t
authorSebastian Andrzej Siewior <bigeasy@linutronix.de>
Fri, 1 Apr 2022 18:28:33 +0000 (11:28 -0700)
committerLinus Torvalds <torvalds@linux-foundation.org>
Fri, 1 Apr 2022 18:46:09 +0000 (11:46 -0700)
commitadb11e78c5dc5e26774acb05f983da36447f7911
treebacf697b157f86712edffebbfd7c08aa7d83e966
parent577e9846f8a9e7b09cd356ae0d59a66e19402e8b
mm/munlock: protect the per-CPU pagevec by a local_lock_t

The access to mlock_pvec is protected by disabling preemption via
get_cpu_var() or implicit by having preemption disabled by the caller
(in mlock_page_drain() case).  This breaks on PREEMPT_RT since
folio_lruvec_lock_irq() acquires a sleeping lock in this section.

Create struct mlock_pvec which consits of the local_lock_t and the
pagevec.  Acquire the local_lock() before accessing the per-CPU pagevec.
Replace mlock_page_drain() with a _local() version which is invoked on
the local CPU and acquires the local_lock_t and a _remote() version
which uses the pagevec from a remote CPU which offline.

Link: https://lkml.kernel.org/r/YjizWi9IY0mpvIfb@linutronix.de
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Acked-by: Hugh Dickins <hughd@google.com>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: Matthew Wilcox <willy@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
mm/internal.h
mm/migrate.c
mm/mlock.c
mm/page_alloc.c
mm/rmap.c
mm/swap.c