asm-generic/tlb: provide MMU_GATHER_TABLE_FREE
authorPeter Zijlstra <peterz@infradead.org>
Tue, 4 Feb 2020 01:37:11 +0000 (17:37 -0800)
committerLinus Torvalds <torvalds@linux-foundation.org>
Tue, 4 Feb 2020 03:05:26 +0000 (03:05 +0000)
commit0d6e24d430ef23280d8dea0ba1faeefc66c26a57
tree6fb47b4bee252781c30105392a143e9a9a688903
parent580a586c409ab3040b7284a19cd9e281692c40c7
asm-generic/tlb: provide MMU_GATHER_TABLE_FREE

As described in the comment, the correct order for freeing pages is:

 1) unhook page
 2) TLB invalidate page
 3) free page

This order equally applies to page directories.

Currently there are two correct options:

 - use tlb_remove_page(), when all page directores are full pages and
   there are no futher contraints placed by things like software
   walkers (HAVE_FAST_GUP).

 - use MMU_GATHER_RCU_TABLE_FREE and tlb_remove_table() when the
   architecture does not do IPI based TLB invalidate and has
   HAVE_FAST_GUP (or software TLB fill).

This however leaves architectures that don't have page based directories
but don't need RCU in a bind.  For those, provide MMU_GATHER_TABLE_FREE,
which provides the independent batching for directories without the
additional RCU freeing.

Link: http://lkml.kernel.org/r/20200116064531.483522-10-aneesh.kumar@linux.ibm.com
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
Cc: Michael Ellerman <mpe@ellerman.id.au>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
arch/Kconfig
arch/arm/include/asm/tlb.h
include/asm-generic/tlb.h
mm/mmu_gather.c