io_uring: provide FIFO ordering for task_work
authorJens Axboe <axboe@kernel.dk>
Wed, 10 Feb 2021 00:03:20 +0000 (00:03 +0000)
committerJens Axboe <axboe@kernel.dk>
Wed, 10 Feb 2021 14:28:43 +0000 (07:28 -0700)
commit7cbf1722d5fc5779946ee8f338e9e38b5de15856
treecafc7b4a2cc122789ca7c1b2045d44ec2a36847a
parent1b4c351f6eb7467c77fc19e0cd7e5f0083ecd847
io_uring: provide FIFO ordering for task_work

task_work is a LIFO list, due to how it's implemented as a lockless
list. For long chains of task_work, this can be problematic as the
first entry added is the last one processed. Similarly, we'd waste
a lot of CPU cycles reversing this list.

Wrap the task_work so we have a single task_work entry per task per
ctx, and use that to run it in the right order.

Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
fs/io-wq.h
fs/io_uring.c
include/linux/io_uring.h