io_uring: only wake up sq thread while current task is in io worker context
authorXiaoguang Wang <xiaoguang.wang@linux.alibaba.com>
Thu, 12 Nov 2020 16:44:08 +0000 (00:44 +0800)
committerJens Axboe <axboe@kernel.dk>
Wed, 9 Dec 2020 19:04:00 +0000 (12:04 -0700)
commit2e9dbe902d1020ef70f968e8675c8d2457c4ffaa
treeaa16acaef8ff5071fa7d7ea74ec15818f04e7aee
parent906a3c6f9ca072e917c701f7421647e169740954
io_uring: only wake up sq thread while current task is in io worker context

If IORING_SETUP_SQPOLL is enabled, sqes are either handled in sq thread
task context or in io worker task context. If current task context is sq
thread, we don't need to check whether should wake up sq thread.

io_iopoll_req_issued() calls wq_has_sleeper(), which has smp_mb() memory
barrier, before this patch, perf shows obvious overhead:
  Samples: 481K of event 'cycles', Event count (approx.): 299807382878
  Overhead  Comma  Shared Object     Symbol
     3.69%  :9630  [kernel.vmlinux]  [k] io_issue_sqe

With this patch, perf shows:
  Samples: 482K of event 'cycles', Event count (approx.): 299929547283
  Overhead  Comma  Shared Object     Symbol
     0.70%  :4015  [kernel.vmlinux]  [k] io_issue_sqe

It shows some obvious improvements.

Signed-off-by: Xiaoguang Wang <xiaoguang.wang@linux.alibaba.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
fs/io_uring.c