leandrof/linux
1
0
Fork 0
forked from Minki/linux
Code Issues Pull Requests Packages Projects Releases Wiki Activity
49f1c68e04
linux/io_uring
History
Pavel Begunkov 49f1c68e04 io_uring: optimise submission side poll_refs 
The final poll_refs put in __io_arm_poll_handler() takes quite some
cycles. When we're arming from the original task context task_work won't
be run, so in this case we can assume that we won't race with task_works
and so not take the initial ownership ref.

One caveat is that after arming a poll we may race with it, so we have
to add a bunch of io_poll_get_ownership() hidden inside of
io_poll_can_finish_inline() whenever we want to complete arming inline.
For the same reason we can't just set REQ_F_DOUBLE_POLL in
__io_queue_proc() and so need to sync with the first poll entry by
taking its wq head lock.

Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Link: https://lore.kernel.org/r/8825315d7f5e182ac1578a031e546f79b1c97d01.1655990418.git.asml.silence@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2022-07-24 18:39:16 -06:00
..
advise.c io_uring: kill extra io_uring_types.h includes 2022-07-24 18:39:14 -06:00
advise.h io_uring: split out fadvise/madvise operations 2022-07-24 18:39:11 -06:00
cancel.c io_uring: add sync cancelation API through io_uring_register() 2022-07-24 18:39:15 -06:00
cancel.h io_uring: add sync cancelation API through io_uring_register() 2022-07-24 18:39:15 -06:00
epoll.c io_uring: kill extra io_uring_types.h includes 2022-07-24 18:39:14 -06:00
epoll.h io_uring: move epoll handler to its own file 2022-07-24 18:39:11 -06:00
fdinfo.c io_uring: move a few private types to local headers 2022-07-24 18:39:14 -06:00
fdinfo.h io_uring: move fdinfo helpers to its own file 2022-07-24 18:39:12 -06:00
filetable.c io_uring: kill extra io_uring_types.h includes 2022-07-24 18:39:14 -06:00
filetable.h io_uring: dedup io_run_task_work 2022-07-24 18:39:15 -06:00
fs.c io_uring: kill extra io_uring_types.h includes 2022-07-24 18:39:14 -06:00
fs.h io_uring: split out filesystem related operations 2022-07-24 18:39:11 -06:00
io_uring.c io_uring: add sync cancelation API through io_uring_register() 2022-07-24 18:39:15 -06:00
io_uring.h io_uring: remove priority tw list optimisation 2022-07-24 18:39:15 -06:00
io-wq.c io_uring: dedup io_run_task_work 2022-07-24 18:39:15 -06:00
io-wq.h io_uring: move list helpers to a separate file 2022-07-24 18:39:15 -06:00
kbuf.c io_uring: kbuf: kill __io_kbuf_recycle() 2022-07-24 18:39:15 -06:00
kbuf.h io_uring: kbuf: kill __io_kbuf_recycle() 2022-07-24 18:39:15 -06:00
Makefile io_uring: move opcode table to opdef.c 2022-07-24 18:39:12 -06:00
msg_ring.c io_uring: kill extra io_uring_types.h includes 2022-07-24 18:39:14 -06:00
msg_ring.h io_uring: move msg_ring into its own file 2022-07-24 18:39:12 -06:00
net.c io_uring: kill extra io_uring_types.h includes 2022-07-24 18:39:14 -06:00
net.h io_uring: split network related opcodes into its own file 2022-07-24 18:39:11 -06:00
nop.c io_uring: kill extra io_uring_types.h includes 2022-07-24 18:39:14 -06:00
nop.h io_uring: move nop into its own file 2022-07-24 18:39:11 -06:00
opdef.c io_uring: kill extra io_uring_types.h includes 2022-07-24 18:39:14 -06:00
opdef.h io_uring: move opcode table to opdef.c 2022-07-24 18:39:12 -06:00
openclose.c io_uring: kill extra io_uring_types.h includes 2022-07-24 18:39:14 -06:00
openclose.h io_uring: split out open/close operations 2022-07-24 18:39:11 -06:00
poll.c io_uring: optimise submission side poll_refs 2022-07-24 18:39:16 -06:00
poll.h io_uring: move a few private types to local headers 2022-07-24 18:39:14 -06:00
refs.h io_uring: make io_uring_types.h public 2022-07-24 18:39:14 -06:00
rsrc.c io_uring: move io_import_fixed() 2022-07-24 18:39:15 -06:00
rsrc.h io_uring: move io_import_fixed() 2022-07-24 18:39:15 -06:00
rw.c io_uring: remove priority tw list optimisation 2022-07-24 18:39:15 -06:00
rw.h io_uring: move read/write related opcodes to its own file 2022-07-24 18:39:12 -06:00
slist.h io_uring: move list helpers to a separate file 2022-07-24 18:39:15 -06:00
splice.c io_uring: kill extra io_uring_types.h includes 2022-07-24 18:39:14 -06:00
splice.h io_uring: split out splice related operations 2022-07-24 18:39:11 -06:00
sqpoll.c io_uring: kill extra io_uring_types.h includes 2022-07-24 18:39:14 -06:00
sqpoll.h io_uring: move SQPOLL related handling into its own file 2022-07-24 18:39:12 -06:00
statx.c io_uring: kill extra io_uring_types.h includes 2022-07-24 18:39:14 -06:00
statx.h io_uring: move statx handling to its own file 2022-07-24 18:39:11 -06:00
sync.c io_uring: kill extra io_uring_types.h includes 2022-07-24 18:39:14 -06:00
sync.h io_uring: split out fs related sync/fallocate functions 2022-07-24 18:39:11 -06:00
tctx.c io_uring: lockless task list 2022-07-24 18:39:15 -06:00
tctx.h io_uring: lockless task list 2022-07-24 18:39:15 -06:00
timeout.c io_uring: have cancelation API accept io_uring_task directly 2022-07-24 18:39:15 -06:00
timeout.h io_uring: move a few private types to local headers 2022-07-24 18:39:14 -06:00
uring_cmd.c io_uring: kill extra io_uring_types.h includes 2022-07-24 18:39:14 -06:00
uring_cmd.h io_uring: move uring_cmd handling to its own file 2022-07-24 18:39:11 -06:00
xattr.c io_uring: kill extra io_uring_types.h includes 2022-07-24 18:39:14 -06:00
xattr.h io_uring: move xattr related opcodes to its own file 2022-07-24 18:39:11 -06:00