linux/io_uring
Pavel Begunkov a26a35e901 io_uring: make poll refs more robust
poll_refs carry two functions, the first is ownership over the request.
The second is notifying the io_poll_check_events() that there was an
event but wake up couldn't grab the ownership, so io_poll_check_events()
should retry.

We want to make poll_refs more robust against overflows. Instead of
always incrementing it, which covers two purposes with one atomic, check
if poll_refs is elevated enough and if so set a retry flag without
attempts to grab ownership. The gap between the bias check and following
atomics may seem racy, but we don't need it to be strict. Moreover there
might only be maximum 4 parallel updates: by the first and the second
poll entries, __io_arm_poll_handler() and cancellation. From those four,
only poll wake ups may be executed multiple times, but they're protected
by a spin.

Cc: stable@vger.kernel.org
Reported-by: Lin Ma <linma@zju.edu.cn>
Fixes: aa43477b04 ("io_uring: poll rework")
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Link: https://lore.kernel.org/r/c762bc31f8683b3270f3587691348a7119ef9c9d.1668963050.git.asml.silence@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
2022-11-25 06:54:46 -07:00
..
advise.c io_uring: make io_kiocb_to_cmd() typesafe 2022-08-12 17:01:00 -06:00
advise.h
alloc_cache.h
cancel.c io_uring: add IORING_SETUP_DEFER_TASKRUN 2022-09-21 10:30:42 -06:00
cancel.h
epoll.c io_uring: make io_kiocb_to_cmd() typesafe 2022-08-12 17:01:00 -06:00
epoll.h
fdinfo.c io_uring: fix fdinfo sqe offsets calculation 2022-10-12 16:30:56 -06:00
fdinfo.h
filetable.c
filetable.h io_uring: kill hot path fixed file bitmap debug checks 2022-10-16 17:07:53 -06:00
fs.c io_uring: make io_kiocb_to_cmd() typesafe 2022-08-12 17:01:00 -06:00
fs.h
io_uring.c io_uring: fix multishot accept request leaks 2022-11-17 12:33:33 -07:00
io_uring.h io_uring: fix multishot accept request leaks 2022-11-17 12:33:33 -07:00
io-wq.c io-wq: Fix memory leak in worker creation 2022-10-20 05:48:59 -07:00
io-wq.h
kbuf.c io_uring: check for rollover of buffer ID when providing buffers 2022-11-10 11:07:41 -07:00
kbuf.h io_uring: allow buffer recycling in READV 2022-09-21 10:30:43 -06:00
Makefile
msg_ring.c io_uring/msg_ring: Fix NULL pointer dereference in io_msg_send_fd() 2022-10-19 12:33:33 -07:00
msg_ring.h
net.c io_uring: fix multishot recv request leaks 2022-11-17 12:33:33 -07:00
net.h io_uring/net: zerocopy sendmsg 2022-09-21 13:15:02 -06:00
nop.c
nop.h
notif.c io_uring/notif: Remove the unused function io_notif_complete() 2022-09-05 11:42:39 -06:00
notif.h io_uring/net: simplify zerocopy send user API 2022-09-01 09:13:33 -06:00
opdef.c io_uring/opdef: remove 'audit_skip' from SENDMSG_ZC 2022-10-12 16:30:56 -06:00
opdef.h io_uring: add custom opcode hooks on fail 2022-09-21 13:15:02 -06:00
openclose.c io_uring: make io_kiocb_to_cmd() typesafe 2022-08-12 17:01:00 -06:00
openclose.h
poll.c io_uring: make poll refs more robust 2022-11-25 06:54:46 -07:00
poll.h
refs.h
rsrc.c io_uring: remove FFS_SCM 2022-10-16 17:07:12 -06:00
rsrc.h io_uring: remove FFS_SCM 2022-10-16 17:07:12 -06:00
rw.c io_uring/rw: remove leftover debug statement 2022-10-16 17:24:10 -06:00
rw.h io_uring/rw: don't lose partial IO result on fail 2022-09-21 13:15:02 -06:00
slist.h
splice.c io_uring: make io_kiocb_to_cmd() typesafe 2022-08-12 17:01:00 -06:00
splice.h
sqpoll.c
sqpoll.h
statx.c io_uring: make io_kiocb_to_cmd() typesafe 2022-08-12 17:01:00 -06:00
statx.h
sync.c io_uring: make io_kiocb_to_cmd() typesafe 2022-08-12 17:01:00 -06:00
sync.h
tctx.c io_uring: remove io_register_submitter 2022-10-07 12:25:30 -06:00
tctx.h io_uring: simplify __io_uring_add_tctx_node 2022-10-07 12:25:30 -06:00
timeout.c io_uring: remove unused return from io_disarm_next 2022-09-21 13:15:01 -06:00
timeout.h io_uring: remove unused return from io_disarm_next 2022-09-21 13:15:01 -06:00
uring_cmd.c io_uring: introduce fixed buffer support for io_uring_cmd 2022-09-30 07:50:59 -06:00
uring_cmd.h
xattr.c __io_setxattr(): constify path 2022-09-01 17:39:05 -04:00
xattr.h