linux/net/ipv4
Mina Almasry 8f0b3cc9a4 tcp: RX path for devmem TCP
In tcp_recvmsg_locked(), detect if the skb being received by the user
is a devmem skb. In this case - if the user provided the MSG_SOCK_DEVMEM
flag - pass it to tcp_recvmsg_devmem() for custom handling.

tcp_recvmsg_devmem() copies any data in the skb header to the linear
buffer, and returns a cmsg to the user indicating the number of bytes
returned in the linear buffer.

tcp_recvmsg_devmem() then loops over the unaccessible devmem skb frags,
and returns to the user a cmsg_devmem indicating the location of the
data in the dmabuf device memory. cmsg_devmem contains this information:

1. the offset into the dmabuf where the payload starts. 'frag_offset'.
2. the size of the frag. 'frag_size'.
3. an opaque token 'frag_token' to return to the kernel when the buffer
is to be released.

The pages awaiting freeing are stored in the newly added
sk->sk_user_frags, and each page passed to userspace is get_page()'d.
This reference is dropped once the userspace indicates that it is
done reading this page.  All pages are released when the socket is
destroyed.

Signed-off-by: Willem de Bruijn <willemb@google.com>
Signed-off-by: Kaiyuan Zhang <kaiyuanz@google.com>
Signed-off-by: Mina Almasry <almasrymina@google.com>
Reviewed-by: Pavel Begunkov <asml.silence@gmail.com>
Reviewed-by: Eric Dumazet <edumazet@google.com>
Link: https://patch.msgid.link/20240910171458.219195-10-almasrymina@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2024-09-11 20:44:32 -07:00
..
netfilter netfilter: nf_dup4: Unmask upper DSCP bits in nf_dup_ipv4_route() 2024-09-09 14:14:53 +01:00
af_inet.c net: gro: initialize network_offset in network layer 2024-05-27 16:46:59 -07:00
ah4.c net: fill in MODULE_DESCRIPTION()s for ipv4 modules 2024-02-09 14:12:02 -08:00
arp.c arp: Convert ioctl(SIOCGARP) to RCU. 2024-05-01 18:37:07 -07:00
bpf_tcp_ca.c bpf: pass bpf_struct_ops_link to callbacks in bpf_struct_ops. 2024-05-30 15:34:13 -07:00
cipso_ipv4.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net 2024-06-20 13:49:59 -07:00
datagram.c ipv4: Set the routing scope properly in ip_route_output_ports(). 2024-02-12 17:33:05 -08:00
devinet.c ipv4: delete redundant judgment statements 2024-08-23 14:27:45 +01:00
esp4_offload.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net 2024-07-15 13:19:17 -07:00
esp4.c net: support non paged skb frags 2024-09-11 20:44:31 -07:00
fib_frontend.c ipv4: Unmask upper DSCP bits in fib_compute_spec_dst() 2024-08-22 16:59:57 -07:00
fib_lookup.h
fib_notifier.c
fib_rules.c ipv4: Centralize TOS matching 2024-08-20 14:57:08 +02:00
fib_semantics.c ipv4: delete redundant judgment statements 2024-08-23 14:27:45 +01:00
fib_trie.c ipv4: Centralize TOS matching 2024-08-20 14:57:08 +02:00
fou_bpf.c ip_tunnel: convert __be16 tunnel flags to bitmaps 2024-04-01 10:49:28 +01:00
fou_core.c fou: Fix null-ptr-deref in GRO. 2024-09-04 16:31:00 -07:00
fou_nl.c
fou_nl.h
gre_demux.c ip_tunnel: convert __be16 tunnel flags to bitmaps 2024-04-01 10:49:28 +01:00
gre_offload.c net: gro: rename skb_gro_header_hard() 2024-03-05 13:30:11 +01:00
icmp.c ipv4: icmp: Unmask upper DSCP bits in icmp_reply() 2024-09-09 14:14:53 +01:00
igmp.c ipv4: Set scope explicitly in ip_route_output(). 2024-04-08 13:20:51 +01:00
inet_connection_sock.c net/ipv4: fix macro definition sk_for_each_bound_bhash 2024-08-26 14:04:25 -07:00
inet_diag.c tcp: remove volatile qualifier on tw_substate 2024-08-28 17:08:16 -07:00
inet_fragment.c net: Rename mono_delivery_time to tstamp_type for scalabilty 2024-05-23 14:14:23 -07:00
inet_hashtables.c inet: constify 'struct net' parameter of various lookup helpers 2024-08-05 16:22:45 -07:00
inet_timewait_sock.c tcp: move inet_twsk_schedule helper out of header 2024-06-10 11:54:18 +01:00
inetpeer.c net: ipv4: Simplify the allocation of slab caches in inet_initpeers 2024-01-31 16:39:42 -08:00
ip_forward.c net: fix IPSTATS_MIB_OUTFORWDATAGRAMS increment after fragment check 2023-10-13 09:58:45 -07:00
ip_fragment.c net: Rename mono_delivery_time to tstamp_type for scalabilty 2024-05-23 14:14:23 -07:00
ip_gre.c ipv4: ip_gre: Unmask upper DSCP bits in ipgre_open() 2024-09-09 14:14:52 +01:00
ip_input.c net/ipv4: make use of the helper macro LIST_HEAD() 2024-09-06 18:10:21 -07:00
ip_options.c
ip_output.c ipv4: Unmask upper DSCP bits in __ip_queue_xmit() 2024-09-04 16:57:11 -07:00
ip_sockglue.c inet: Add getsockopt support for IP_ROUTER_ALERT and IPV6_ROUTER_ALERT 2024-03-06 12:37:06 +00:00
ip_tunnel_core.c ip_tunnel: convert __be16 tunnel flags to bitmaps 2024-04-01 10:49:28 +01:00
ip_tunnel.c ipv4: ip_tunnel: Unmask upper DSCP bits in ip_tunnel_xmit() 2024-09-09 14:14:53 +01:00
ip_vti.c netdev_features: convert NETIF_F_LLTX to dev->lltx 2024-09-03 11:36:43 +02:00
ipcomp.c
ipconfig.c
ipip.c netdev_features: convert NETIF_F_LLTX to dev->lltx 2024-09-03 11:36:43 +02:00
ipmr_base.c
ipmr.c ipv4: ipmr: Unmask upper DSCP bits in ipmr_queue_xmit() 2024-09-04 16:57:11 -07:00
Kconfig net/tcp: Expand goo.gl link 2024-07-30 18:35:12 -07:00
Makefile bpfilter: remove bpfilter 2024-01-04 10:23:10 -08:00
metrics.c net: remove NULL-pointer net parameter in ip_metrics_convert 2024-06-05 10:06:00 +01:00
netfilter.c ipv4: netfilter: Unmask upper DSCP bits in ip_route_me_harder() 2024-09-09 14:14:53 +01:00
netlink.c
nexthop.c net: nexthop: delete redundant judgment statements 2024-08-23 14:27:45 +01:00
ping.c ping: use sk_skb_reason_drop to free rx packets 2024-06-19 12:44:22 +01:00
proc.c minmax: add a few more MIN_T/MAX_T users 2024-07-28 13:41:14 -07:00
protocol.c
raw_diag.c inet_diag: add module pointer to "struct inet_diag_handler" 2024-01-23 15:13:54 +01:00
raw.c net: raw: use sk_skb_reason_drop to free rx packets 2024-06-19 12:44:22 +01:00
route.c ipv4: Unmask upper DSCP bits when building flow key 2024-08-31 17:44:51 +01:00
syncookies.c tcp: use sk_skb_reason_drop to free rx packets 2024-06-19 12:44:22 +01:00
sysctl_net_ipv4.c icmp: icmp_msgs_per_sec and icmp_msgs_burst sysctls become per netns 2024-08-30 11:14:06 -07:00
tcp_ao.c net/tcp: Disable TCP-AO static key after RCU grace period 2024-08-04 13:21:50 +01:00
tcp_bbr.c tcp: Add new args for cong_control in tcp_congestion_ops 2024-05-02 16:26:56 -07:00
tcp_bic.c
tcp_bpf.c tcp_bpf: fix return value of tcp_bpf_sendmsg() 2024-08-30 11:09:10 -07:00
tcp_cdg.c
tcp_cong.c tcp: Replace strncpy() with strscpy() 2024-07-16 07:52:15 -07:00
tcp_cubic.c bpf: Remove CONFIG_X86 and CONFIG_DYNAMIC_FTRACE guard from the tcp-cc kfuncs 2024-03-28 18:31:40 -07:00
tcp_dctcp.c tcp: Fix shift-out-of-bounds in dctcp_update_alpha(). 2024-05-21 13:34:50 +02:00
tcp_dctcp.h
tcp_diag.c inet_diag: add module pointer to "struct inet_diag_handler" 2024-01-23 15:13:54 +01:00
tcp_fastopen.c net: use unrcu_pointer() helper 2024-06-06 11:52:52 +02:00
tcp_highspeed.c
tcp_htcp.c tcp: Use clamp() in htcp_alpha_update() 2024-08-06 12:16:25 -07:00
tcp_hybla.c
tcp_illinois.c
tcp_input.c net: add support for skbs with unreadable frags 2024-09-11 20:44:31 -07:00
tcp_ipv4.c tcp: RX path for devmem TCP 2024-09-11 20:44:32 -07:00
tcp_lp.c tcp: rename tcp_time_stamp() to tcp_time_stamp_ts() 2023-10-23 09:35:01 +01:00
tcp_metrics.c tcp_metrics: use netlink policy for IPv6 addr len validation 2024-08-19 17:42:57 -07:00
tcp_minisocks.c tcp: RX path for devmem TCP 2024-09-11 20:44:32 -07:00
tcp_nv.c
tcp_offload.c net: drop bad gso csum_start and offset in virtio_net_hdr 2024-07-30 18:34:13 -07:00
tcp_output.c net: add support for skbs with unreadable frags 2024-09-11 20:44:31 -07:00
tcp_plb.c
tcp_rate.c
tcp_recovery.c tcp: fix excessive TLP and RACK timeouts from HZ rounding 2023-10-17 17:25:42 -07:00
tcp_scalable.c
tcp_sigpool.c net/tcp_sigpool: Use nested-BH locking for sigpool_scratch. 2024-06-24 16:41:22 -07:00
tcp_timer.c mptcp: fallback to TCP after SYN+MPC drops 2024-09-11 15:57:50 -07:00
tcp_ulp.c
tcp_vegas.c
tcp_vegas.h
tcp_veno.c
tcp_westwood.c
tcp_yeah.c
tcp.c tcp: RX path for devmem TCP 2024-09-11 20:44:32 -07:00
tunnel4.c net: fill in MODULE_DESCRIPTION()s for ipv4 modules 2024-02-09 14:12:02 -08:00
udp_bpf.c
udp_diag.c inet_diag: add module pointer to "struct inet_diag_handler" 2024-01-23 15:13:54 +01:00
udp_impl.h
udp_offload.c udp: fix receiving fraglist GSO packets 2024-08-21 17:15:05 -07:00
udp_tunnel_core.c ipv4: udp_tunnel: Unmask upper DSCP bits in udp_tunnel_dst_lookup() 2024-09-09 14:14:53 +01:00
udp_tunnel_nic.c udp_tunnel: Use flex array to simplify code 2023-10-03 11:39:34 +02:00
udp_tunnel_stub.c
udp.c ipv4: udp: Unmask upper DSCP bits during early demux 2024-08-22 16:59:58 -07:00
udplite.c
xfrm4_input.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net 2024-05-09 10:01:01 -07:00
xfrm4_output.c
xfrm4_policy.c net: ipv{6,4}: Remove the now superfluous sentinel elements from ctl_table array 2024-05-03 13:29:42 +01:00
xfrm4_protocol.c
xfrm4_state.c
xfrm4_tunnel.c net: fill in MODULE_DESCRIPTION()s for ipv4 modules 2024-02-09 14:12:02 -08:00