linux/net/ipv6
Craig Gallek e32ea7e747 soreuseport: fast reuseport UDP socket selection
Include a struct sock_reuseport instance when a UDP socket binds to
a specific address for the first time with the reuseport flag set.
When selecting a socket for an incoming UDP packet, use the information
available in sock_reuseport if present.

This required adding an additional field to the UDP source address
equality function to differentiate between exact and wildcard matches.
The original use case allowed wildcard matches when checking for
existing port uses during bind.  The new use case of adding a socket
to a reuseport group requires exact address matching.

Performance test (using a machine with 2 CPU sockets and a total of
48 cores):  Create reuseport groups of varying size.  Use one socket
from this group per user thread (pinning each thread to a different
core) calling recvmmsg in a tight loop.  Record number of messages
received per second while saturating a 10G link.
  10 sockets: 18% increase (~2.8M -> 3.3M pkts/s)
  20 sockets: 14% increase (~2.9M -> 3.3M pkts/s)
  40 sockets: 13% increase (~3.0M -> 3.4M pkts/s)

This work is based off a similar implementation written by
Ying Cai <ycai@google.com> for implementing policy-based reuseport
selection.

Signed-off-by: Craig Gallek <kraig@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2016-01-04 22:49:58 -05:00
..
ila ila: Add generic ILA translation facility 2015-12-15 23:25:20 -05:00
netfilter Merge git://git.kernel.org/pub/scm/linux/kernel/git/pablo/nf-next 2015-12-18 15:37:42 -05:00
addrconf_core.c ipv6: change ipv6_stub_impl.ipv6_dst_lookup to take net argument 2015-07-31 15:21:30 -07:00
addrconf.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2015-12-31 18:20:10 -05:00
addrlabel.c ipv6/addrlabel: fix ip6addrlbl_get() 2015-12-22 15:57:54 -05:00
af_inet6.c net: add validation for the socket syscall protocol argument 2015-12-14 16:09:30 -05:00
ah6.c ah6: fix error return code 2015-08-25 13:37:31 -07:00
anycast.c ipv6: coding style: comparison for equality with NULL 2015-03-31 13:51:54 -04:00
datagram.c ipv6: add complete rcu protection around np->opt 2015-12-02 23:37:16 -05:00
esp6.c esp6: Switch to new AEAD interface 2015-05-28 11:23:20 +08:00
exthdrs_core.c ipv6: coding style: comparison for equality with NULL 2015-03-31 13:51:54 -04:00
exthdrs_offload.c ipv6: fix exthdrs offload registration in out_rt path 2015-09-02 15:31:00 -07:00
exthdrs.c ipv6: add complete rcu protection around np->opt 2015-12-02 23:37:16 -05:00
fib6_rules.c ipv6: fix the incorrect return value of throw route 2015-10-23 02:38:18 -07:00
icmp.c ipv6: kill sk_dst_lock 2015-12-03 11:32:06 -05:00
inet6_connection_sock.c soreuseport: fast reuseport UDP socket selection 2016-01-04 22:49:58 -05:00
inet6_hashtables.c net: SO_INCOMING_CPU setsockopt() support 2015-10-12 19:28:20 -07:00
ip6_checksum.c udp: Generic functions to set checksum 2014-06-04 22:46:38 -07:00
ip6_fib.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2015-10-24 06:54:12 -07:00
ip6_flowlabel.c ipv6: Flow label state ranges 2015-05-03 21:58:01 -04:00
ip6_gre.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2015-12-17 22:08:28 -05:00
ip6_icmp.c ipv6: White-space cleansing : Line Layouts 2014-08-24 22:37:52 -07:00
ip6_input.c netfilter: Pass net into okfn 2015-09-17 17:18:37 -07:00
ip6_offload.c ipv6: gro: support sit protocol 2015-10-21 19:36:11 -07:00
ip6_offload.h
ip6_output.c net: Eliminate NETIF_F_GEN_CSUM and NETIF_F_V[46]_CSUM 2015-12-15 16:50:20 -05:00
ip6_tunnel.c net/ip6_tunnel: fix dst leak 2015-11-18 16:25:01 -05:00
ip6_udp_tunnel.c vxlan: do not receive IPv4 packets on IPv6 socket 2015-08-29 13:07:54 -07:00
ip6_vti.c net: Pass net into dst_output and remove dst_output_okfn 2015-10-08 04:26:54 -07:00
ip6mr.c net: ipmr, ip6mr: fix vif/tunnel failure race condition 2015-11-24 17:15:56 -05:00
ipcomp6.c ipv6: White-space cleansing : Structure layouts 2014-08-24 22:37:52 -07:00
ipv6_sockglue.c ipv6: add complete rcu protection around np->opt 2015-12-02 23:37:16 -05:00
Kconfig ila: add NETFILTER dependency 2015-12-18 14:19:28 -05:00
Makefile ila: Create net/ipv6/ila directory 2015-12-15 23:25:20 -05:00
mcast_snoop.c net: fix wrong skb_get() usage / crash in IGMP/MLD parsing code 2015-08-13 17:08:39 -07:00
mcast.c snmp: Remove duplicate OUTMCAST stat increment 2015-11-16 16:36:32 -05:00
mip6.c ipv6: use ktime_t for internal timestamps 2015-10-05 03:16:47 -07:00
ndisc.c ipv6: honor ifindex in case we receive ll addresses in router advertisements 2015-12-23 22:03:54 -05:00
netfilter.c ipv6: Pass struct net into ip6_route_me_harder 2015-09-29 20:21:32 +02:00
output_core.c ipv4, ipv6: Pass net into ip_local_out and ip6_local_out 2015-10-08 04:27:02 -07:00
ping.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2015-03-09 23:38:02 -04:00
proc.c udp: Increment UDP_MIB_IGNOREDMULTI for arriving unmatched multicasts 2014-11-07 15:45:50 -05:00
protocol.c net: Export inet_offloads and inet6_offloads 2014-09-19 17:15:31 -04:00
raw.c ipv6: add IPV6_HDRINCL option for raw sockets 2015-12-17 15:12:28 -05:00
reassembly.c ipv6: distinguish frag queues by device for multicast and link-local packets 2015-11-24 16:45:47 -05:00
route.c ipv6: allow routes to be configured with expire values 2015-12-17 15:08:51 -05:00
sit.c ip_tunnel: Move stats update to iptunnel_xmit() 2015-12-25 23:32:23 -05:00
syncookies.c net: Allow accepted sockets to be bound to l3mdev domain 2015-12-18 14:43:38 -05:00
sysctl_net_ipv6.c ipv6: Implement different admin modes for automatic flow labels 2015-07-31 17:07:11 -07:00
tcp_ipv6.c tcp: honour SO_BINDTODEVICE for TW_RST case too 2015-12-22 17:03:05 -05:00
tcpv6_offload.c tcp: cleanup static functions 2015-02-28 16:56:51 -05:00
tunnel6.c ipv6: fix tunnel error handling 2015-11-03 10:52:13 -05:00
udp_impl.h net: Remove iocb argument from sendmsg and recvmsg 2015-03-02 13:06:31 -05:00
udp_offload.c ipv6: hash net ptr into fragmentation bucket selection 2015-03-25 14:07:04 -04:00
udp.c soreuseport: fast reuseport UDP socket selection 2016-01-04 22:49:58 -05:00
udplite.c net: Eliminate no_check from protosw 2014-05-23 16:28:53 -04:00
xfrm6_input.c netfilter: Pass struct net into the netfilter hooks 2015-09-17 17:18:37 -07:00
xfrm6_mode_beet.c xfrm: simplify xfrm_address_t use 2015-03-31 13:58:35 -04:00
xfrm6_mode_ro.c ipv4/ipv6: Fix FSF address in file headers 2013-12-06 12:37:56 -05:00
xfrm6_mode_transport.c
xfrm6_mode_tunnel.c xfrm6: Fix IPv6 ECN decapsulation 2015-08-11 12:41:34 +02:00
xfrm6_output.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2015-10-24 06:54:12 -07:00
xfrm6_policy.c Merge branch 'master' of git://git.kernel.org/pub/scm/linux/kernel/git/klassert/ipsec 2015-12-22 16:26:31 -05:00
xfrm6_protocol.c xfrm6: Properly handle unsupported protocols 2014-05-06 07:08:38 +02:00
xfrm6_state.c ipv6: White-space cleansing : Line Layouts 2014-08-24 22:37:52 -07:00
xfrm6_tunnel.c ipv6: White-space cleansing : gaps between function and symbol export 2014-08-24 22:37:52 -07:00