Minki/linux
1
0
Fork 1
mirror of https://github.com/torvalds/linux.git synced 2024-11-20 02:51:44 +00:00
Code Issues Packages Projects Releases Wiki Activity
79640a4ca6
linux/include/net
History
Eric Dumazet 79640a4ca6 net: add additional lock to qdisc to increase throughput 
When many cpus compete for sending frames on a given qdisc, the qdisc
spinlock suffers from very high contention.

The cpu owning __QDISC_STATE_RUNNING bit has same priority to acquire
the lock, and cannot dequeue packets fast enough, since it must wait for
this lock for each dequeued packet.

One solution to this problem is to force all cpus spinning on a second
lock before trying to get the main lock, when/if they see
__QDISC_STATE_RUNNING already set.

The owning cpu then compete with at most one other cpu for the main
lock, allowing for higher dequeueing rate.

Based on a previous patch from Alexander Duyck. I added the heuristic to
avoid the atomic in fast path, and put the new lock far away from the
cache line used by the dequeue worker. Also try to release the busylock
lock as late as possible.

Tests with following script gave a boost from ~50.000 pps to ~600.000
pps on a dual quad core machine (E5450 @3.00GHz), tg3 driver.
(A single netperf flow can reach ~800.000 pps on this platform)

for j in `seq 0 3`; do
  for i in `seq 0 7`; do
    netperf -H 192.168.0.1 -t UDP_STREAM -l 60 -N -T $i -- -m 6 &
  done
done

Signed-off-by: Eric Dumazet <eric.dumazet@gmail.com>
Acked-by: Alexander Duyck <alexander.h.duyck@intel.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2010-06-02 05:09:29 -07:00
..
9p 9p: add 9P2000.L rename operation 2010-05-21 16:44:34 -05:00
bluetooth Bluetooth: Create per controller workqueue 2010-05-10 09:34:03 +02:00
caif caif: Bugfix - use standard Linux lists 2010-05-23 23:57:41 -07:00
irda tree-wide: fix typos "ass?o[sc]iac?te" -> "associate" in comments 2010-02-15 15:38:10 +01:00
iucv include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h 2010-03-30 22:02:32 +09:00
netfilter netfilter: nf_conntrack: fix a race in __nf_conntrack_confirm against nf_ct_get_next_corpse() 2010-05-20 15:55:30 +02:00
netns ipv6: ip6mr: support multiple tables 2010-05-11 14:40:55 +02:00
phonet Phonet: zero-copy GPRS TX 2010-01-07 00:24:55 -08:00
sctp sctp: dubious bitfields in sctp_transport 2010-05-26 00:40:11 -07:00
tc_act pkt_sched: skbedit add support for setting mark 2009-10-22 21:56:42 -07:00
tipc tipc: Update commenting in TIPC API 2010-05-12 23:02:23 -07:00
act_api.h net: restore gnet_stats_basic to previous definition 2009-08-17 21:33:49 -07:00
addrconf.h net: Add checking to rcu_dereference() primitives 2010-02-25 09:41:03 +01:00
af_ieee802154.h af_ieee802154: add support for WANT_ACK socket option 2009-08-12 21:54:50 -07:00
af_rxrpc.h
af_unix.h net: sock_def_readable() and friends RCU conversion 2010-05-01 15:00:15 -07:00
ah.h net: cleanup include/net 2009-11-04 05:06:25 -08:00
arp.h net: make neigh_ops constant 2009-09-01 17:40:57 -07:00
atmclip.h clip: convert to internal network_device_stats 2009-01-21 14:01:59 -08:00
ax25.h include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h 2010-03-30 22:02:32 +09:00
ax88796.h ax88796: Add method to take MAC from platform data 2009-03-24 23:32:03 -07:00
cfg80211.h cfg80211/mac80211: better channel handling 2010-05-07 14:55:50 -04:00
checksum.h include/net net/ - csum_partial - remove unnecessary casts 2008-11-19 15:44:53 -08:00
cipso_ipv4.h netlabel: Label incoming TCP connections correctly in SELinux 2009-03-28 15:01:36 +11:00
cls_cgroup.h cls_cgroup: Initialise classid when module is absent 2010-05-25 18:53:57 -07:00
compat.h net: fix compat_sys_recvmmsg parameter type 2009-12-11 15:07:56 -08:00
datalink.h
dcbnl.h dcbnl: Add support for setapp/getapp to netdev dcbnl_rtnl_ops 2009-09-01 01:24:30 -07:00
dn_dev.h Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next-2.6 2009-12-08 07:55:01 -08:00
dn_fib.h decnet: Remove unused FIB metric macros. 2010-03-27 19:23:46 -07:00
dn_neigh.h
dn_nsp.h net: cleanup include/net 2009-11-04 05:06:25 -08:00
dn_route.h
dn.h decnet: compile fix for removal of byteorder wrapper 2008-11-27 23:04:13 -08:00
dsa.h dsa: add switch chip cascading support 2009-03-21 19:06:54 -07:00
dsfield.h
dst_ops.h netns: embed ip6_dst_ops directly 2009-09-01 17:40:31 -07:00
dst.h net: Introduce skb_tunnel_rx() helper 2010-05-17 22:36:55 -07:00
esp.h
ethoc.h net: Add support for the OpenCores 10/100 Mbps Ethernet MAC. 2009-03-27 00:16:21 -07:00
fib_rules.h net: fib_rules: mark arguments to fib_rules_register const and __net_initdata 2010-04-26 16:02:04 +02:00
flow.h flow: virtualize flow cache entry methods 2010-04-07 03:43:18 -07:00
garp.h
gen_stats.h net: cleanup include/net 2009-11-04 05:06:25 -08:00
genetlink.h net: cleanup include/net 2009-11-04 05:06:25 -08:00
icmp.h ipv4: raw: move struct raw_sock and raw_sk() to include/net/raw.h 2010-04-13 14:49:31 -07:00
ieee80211_radiotap.h wireless: update radiotap parser 2010-02-08 16:50:53 -05:00
ieee802154_netdev.h ieee802154: add an mlme_ops call to retrieve PHY object 2009-11-06 14:32:18 +03:00
ieee802154.h ieee802154: move headers out of extra directory 2009-07-23 17:08:51 +04:00
if_inet6.h ipv6: Replace inet6_ifaddr->dead with state 2010-05-18 15:36:06 -07:00
inet6_connection_sock.h net: replace ipfragok with skb->local_df 2010-04-15 23:36:37 -07:00
inet6_hashtables.h tcp: Fix a connect() race with timewait sockets 2009-12-08 20:17:51 -08:00
inet_common.h
inet_connection_sock.h net: replace ipfragok with skb->local_df 2010-04-15 23:36:37 -07:00
inet_ecn.h net: replace __constant_{endian} uses in net headers 2009-02-14 22:58:35 -08:00
inet_frag.h inet fragments: fix sparse warning: context imbalance 2009-02-26 23:13:35 -08:00
inet_hashtables.h tcp: Fix a connect() race with timewait sockets 2009-12-08 20:17:51 -08:00
inet_sock.h net: Make RFS socket operations not be inet specific. 2010-04-27 15:11:48 -07:00
inet_timewait_sock.h net: suppress RCU lockdep false positive in twsk_net() 2010-04-27 12:39:01 -07:00
inetpeer.h inetpeer: Optimize inet_getid() 2009-11-13 20:46:58 -08:00
ip6_checksum.h
ip6_fib.h ipv6 fib: Make rt6_info{} more cache-line aware. 2010-04-01 18:41:41 -07:00
ip6_route.h net: sk_dst_cache RCUification 2010-04-13 01:41:33 -07:00
ip6_tunnel.h ipv6 ip6_tunnel: eliminate unused recursion field from ip6_tnl{}. 2010-03-10 07:32:29 -08:00
ip_fib.h Merge branch 'master' of master.kernel.org:/pub/scm/linux/kernel/git/davem/net-2.6 2009-11-06 00:55:55 -08:00
ip_vs.h ipvs: SCTP Trasport Loadbalancing Support 2010-02-18 12:31:05 +01:00
ip.h kernel-wide: replace USHORT_MAX, SHORT_MAX and SHORT_MIN with USHRT_MAX, SHRT_MAX and SHRT_MIN 2010-05-25 08:07:02 -07:00
ipcomp.h percpu: add __percpu sparse annotations to net 2010-02-16 23:05:38 -08:00
ipconfig.h
ipip.h net: cleanup include/net 2009-11-04 05:06:25 -08:00
ipv6.h kernel-wide: replace USHORT_MAX, SHORT_MAX and SHORT_MIN with USHRT_MAX, SHRT_MAX and SHRT_MIN 2010-05-25 08:07:02 -07:00
ipx.h include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h 2010-03-30 22:02:32 +09:00
iw_handler.h include/net/iw_handler.h: Use SIOCIWFIRST not SIOCSIWCOMMIT in comment 2010-03-31 14:49:12 -04:00
lapb.h
lib80211.h wireless: missing include in lib80211.h 2008-11-21 11:42:55 -05:00
llc_c_ac.h
llc_c_ev.h
llc_c_st.h
llc_conn.h llc: use a device based hash table to speed up multicast delivery 2009-12-26 20:43:57 -08:00
llc_if.h
llc_pdu.h
llc_s_ac.h
llc_s_ev.h
llc_s_st.h
llc_sap.h
llc.h llc: convert llc_sap_list to RCU 2009-12-26 20:46:28 -08:00
mac80211.h wireless: fix mac80211.h kernel-doc warnings 2010-05-24 15:07:42 -04:00
mip6.h
mld.h ipv6 mcast: Introduce include/net/mld.h for MLD definitions. 2010-04-23 13:35:55 +09:00
ndisc.h sysctl: remove "struct file *" argument of ->proc_handler 2009-09-24 07:21:04 -07:00
neighbour.h netfilter: bridge-netfilter: Fix MAC header handling with IP DNAT 2010-04-15 12:26:39 +02:00
net_namespace.h nsproxy: remove INIT_NSPROXY() 2010-03-12 15:52:40 -08:00
netdma.h net_dma: convert to dma_find_channel 2009-01-06 11:38:15 -07:00
netevent.h
netlabel.h include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h 2010-03-30 22:02:32 +09:00
netlink.h netlink: fix unaligned access in nla_get_be64() 2010-03-19 22:47:23 -07:00
netrom.h include cleanup: Update gfp.h and slab.h includes to prepare for breaking implicit slab.h inclusion from percpu.h 2010-03-30 22:02:32 +09:00
nexthop.h
nl802154.h ieee802154: add support for channel pages from IEEE 802.15.4-2006 2009-08-19 23:08:22 +04:00
p8022.h
pkt_cls.h net: rename skb->iif to skb->skb_iif 2009-11-20 15:35:04 -08:00
pkt_sched.h net: Define accessors to manipulate QDISC_STATE_RUNNING 2010-06-02 03:23:51 -07:00
protocol.h net: drop capability from protocol definitions 2009-11-05 21:40:17 -08:00
psnap.h snap: use const for descriptor 2009-03-21 19:06:50 -07:00
raw.h ipv4: ipmr: support multiple tables 2010-04-13 14:49:34 -07:00
rawv6.h ipv6: Use correct data types for ICMPv6 type and code 2009-06-23 04:31:07 -07:00
red.h net: cleanup include/net 2009-11-04 05:06:25 -08:00
regulatory.h cfg80211: add regulatory hint disconnect support 2010-02-01 15:40:06 -05:00
request_sock.h tcp: account SYN-ACK timeouts & retransmissions 2010-01-17 19:09:39 -08:00
rose.h NET: ROSE: Don't use static buffer. 2009-07-26 19:11:14 -07:00
route.h net: implements ip_route_input_noref() 2010-05-17 17:18:51 -07:00
rtnetlink.h rtnetlink: support specifying device flags on device creation 2010-02-27 02:43:40 -08:00
sch_generic.h net: add additional lock to qdisc to increase throughput 2010-06-02 05:09:29 -07:00
scm.h net: cleanup include/net 2009-11-04 05:06:25 -08:00
slhc_vj.h
snmp.h tcp: fix outsegs stat for TSO segments 2010-04-22 16:00:00 -07:00
sock.h net: fix lock_sock_bh/unlock_sock_bh 2010-05-27 00:30:53 -07:00
stp.h
tcp_states.h
tcp.h Merge branch 'master' of master.kernel.org:/pub/scm/linux/kernel/git/davem/net-2.6 2010-05-16 22:26:58 -07:00
timewait_sock.h net: Fix memory leak in the proto_register function 2008-11-21 16:45:22 -08:00
transp_v6.h IPv6: Add dontfrag argument to relevant functions 2010-04-23 23:35:28 -07:00
udp.h udp: bind() optimisation 2009-11-10 20:54:38 -08:00
udplite.h
wext.h wext: refactor 2009-10-07 16:39:43 -04:00
wimax.h Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/jikos/trivial 2009-12-09 19:43:33 -08:00
wpan-phy.h ieee802154: add support for creation/removal of logic interfaces 2009-11-06 14:32:24 +03:00
x25.h X25: Move accept approve flag to bitfield 2010-05-17 17:39:27 -07:00
x25device.h X25: Add if_x25.h and x25 to device identifiers 2010-04-22 16:12:36 -07:00
xfrm.h Merge branch 'master' of master.kernel.org:/pub/scm/linux/kernel/git/davem/net-2.6 2010-04-11 14:53:53 -07:00