linux/net/netfilter
Liping Zhang 03e5fd0e9b netfilter: nft_set_rbtree: use per-set rwlock to improve the scalability
Karel Rericha reported that in his test case, ICMP packets going through
boxes had normally about 5ms latency. But when running nft, actually
listing the sets with interval flags, latency would go up to 30-100ms.
This was observed when router throughput is from 600Mbps to 2Gbps.

This is because we use a single global spinlock to protect the whole
rbtree sets, so "dumping sets" will race with the "key lookup" inevitably.
But actually they are all _readers_, so it's ok to convert the spinlock
to rwlock to avoid competition between them. Also use per-set rwlock since
each set is independent.

Reported-by: Karel Rericha <karel@unitednetworks.cz>
Tested-by: Karel Rericha <karel@unitednetworks.cz>
Signed-off-by: Liping Zhang <zlpnobody@gmail.com>
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
2017-03-13 19:30:43 +01:00
..
ipset netfilter: ipset: Null pointer exception in ipset list:set 2017-02-19 19:08:47 +01:00
ipvs lib/vsprintf.c: remove %Z support 2017-02-27 18:43:47 -08:00
core.c netfilter: merge ctinfo into nfct pointer storage area 2017-02-02 14:31:56 +01:00
Kconfig netfilter: nft_exthdr: add TCP option matching 2017-02-08 14:17:09 +01:00
Makefile netfilter: nf_tables: add bitmap set type 2017-02-08 14:16:21 +01:00
nf_conntrack_acct.c
nf_conntrack_amanda.c
nf_conntrack_broadcast.c
nf_conntrack_core.c netfilter: nf_conntrack: reduce resolve_normal_ct args 2017-03-13 19:30:20 +01:00
nf_conntrack_ecache.c netfilter: don't rely on DYING bit to detect when destroy event was sent 2016-08-30 11:43:08 +02:00
nf_conntrack_expect.c netfilter: nf_ct_expect: Change __nf_ct_expect_check() return value. 2017-02-26 17:06:59 +01:00
nf_conntrack_extend.c netfilter: move nat hlist_head to nf_conn 2016-07-11 11:47:50 +02:00
nf_conntrack_ftp.c lib/vsprintf.c: remove %Z support 2017-02-27 18:43:47 -08:00
nf_conntrack_h323_asn1.c netfilter: nf_conntrack_h323: fix off-by-one in DecodeQ931 2016-07-11 12:32:45 +02:00
nf_conntrack_h323_main.c netfilter: Remove explicit rcu_read_lock in nf_hook_slow 2016-09-24 21:29:53 +02:00
nf_conntrack_h323_types.c
nf_conntrack_helper.c netfilter: nf_ct_helper: warn when not applying default helper assignment 2017-02-06 12:03:35 +01:00
nf_conntrack_irc.c netfilter: Add helper array register/unregister functions 2016-07-21 02:31:53 +02:00
nf_conntrack_l3proto_generic.c
nf_conntrack_labels.c netfilter: connlabels: move set helper to xt_connlabel 2016-07-22 17:05:10 +02:00
nf_conntrack_netbios_ns.c
nf_conntrack_netlink.c netfilter: ctnetlink: Fix regression in CTA_HELP processing 2017-02-06 12:49:05 +01:00
nf_conntrack_pptp.c netfilter: conntrack: get rid of conntrack timer 2016-08-30 11:43:09 +02:00
nf_conntrack_proto_dccp.c netfilter: conntrack: no need to pass ctinfo to error handler 2017-02-02 14:31:51 +01:00
nf_conntrack_proto_generic.c netfilter: remove ip_conntrack* sysctl compat code 2016-08-13 13:27:13 +02:00
nf_conntrack_proto_gre.c netns: make struct pernet_operations::id unsigned int 2016-11-18 10:59:15 -05:00
nf_conntrack_proto_sctp.c netfilter: conntrack: no need to pass ctinfo to error handler 2017-02-02 14:31:51 +01:00
nf_conntrack_proto_tcp.c netfilter: conntrack: no need to pass ctinfo to error handler 2017-02-02 14:31:51 +01:00
nf_conntrack_proto_udp.c netfilter: conntrack: no need to pass ctinfo to error handler 2017-02-02 14:31:51 +01:00
nf_conntrack_proto.c netfilter: conntrack: add nf_conntrack_default_on sysctl 2016-12-04 21:17:25 +01:00
nf_conntrack_sane.c netfilter: Add helper array register/unregister functions 2016-07-21 02:31:53 +02:00
nf_conntrack_seqadj.c netfilter: seqadj: Fix the wrong ack adjust for the RST packet without ack 2016-09-25 14:54:01 +02:00
nf_conntrack_sip.c netfilter: nf_conntrack_sip: fix wrong memory initialisation 2017-03-03 13:48:31 +01:00
nf_conntrack_snmp.c
nf_conntrack_standalone.c netfilter: merge ctinfo into nfct pointer storage area 2017-02-02 14:31:56 +01:00
nf_conntrack_tftp.c netfilter: Add helper array register/unregister functions 2016-07-21 02:31:53 +02:00
nf_conntrack_timeout.c
nf_conntrack_timestamp.c
nf_dup_netdev.c netfilter: add and use nf_fwd_netdev_egress 2016-12-06 21:48:22 +01:00
nf_internals.h netfilter: merge nf_iterate() into nf_hook_slow() 2016-11-03 11:52:59 +01:00
nf_log_common.c netfilter: nf_log: do not assume ethernet header in netdev family 2016-12-04 20:45:33 +01:00
nf_log_netdev.c netfilter: nf_log: do not assume ethernet header in netdev family 2016-12-04 20:45:33 +01:00
nf_log.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/pablo/nf-next 2017-02-03 16:58:20 -05:00
nf_nat_amanda.c
nf_nat_core.c netfilter: built-in NAT support for UDPlite 2016-12-04 20:45:32 +01:00
nf_nat_ftp.c
nf_nat_helper.c skbuff: add and use skb_nfct helper 2017-02-02 14:31:53 +01:00
nf_nat_irc.c
nf_nat_proto_common.c
nf_nat_proto_dccp.c netfilter: built-in NAT support for DCCP 2016-12-04 20:45:30 +01:00
nf_nat_proto_sctp.c netfilter: nat: skip checksum on offload SCTP packets 2016-12-07 13:22:50 +01:00
nf_nat_proto_tcp.c
nf_nat_proto_udp.c netfilter: nat: merge udp and udplite helpers 2017-01-03 14:33:25 +01:00
nf_nat_proto_unknown.c
nf_nat_redirect.c
nf_nat_sip.c
nf_nat_tftp.c
nf_queue.c netfilter: introduce accessor functions for hook entries 2016-12-06 21:42:15 +01:00
nf_sockopt.c
nf_synproxy_core.c netns: make struct pernet_operations::id unsigned int 2016-11-18 10:59:15 -05:00
nf_tables_api.c netfilter: provide nft_ctx in object init function 2017-03-13 13:42:00 +01:00
nf_tables_core.c netfilter: nf_tables: simplify the basic expressions' init routine 2016-11-09 23:42:23 +01:00
nf_tables_inet.c netfilter: Add the missed return value check of nft_register_chain_type 2016-09-12 19:54:45 +02:00
nf_tables_netdev.c netfilter: Add the missed return value check of nft_register_chain_type 2016-09-12 19:54:45 +02:00
nf_tables_trace.c netfilter: nf_tables: use hook state from xt_action_param structure 2016-11-03 11:52:34 +01:00
nfnetlink_acct.c netfilter: nfnetlink: use list_for_each_entry_safe to delete all objects 2016-08-25 13:11:00 +02:00
nfnetlink_cthelper.c scripts/spelling.txt: add "an user" pattern and fix typo instances 2017-02-27 18:43:46 -08:00
nfnetlink_cttimeout.c netfilter: cttimeout: unlink timeout objs in the unconfirmed ct lists 2016-08-25 13:11:30 +02:00
nfnetlink_log.c ktime: Get rid of the union 2016-12-25 17:21:22 +01:00
nfnetlink_queue.c ktime: Get rid of the union 2016-12-25 17:21:22 +01:00
nfnetlink.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/pablo/nf 2017-02-23 10:59:15 -05:00
nft_bitwise.c netfilter: nf_tables: simplify the basic expressions' init routine 2016-11-09 23:42:23 +01:00
nft_byteorder.c netfilter: nf_tables: simplify the basic expressions' init routine 2016-11-09 23:42:23 +01:00
nft_cmp.c netfilter: nf_tables: simplify the basic expressions' init routine 2016-11-09 23:42:23 +01:00
nft_compat.c netfilter: nf_tables: validate the expr explicitly after init successfully 2017-03-06 18:22:12 +01:00
nft_counter.c netfilter: provide nft_ctx in object init function 2017-03-13 13:42:00 +01:00
nft_ct.c netfilter: nft_ct: add helper set support 2017-03-13 13:42:09 +01:00
nft_dup_netdev.c netfilter: nf_tables: add packet duplication to the netdev family 2016-01-03 21:04:23 +01:00
nft_dynset.c netfilter: nf_tables: add nft_set_lookup() 2017-03-06 18:23:23 +01:00
nft_exthdr.c netfilter: nft_exthdr: Allow checking TCP option presence, too 2017-03-06 17:52:56 +01:00
nft_fib_inet.c netfilter: nf_tables: use hook state from xt_action_param structure 2016-11-03 11:52:34 +01:00
nft_fib.c netfilter: nft_fib: Support existence check 2017-03-13 13:45:36 +01:00
nft_fwd_netdev.c netfilter: add and use nf_fwd_netdev_egress 2016-12-06 21:48:22 +01:00
nft_hash.c netfilter: nft_hash: support of symmetric hash 2017-03-06 17:57:42 +01:00
nft_immediate.c netfilter: nf_tables: simplify the basic expressions' init routine 2016-11-09 23:42:23 +01:00
nft_limit.c netfilter: limit: use per-rule spinlock to improve the scalability 2017-03-13 19:30:31 +01:00
nft_log.c netfilter: nft_log: restrict the log prefix length to 127 2017-01-24 21:46:29 +01:00
nft_lookup.c netfilter: nf_tables: add nft_set_lookup() 2017-03-06 18:23:23 +01:00
nft_masq.c netfilter: nf_tables: validate the expr explicitly after init successfully 2017-03-06 18:22:12 +01:00
nft_meta.c netfilter: nf_tables: validate the expr explicitly after init successfully 2017-03-06 18:22:12 +01:00
nft_nat.c netfilter: nf_tables: validate the expr explicitly after init successfully 2017-03-06 18:22:12 +01:00
nft_numgen.c netfilter: nft_numgen: start round robin from zero 2016-10-26 16:35:16 +02:00
nft_objref.c netfilter: nf_tables: add nft_set_lookup() 2017-03-06 18:23:23 +01:00
nft_payload.c netfilter: nft_payload: mangle ckecksum if NFT_PAYLOAD_L4CSUM_PSEUDOHDR is set 2016-12-14 23:39:11 +01:00
nft_queue.c netfilter: nft_queue: use raw_smp_processor_id() 2016-12-14 23:39:01 +01:00
nft_quota.c netfilter: provide nft_ctx in object init function 2017-03-13 13:42:00 +01:00
nft_range.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net 2016-12-03 12:29:53 -05:00
nft_redir.c netfilter: nf_tables: validate the expr explicitly after init successfully 2017-03-06 18:22:12 +01:00
nft_reject_inet.c netfilter: nf_tables: validate the expr explicitly after init successfully 2017-03-06 18:22:12 +01:00
nft_reject.c netfilter: nf_tables: validate the expr explicitly after init successfully 2017-03-06 18:22:12 +01:00
nft_rt.c netfilter: nf_tables: use hook state from xt_action_param structure 2016-11-03 11:52:34 +01:00
nft_set_bitmap.c netfilter: nft_set_bitmap: incorrect bitmap size 2017-02-26 21:00:19 +01:00
nft_set_hash.c netfilter: nf_tables: honor NFT_SET_OBJECT in set backend selection 2017-02-12 14:45:14 +01:00
nft_set_rbtree.c netfilter: nft_set_rbtree: use per-set rwlock to improve the scalability 2017-03-13 19:30:43 +01:00
x_tables.c scripts/spelling.txt: add "aligment" pattern and fix typo instances 2017-02-27 18:43:46 -08:00
xt_addrtype.c netfilter: x_tables: move hook state into xt_action_param structure 2016-11-03 10:56:21 +01:00
xt_AUDIT.c netfilter: x_tables: move hook state into xt_action_param structure 2016-11-03 10:56:21 +01:00
xt_bpf.c xtables: extend matches and targets with .usersize 2017-01-09 17:24:55 +01:00
xt_cgroup.c xtables: extend matches and targets with .usersize 2017-01-09 17:24:55 +01:00
xt_CHECKSUM.c
xt_CLASSIFY.c
xt_cluster.c netfilter: x_tables: move hook state into xt_action_param structure 2016-11-03 10:56:21 +01:00
xt_comment.c
xt_connbytes.c netfilter: add and use nf_ct_netns_get/put 2016-12-04 21:16:50 +01:00
xt_connlabel.c netfilter: add and use nf_ct_netns_get/put 2016-12-04 21:16:50 +01:00
xt_connlimit.c xtables: extend matches and targets with .usersize 2017-01-09 17:24:55 +01:00
xt_connmark.c netfilter: add and use nf_ct_netns_get/put 2016-12-04 21:16:50 +01:00
xt_CONNSECMARK.c netfilter: add and use nf_ct_netns_get/put 2016-12-04 21:16:50 +01:00
xt_conntrack.c netfilter: add and use nf_ct_netns_get/put 2016-12-04 21:16:50 +01:00
xt_cpu.c
xt_CT.c netfilter: merge ctinfo into nfct pointer storage area 2017-02-02 14:31:56 +01:00
xt_dccp.c
xt_devgroup.c netfilter: x_tables: move hook state into xt_action_param structure 2016-11-03 10:56:21 +01:00
xt_dscp.c netfilter: x_tables: move hook state into xt_action_param structure 2016-11-03 10:56:21 +01:00
xt_DSCP.c
xt_ecn.c
xt_esp.c
xt_hashlimit.c Merge git://git.kernel.org/pub/scm/linux/kernel/git/pablo/nf 2017-02-23 10:59:15 -05:00
xt_helper.c netfilter: add and use nf_ct_netns_get/put 2016-12-04 21:16:50 +01:00
xt_hl.c
xt_HL.c
xt_HMARK.c
xt_IDLETIMER.c netfilter: IDLETIMER: fix race condition when destroy the target 2016-04-29 14:28:48 +02:00
xt_ipcomp.c netfilter: xt_ipcomp: add "ip[6]t_ipcomp" module alias name 2016-10-17 17:38:19 +02:00
xt_iprange.c
xt_ipvs.c netfilter: x_tables: move hook state into xt_action_param structure 2016-11-03 10:56:21 +01:00
xt_l2tp.c
xt_LED.c
xt_length.c
xt_limit.c netfilter: limit: use per-rule spinlock to improve the scalability 2017-03-13 19:30:31 +01:00
xt_LOG.c netfilter: x_tables: move hook state into xt_action_param structure 2016-11-03 10:56:21 +01:00
xt_mac.c
xt_mark.c
xt_multiport.c netfilter: xt_multiport: Fix wrong unmatch result with multiple ports 2016-12-06 21:48:20 +01:00
xt_nat.c netfilter: nat: add dependencies on conntrack module 2016-12-04 21:16:51 +01:00
xt_NETMAP.c netfilter: nat: add dependencies on conntrack module 2016-12-04 21:16:51 +01:00
xt_nfacct.c netfilter: x_tables: move hook state into xt_action_param structure 2016-11-03 10:56:21 +01:00
xt_NFLOG.c netfilter: x_tables: move hook state into xt_action_param structure 2016-11-03 10:56:21 +01:00
xt_NFQUEUE.c netfilter: x_tables: move hook state into xt_action_param structure 2016-11-03 10:56:21 +01:00
xt_osf.c netfilter: x_tables: move hook state into xt_action_param structure 2016-11-03 10:56:21 +01:00
xt_owner.c sched/headers: Prepare to remove <linux/cred.h> inclusion from <linux/sched.h> 2017-03-02 08:42:31 +01:00
xt_physdev.c netfilter: physdev: add missed blank 2016-08-12 00:42:14 +02:00
xt_pkttype.c netfilter: pkttype: unnecessary to check ipv6 multicast address 2017-01-18 20:32:43 +01:00
xt_policy.c netfilter: x_tables: move hook state into xt_action_param structure 2016-11-03 10:56:21 +01:00
xt_quota.c xtables: extend matches and targets with .usersize 2017-01-09 17:24:55 +01:00
xt_rateest.c xtables: extend matches and targets with .usersize 2017-01-09 17:24:55 +01:00
xt_RATEEST.c xtables: extend matches and targets with .usersize 2017-01-09 17:24:55 +01:00
xt_realm.c
xt_recent.c netns: make struct pernet_operations::id unsigned int 2016-11-18 10:59:15 -05:00
xt_REDIRECT.c netfilter: nat: add dependencies on conntrack module 2016-12-04 21:16:51 +01:00
xt_repldata.h
xt_sctp.c sctp: rename WORD_TRUNC/ROUND macros 2016-09-22 03:13:26 -04:00
xt_SECMARK.c
xt_set.c netfilter: ipset: Improve skbinfo get/init helpers 2016-11-10 13:28:42 +01:00
xt_socket.c netfilter: defrag: only register defrag functionality if needed 2016-12-06 21:42:00 +01:00
xt_state.c netfilter: add and use nf_ct_netns_get/put 2016-12-04 21:16:50 +01:00
xt_statistic.c
xt_string.c xtables: extend matches and targets with .usersize 2017-01-09 17:24:55 +01:00
xt_tcpmss.c
xt_TCPMSS.c netfilter: x_tables: move hook state into xt_action_param structure 2016-11-03 10:56:21 +01:00
xt_TCPOPTSTRIP.c
xt_tcpudp.c netfilter: Convert FWINV<[foo]> macros and uses to NF_INVF 2016-07-03 10:55:07 +02:00
xt_TEE.c xtables: extend matches and targets with .usersize 2017-01-09 17:24:55 +01:00
xt_time.c ktime: Get rid of the union 2016-12-25 17:21:22 +01:00
xt_TPROXY.c netfilter: defrag: only register defrag functionality if needed 2016-12-06 21:42:00 +01:00
xt_TRACE.c netfilter: xt_TRACE: add explicitly nf_logger_find_get call 2016-06-23 13:26:49 +02:00
xt_u32.c