linux/kernel/bpf
Martin KaFai Lau cc555421bc bpf: Inline LRU map lookup
Inline the lru map lookup to save the cost in making calls to
bpf_map_lookup_elem() and htab_lru_map_lookup_elem().

Different LRU hash size is tested.  The benefit diminishes when
the cache miss starts to dominate in the bigger LRU hash.
Considering the change is simple, it is still worth to optimize.

First column: Size of the LRU hash
Second column: Number of lookups/s

Before:
> for i in $(seq 9 20); do echo "$((2**i+1)): $(./map_perf_test 1024 1 $((2**i+1)) 10000000 | awk '{print $3}')"; done
513: 1132020
1025: 1056826
2049: 1007024
4097: 853298
8193: 742723
16385: 712600
32769: 688142
65537: 677028
131073: 619437
262145: 498770
524289: 316695
1048577: 260038

After:
> for i in $(seq 9 20); do echo "$((2**i+1)): $(./map_perf_test 1024 1 $((2**i+1)) 10000000 | awk '{print $3}')"; done
513: 1221851
1025: 1144695
2049: 1049902
4097: 884460
8193: 773731
16385: 729673
32769: 721989
65537: 715530
131073: 671665
262145: 516987
524289: 321125
1048577: 260048

Signed-off-by: Martin KaFai Lau <kafai@fb.com>
Acked-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Alexei Starovoitov <ast@kernel.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
2017-09-01 09:57:38 -07:00
..
arraymap.c bpf: inline map in map lookup functions for array and htab 2017-08-19 21:56:34 -07:00
bpf_lru_list.c bpf: lru: Lower the PERCPU_NR_SCANS from 16 to 4 2017-04-17 13:55:52 -04:00
bpf_lru_list.h bpf: Add percpu LRU list 2016-11-15 11:50:20 -05:00
cgroup.c bpf: BPF support for sock_ops 2017-07-01 16:15:13 -07:00
core.c bpf: sock_map fixes for !CONFIG_BPF_SYSCALL and !STREAM_PARSER 2017-08-16 15:34:13 -07:00
devmap.c bpf: netdev is never null in __dev_map_flush 2017-08-23 22:43:40 -07:00
hashtab.c bpf: Inline LRU map lookup 2017-09-01 09:57:38 -07:00
helpers.c bpf: rename ARG_PTR_TO_STACK 2017-01-09 16:56:27 -05:00
inode.c bpf: Implement show_options 2017-07-06 03:31:46 -04:00
lpm_trie.c bpf: Allow selecting numa node during map creation 2017-08-19 21:35:43 -07:00
Makefile bpf: sock_map fixes for !CONFIG_BPF_SYSCALL and !STREAM_PARSER 2017-08-16 15:34:13 -07:00
map_in_map.c bpf: Add syscall lookup support for fd array and htab 2017-06-29 13:13:25 -04:00
map_in_map.h bpf: Add syscall lookup support for fd array and htab 2017-06-29 13:13:25 -04:00
percpu_freelist.c bpf: introduce percpu_freelist 2016-03-08 15:28:31 -05:00
percpu_freelist.h bpf: introduce percpu_freelist 2016-03-08 15:28:31 -05:00
sockmap.c bpf: fix oops on allocation failure 2017-08-28 15:23:34 -07:00
stackmap.c bpf: Allow selecting numa node during map creation 2017-08-19 21:35:43 -07:00
syscall.c bpf: convert sockmap field attach_bpf_fd2 to type 2017-08-28 11:13:21 -07:00
tnum.c bpf/verifier: track signed and unsigned min/max values 2017-08-08 17:51:34 -07:00
verifier.c bpf/verifier: document liveness analysis 2017-08-23 22:38:08 -07:00