linux/kernel/bpf
Alexei Starovoitov 8c290e60fa bpf: fix hashmap extra_elems logic
In both kmalloc and prealloc mode the bpf_map_update_elem() is using
per-cpu extra_elems to do atomic update when the map is full.
There are two issues with it. The logic can be misused, since it allows
max_entries+num_cpus elements to be present in the map. And alloc_extra_elems()
at map creation time can fail percpu alloc for large map values with a warn:
WARNING: CPU: 3 PID: 2752 at ../mm/percpu.c:892 pcpu_alloc+0x119/0xa60
illegal size (32824) or align (8) for percpu allocation

The fixes for both of these issues are different for kmalloc and prealloc modes.
For prealloc mode allocate extra num_possible_cpus elements and store
their pointers into extra_elems array instead of actual elements.
Hence we can use these hidden(spare) elements not only when the map is full
but during bpf_map_update_elem() that replaces existing element too.
That also improves performance, since pcpu_freelist_pop/push is avoided.
Unfortunately this approach cannot be used for kmalloc mode which needs
to kfree elements after rcu grace period. Therefore switch it back to normal
kmalloc even when full and old element exists like it was prior to
commit 6c90598174 ("bpf: pre-allocate hash map elements").

Add tests to check for over max_entries and large map values.

Reported-by: Dave Jones <davej@codemonkey.org.uk>
Fixes: 6c90598174 ("bpf: pre-allocate hash map elements")
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Martin KaFai Lau <kafai@fb.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2017-03-22 14:12:18 -07:00
..
arraymap.c bpf: mark all registered map/prog types as __ro_after_init 2017-02-17 13:40:04 -05:00
bpf_lru_list.c bpf: Make unnecessarily global functions static 2017-01-10 21:00:59 -05:00
bpf_lru_list.h bpf: Add percpu LRU list 2016-11-15 11:50:20 -05:00
cgroup.c bpf: introduce BPF_F_ALLOW_OVERRIDE flag 2017-02-12 21:52:19 -05:00
core.c bpf: make jited programs visible in traces 2017-02-17 13:40:05 -05:00
hashtab.c bpf: fix hashmap extra_elems logic 2017-03-22 14:12:18 -07:00
helpers.c bpf: rename ARG_PTR_TO_STACK 2017-01-09 16:56:27 -05:00
inode.c bpf: add initial bpf tracepoints 2017-01-25 13:17:47 -05:00
lpm_trie.c bpf: add get_next_key callback to LPM map 2017-03-05 17:55:29 -08:00
Makefile bpf: add a longest prefix match trie map implementation 2017-01-23 16:10:38 -05:00
percpu_freelist.c bpf: introduce percpu_freelist 2016-03-08 15:28:31 -05:00
percpu_freelist.h bpf: introduce percpu_freelist 2016-03-08 15:28:31 -05:00
stackmap.c bpf: mark all registered map/prog types as __ro_after_init 2017-02-17 13:40:04 -05:00
syscall.c sched/headers: Prepare for new header dependencies before moving code to <linux/sched/signal.h> 2017-03-02 08:42:29 +01:00
verifier.c bpf: update the comment about the length of analysis 2017-03-01 14:56:50 -08:00