linux/kernel/bpf
Andrii Nakryiko f655badf2a bpf: fix propagate_precision() logic for inner frames
Fix propagate_precision() logic to perform propagation of all necessary
registers and stack slots across all active frames *in one batch step*.

Doing this for each register/slot in each individual frame is wasteful,
but the main problem is that backtracking of instruction in any frame
except the deepest one just doesn't work. This is due to backtracking
logic relying on jump history, and available jump history always starts
(or ends, depending how you view it) in current frame. So, if
prog A (frame #0) called subprog B (frame #1) and we need to propagate
precision of, say, register R6 (callee-saved) within frame #0, we
actually don't even know where jump history that corresponds to prog
A even starts. We'd need to skip subprog part of jump history first to
be able to do this.

Luckily, with struct backtrack_state and __mark_chain_precision()
handling bitmasks tracking/propagation across all active frames at the
same time (added in previous patch), propagate_precision() can be both
fixed and sped up by setting all the necessary bits across all frames
and then performing one __mark_chain_precision() pass. This makes it
unnecessary to skip subprog parts of jump history.

We also improve logging along the way, to clearly specify which
registers' and slots' precision markings are propagated within which
frame. Each frame will have dedicated line and all registers and stack
slots from that frame will be reported in format similar to precision
backtrack regs/stack logging. E.g.:

frame 1: propagating r1,r2,r3,fp-8,fp-16
frame 0: propagating r3,r9,fp-120

Fixes: 529409ea92 ("bpf: propagate precision across all frames, not just the last one")
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Link: https://lore.kernel.org/r/20230505043317.3629845-7-andrii@kernel.org
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
2023-05-04 22:35:35 -07:00
..
preload bpf: iterators: Split iterators.lskel.h into little- and big- endian versions 2023-01-28 12:45:15 -08:00
arraymap.c bpf: return long from bpf_map_ops funcs 2023-03-22 15:11:30 -07:00
bloom_filter.c bpf: compute hashes in bloom filter similar to hashmap 2023-04-02 08:44:49 -07:00
bpf_cgrp_storage.c bpf: Teach verifier that certain helpers accept NULL pointer. 2023-04-04 16:57:16 -07:00
bpf_inode_storage.c Networking changes for 6.4. 2023-04-26 16:07:23 -07:00
bpf_iter.c bpf: implement numbers iterator 2023-03-08 16:19:51 -08:00
bpf_local_storage.c bpf: Handle NULL in bpf_local_storage_free. 2023-04-12 10:27:50 -07:00
bpf_lru_list.c
bpf_lru_list.h
bpf_lsm.c bpf: Fix the kernel crash caused by bpf_setsockopt(). 2023-01-26 23:26:40 -08:00
bpf_struct_ops_types.h
bpf_struct_ops.c bpf: Check IS_ERR for the bpf_map_get() return value 2023-03-24 12:40:47 -07:00
bpf_task_storage.c bpf: Teach verifier that certain helpers accept NULL pointer. 2023-04-04 16:57:16 -07:00
btf.c bpf: minimal support for programs hooked into netfilter framework 2023-04-21 11:34:14 -07:00
cgroup_iter.c bpf: Pin the start cgroup in cgroup_iter_seq_init() 2022-11-21 17:40:42 +01:00
cgroup.c bpf: Don't EFAULT for getsockopt with optval=NULL 2023-04-21 17:09:53 +02:00
core.c bpf: Support 64-bit pointers to kfuncs 2023-04-13 21:36:41 -07:00
cpumap.c bpf: return long from bpf_map_ops funcs 2023-03-22 15:11:30 -07:00
cpumask.c bpf: Treat KF_RELEASE kfuncs as KF_TRUSTED_ARGS 2023-03-25 16:56:22 -07:00
devmap.c bpf: return long from bpf_map_ops funcs 2023-03-22 15:11:30 -07:00
disasm.c
disasm.h
dispatcher.c bpf: Synchronize dispatcher update with bpf_dispatcher_xdp_func 2022-12-14 12:02:14 -08:00
hashtab.c bpf: optimize hashmap lookups when key_size is divisible by 4 2023-04-01 15:08:19 -07:00
helpers.c bpf: Add bpf_dynptr_clone 2023-04-27 10:40:47 +02:00
inode.c fs: port inode_init_owner() to mnt_idmap 2023-01-19 09:24:28 +01:00
Kconfig
link_iter.c
local_storage.c bpf: return long from bpf_map_ops funcs 2023-03-22 15:11:30 -07:00
log.c bpf: Relax log_buf NULL conditions when log_level>0 is requested 2023-04-11 18:05:44 +02:00
lpm_trie.c bpf: return long from bpf_map_ops funcs 2023-03-22 15:11:30 -07:00
Makefile bpf: Split off basic BPF verifier log into separate file 2023-04-11 18:05:42 +02:00
map_in_map.c bpf: Remove btf_field_offs, use btf_record's fields instead 2023-04-15 17:36:49 -07:00
map_in_map.h
map_iter.c
memalloc.c bpf: Add a few bpf mem allocator functions 2023-03-25 19:52:51 -07:00
mmap_unlock_work.h
net_namespace.c
offload.c bpf: offload map memory usage 2023-03-07 09:33:43 -08:00
percpu_freelist.c bpf: Initialize same number of free nodes for each pcpu_freelist 2022-11-11 12:05:14 -08:00
percpu_freelist.h
prog_iter.c
queue_stack_maps.c bpf: return long from bpf_map_ops funcs 2023-03-22 15:11:30 -07:00
reuseport_array.c bpf: return long from bpf_map_ops funcs 2023-03-22 15:11:30 -07:00
ringbuf.c bpf: return long from bpf_map_ops funcs 2023-03-22 15:11:30 -07:00
stackmap.c bpf: return long from bpf_map_ops funcs 2023-03-22 15:11:30 -07:00
syscall.c bpf: Print a warning only if writing to unprivileged_bpf_disabled. 2023-05-02 16:20:31 -07:00
sysfs_btf.c
task_iter.c bpf: keep a reference to the mm, in case the task is dead. 2022-12-28 14:11:48 -08:00
tnum.c
trampoline.c Networking changes for 6.4. 2023-04-26 16:07:23 -07:00
verifier.c bpf: fix propagate_precision() logic for inner frames 2023-05-04 22:35:35 -07:00