linux/kernel/bpf
Joanne Koong bc34dee65a bpf: Dynptr support for ring buffers
Currently, our only way of writing dynamically-sized data into a ring
buffer is through bpf_ringbuf_output but this incurs an extra memcpy
cost. bpf_ringbuf_reserve + bpf_ringbuf_commit avoids this extra
memcpy, but it can only safely support reservation sizes that are
statically known since the verifier cannot guarantee that the bpf
program won’t access memory outside the reserved space.

The bpf_dynptr abstraction allows for dynamically-sized ring buffer
reservations without the extra memcpy.

There are 3 new APIs:

long bpf_ringbuf_reserve_dynptr(void *ringbuf, u32 size, u64 flags, struct bpf_dynptr *ptr);
void bpf_ringbuf_submit_dynptr(struct bpf_dynptr *ptr, u64 flags);
void bpf_ringbuf_discard_dynptr(struct bpf_dynptr *ptr, u64 flags);

These closely follow the functionalities of the original ringbuf APIs.
For example, all ringbuffer dynptrs that have been reserved must be
either submitted or discarded before the program exits.

Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Andrii Nakryiko <andrii@kernel.org>
Acked-by: Andrii Nakryiko <andrii@kernel.org>
Acked-by: David Vernet <void@manifault.com>
Link: https://lore.kernel.org/bpf/20220523210712.3641569-4-joannelkoong@gmail.com
2022-05-23 14:31:28 -07:00
..
preload bpf: Remove redundant slash 2022-03-07 22:19:32 -08:00
arraymap.c bpf: add bpf_map_lookup_percpu_elem for percpu map 2022-05-11 18:16:54 -07:00
bloom_filter.c bpf: Compute map_btf_id during build time 2022-04-26 11:35:21 -07:00
bpf_inode_storage.c bpf: Compute map_btf_id during build time 2022-04-26 11:35:21 -07:00
bpf_iter.c bpf: Remove redundant assignment to meta.seq in __task_seq_show() 2022-04-11 21:14:34 +02:00
bpf_local_storage.c bpf: Fix usage of trace RCU in local storage. 2022-04-19 17:55:45 -07:00
bpf_lru_list.c bpf_lru_list: Read double-checked variable once without lock 2021-02-10 15:54:26 -08:00
bpf_lru_list.h bpf: Fix a typo "inacitve" -> "inactive" 2020-04-06 21:54:10 +02:00
bpf_lsm.c bpf, x86: Attach a cookie to fentry/fexit/fmod_ret/lsm. 2022-05-10 21:58:31 -07:00
bpf_struct_ops_types.h bpf: Add dummy BPF STRUCT_OPS for test purpose 2021-11-01 14:10:00 -07:00
bpf_struct_ops.c bpf, x86: Generate trampolines from bpf_tramp_links 2022-05-10 17:50:40 -07:00
bpf_task_storage.c bpf: Compute map_btf_id during build time 2022-04-26 11:35:21 -07:00
btf.c bpf: Allow kfunc in tracing and syscall programs. 2022-05-20 19:28:33 -07:00
cgroup.c bpf: Use bpf_prog_run_array_cg_flags everywhere 2022-04-25 17:03:57 -07:00
core.c bpf: Introduce bpf_arch_text_invalidate for bpf_prog_pack 2022-05-23 23:08:11 +02:00
cpumap.c bpf: Compute map_btf_id during build time 2022-04-26 11:35:21 -07:00
devmap.c bpf: Compute map_btf_id during build time 2022-04-26 11:35:21 -07:00
disasm.c bpf: Relicense disassembler as GPL-2.0-only OR BSD-2-Clause 2021-09-02 14:49:23 +02:00
disasm.h bpf: Relicense disassembler as GPL-2.0-only OR BSD-2-Clause 2021-09-02 14:49:23 +02:00
dispatcher.c bpf: Remove bpf_image tree 2020-03-13 12:49:52 -07:00
hashtab.c bpf: add bpf_map_lookup_percpu_elem for percpu map 2022-05-11 18:16:54 -07:00
helpers.c bpf: Dynptr support for ring buffers 2022-05-23 14:31:28 -07:00
inode.c bpf: Convert bpf_preload.ko to use light skeleton. 2022-02-10 23:31:51 +01:00
Kconfig bpf: Add "live packet" mode for XDP in BPF_PROG_RUN 2022-03-09 14:19:22 -08:00
link_iter.c bpf: Add bpf_link iterator 2022-05-10 11:20:45 -07:00
local_storage.c bpf: Compute map_btf_id during build time 2022-04-26 11:35:21 -07:00
lpm_trie.c bpf: Compute map_btf_id during build time 2022-04-26 11:35:21 -07:00
Makefile bpf: Add bpf_link iterator 2022-05-10 11:20:45 -07:00
map_in_map.c bpf: Allow storing unreferenced kptr in map 2022-04-25 17:31:35 -07:00
map_in_map.h bpf: Add map_meta_equal map ops 2020-08-28 15:41:30 +02:00
map_iter.c bpf: Introduce MEM_RDONLY flag 2021-12-18 13:27:41 -08:00
mmap_unlock_work.h bpf: Introduce helper bpf_find_vma 2021-11-07 11:54:51 -08:00
net_namespace.c net: Add includes masked by netdevice.h including uapi/bpf.h 2021-12-29 20:03:05 -08:00
offload.c bpf, offload: Replace bitwise AND by logical AND in bpf_prog_offload_info_fill 2020-02-17 16:53:49 +01:00
percpu_freelist.c bpf: Use raw_spin_trylock() for pcpu_freelist_push/pop in NMI 2020-10-06 00:04:11 +02:00
percpu_freelist.h bpf: Use raw_spin_trylock() for pcpu_freelist_push/pop in NMI 2020-10-06 00:04:11 +02:00
prog_iter.c bpf: Refactor bpf_iter_reg to have separate seq_info member 2020-07-25 20:16:32 -07:00
queue_stack_maps.c bpf: Compute map_btf_id during build time 2022-04-26 11:35:21 -07:00
reuseport_array.c bpf: Compute map_btf_id during build time 2022-04-26 11:35:21 -07:00
ringbuf.c bpf: Dynptr support for ring buffers 2022-05-23 14:31:28 -07:00
stackmap.c bpf: Compute map_btf_id during build time 2022-04-26 11:35:21 -07:00
syscall.c bpf: refine kernel.unprivileged_bpf_disabled behaviour 2022-05-20 19:48:29 -07:00
sysfs_btf.c bpf: Load and verify kernel module BTFs 2020-11-10 15:25:53 -08:00
task_iter.c bpf: Remove redundant assignment to meta.seq in __task_seq_show() 2022-04-11 21:14:34 +02:00
tnum.c bpf, tnums: Provably sound, faster, and more precise algorithm for tnum_mul 2021-06-01 13:34:15 +02:00
trampoline.c bpf: Fix potential array overflow in bpf_trampoline_get_progs() 2022-05-11 21:24:20 -07:00
verifier.c bpf: Dynptr support for ring buffers 2022-05-23 14:31:28 -07:00