David Windsor [Fri, 5 Jun 2026 14:57:07 +0000 (10:57 -0400)]
bpf: Reject sleepable BPF_LSM_CGROUP programs at load time
The cgroup shim runs under rcu_read_lock_dont_migrate(), so we should
not attach any sleepable BPF programs there. Add support to the verifier
to explicitly reject attempts to load sleepable BPF programs destined
for LSM cgroup attachment.
Without this, we get the following splat from a BPF_LSM_CGROUP
program marked BPF_F_SLEEPABLE attached to file_open when it calls
bpf_get_dentry_xattr():
libbpf provides a .s named section variant for every sleepable
program type except lsm_cgroup, reflecting that per-cgroup LSM programs
are intended to only run in a non-sleepable context.
The above splat was obtained by bypassing libbpf by using bpf(2)
directly.
Fixes: 69fd337a975c ("bpf: per-cgroup lsm flavor") Signed-off-by: David Windsor <dwindsor@gmail.com> Acked-by: Yonghong Song <yonghong.song@linux.dev> Acked-by: Song Liu <song@kernel.org> Link: https://lore.kernel.org/bpf/20260605145707.608579-1-dwindsor@gmail.com Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
A constant offset added to a PTR_TO_FLOW_KEYS register lands in
reg->var_off, but check_flow_keys_access() bounds-checks only insn->off
and never folds reg->var_off.value. A BPF_PROG_TYPE_FLOW_DISSECTOR
program can therefore do "flow_keys += 0x1000; *(flow_keys + 0)" and have
it accepted, then read/write kernel stack past struct bpf_flow_keys at
runtime. Patch 1 folds reg->var_off.value into the offset (and rejects
non-constant offsets), mirroring check_ctx_access(); patch 2 adds verifier
selftests.
This is a regression introduced in the 7.1 development cycle by commit 022ac0750883 ("bpf: use reg->var_off instead of reg->off for pointers"),
which moved the constant offset from reg->off (folded generically before 022ac0750883) into reg->var_off without updating the flow_keys path. No
released kernel is affected: v7.0.x rejects the program above, and the bug
reproduces only on v7.1-rc1..rc5, so no stable backport is needed.
It was first reported privately to security@kernel.org; per their guidance
it is handled in the open as a normal regression fix. Found by manual
verifier audit and confirmed dynamically in a disposable QEMU/KVM guest:
the load above is accepted, a runtime read leaked a kernel-stack pointer
0x1000 past bpf_flow_keys, and a runtime write of a marker faulted the
guest in net_rx_action.
An alternative -- forbidding pointer arithmetic on PTR_TO_FLOW_KEYS
outright by dropping "if (known) break;" in adjust_ptr_min_max_vals() --
was rejected because v7.0.x accepted (and correctly bounds-checked)
constant arithmetic on the keys pointer; restoring the fold preserves that
behaviour while closing the divergence.
Signed-off-by: Nuoqi Gui <gnq25@mails.tsinghua.edu.cn>
---
v2 -> v3:
- Pass existing reg/argno context into check_flow_keys_access(), avoiding
a stale regno reference in check_mem_access().
- Add a variable-offset selftest using bpf_get_prandom_u32().
v1 -> v2:
- Target bpf-next instead of bpf (per reviewer feedback).
- Base-commit updated to bpf-next/master.
Nuoqi Gui [Sat, 6 Jun 2026 10:50:38 +0000 (18:50 +0800)]
selftests/bpf: add tests for PTR_TO_FLOW_KEYS offset bounds
Add verifier tests covering pointer arithmetic on a PTR_TO_FLOW_KEYS
register. This covers the bpf-next regression where an out-of-bounds
constant offset introduced as flow_keys += K and then dereferenced at
insn->off 0 was accepted, while the equivalent flow_keys + K direct offset
was rejected.
The tests check that in-bounds constant arithmetic on the keys pointer is
still accepted, out-of-bounds constant arithmetic is rejected for both read
and write, and a truly varying offset from bpf_get_prandom_u32() remains
rejected by the existing PTR_TO_FLOW_KEYS pointer arithmetic rules.
Nuoqi Gui [Sat, 6 Jun 2026 10:50:37 +0000 (18:50 +0800)]
bpf: Fold reg->var_off into PTR_TO_FLOW_KEYS bounds check
Constant pointer arithmetic on a PTR_TO_FLOW_KEYS register lands the
constant in reg->var_off (e.g. flow_keys(imm=4096)), but the
PTR_TO_FLOW_KEYS path in check_mem_access() passes only insn->off to
check_flow_keys_access() and never folds reg->var_off.value. The
verifier therefore accepts an access that, at runtime, dereferences past
struct bpf_flow_keys -- a verifier/runtime divergence that yields an
out-of-bounds read and write of kernel stack memory.
Commit 022ac0750883 ("bpf: use reg->var_off instead of reg->off for
pointers") removed the generic "off += reg->off" that check_mem_access()
applied before the per-type dispatch and replaced it with per-path
folding of reg->var_off.value (for example the ctx path now folds the
register offset via check_ctx_access()). The PTR_TO_FLOW_KEYS path was
not given the equivalent fold, so a constant offset that used to be
folded and rejected is now silently accepted:
before 022ac0750883: the offset stays in reg->off and is folded
generically, so the access is checked with off=4096 and rejected.
after 022ac0750883: the offset lands in reg->var_off, the flow_keys
path checks off=0 and accepts; at runtime the access dereferences
base + 0x1000.
For a BPF_PROG_TYPE_FLOW_DISSECTOR program the following is accepted:
has the same effective offset but is correctly rejected with
"invalid access to flow keys off=4096 size=8", which isolates the defect
to the missing var_off fold. Once attached as a flow dissector, the
accepted program reads kernel stack past struct bpf_flow_keys (a
kernel-stack / KASLR information leak) and can likewise write past it,
corrupting kernel memory.
Fix it by folding reg->var_off.value into the offset before the bounds
check and rejecting non-constant offsets, mirroring the other pointer
types (e.g. check_ctx_access()).
Daniel Borkmann [Fri, 5 Jun 2026 21:35:18 +0000 (23:35 +0200)]
selftests/bpf: Inspect the signature verdict exposed to BPF LSM
Add a minimal BPF LSM program on lsm/bpf_prog_load that, for loads on
the monitored thread, reads back prog->aux->sig.{verdict,keyring_type,
keyring_serial}, and a signed_loader subtest that drives the same
gen_loader loader through the hook twice: i) /unsigned/ where the LSM
must observe UNSIGNED, no keyring and serial 0; ii) /signed/ where the
very same insns signed against the session keyring must be observed as
VERIFIED with a user keyring, and the recorded keyring_serial must be
equal to the resolved session keyring serial. Loading (not running) the
loader is sufficient since the verdict is attached at load time.
KP Singh [Fri, 5 Jun 2026 21:35:17 +0000 (23:35 +0200)]
bpf: Expose signature verdict via bpf_prog_aux
BPF_PROG_LOAD verifies the loader signature but does not record the
outcome on the BPF program. [BPF] LSMs and audit can read attr->signature
and attr->keyring_id to infer "was this signed, and if so, against which
keyring".
Add prog->aux->sig (verdict + keyring_{type,serial}), populated by
bpf_prog_load before the LSM hook. keyring_type classifies the keyring
the load referenced (builtin, secondary, platform or user), while
keyring_serial records the serial of the keyring the signature was
actually validated against. System keyrings carry a pseudo key pointer
with no user-visible serial and are reported as 0, as are unsigned loads.
Failed verifications reject the load before the hook runs, so it observes
only either UNSIGNED or VERIFIED.
Signed-off-by: KP Singh <kpsingh@kernel.org> Co-developed-by: Daniel Borkmann <daniel@iogearbox.net> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Link: https://lore.kernel.org/r/20260605213518.544262-1-daniel@iogearbox.net Signed-off-by: Alexei Starovoitov <ast@kernel.org>
====================
selftests/bpf: libarena: Add initial data structures
Add two new data structures to libarena. These data structures initially
resided in the sched-ext repo (https://github.com/sched-ext/scx) and
have been adapted to the internal libarena build system. The data
structures are:
- Red black tree: Fundamental tree data structure that can also serve
as a base for more domain-specific data structures.
- Lev-Chase deque: Queue data structure that allows efficient work
stealing, useful in scheduling scenarios.
The data structures are accompanied by selftests that are automatically
discovered by the existing libarena test_progs selftest and incorporated
in the CI.
CHANGELOG
=========
v3 -> v4 (https://lore.kernel.org/bpf/20260604235016.20856-1-emil@etsalapatis.com/)
- Turn off load_acquire/store_relesase - dependent selftests for s390 (CI)
- Various style/non-functional nits (AI)
- Add workaround to handle LLVM 21 and GCC 15 assignment-to-memset promotions
that are causing verification failures for arena programs (CI)
- Incorporate Sashiko feedback for cleanup edge cases (Sashiko)
- Simplify some of the ordering semantics in spmc
- Rename tests from st_ to test_ (Alexei)
- Removed the freelist caches from the rbtrees, previously used to defer freeing (Alexei)
- Moved the type and function definitions to use the __arena identifier
- Removed the typecasts during function return and directly return __arena
pointers (Alexei)
- Renamed queues to spmc queues to abstract away the algorithm (Alexei)
- Adjusted the memory barriers in the spmc queue
- Added multithreaded testing harness for libarena programs (Alexei)
- Added parallel selftest for queues (Alexei)
- Split least upper bound and exact find operations back into separate
functions to prevent RB_DUPLICATE-related bug (AI)
====================
Emil Tsalapatis [Fri, 5 Jun 2026 22:20:20 +0000 (18:20 -0400)]
selftests/bpf: libarena: parallel test harness and spmc parallel selftest
Add a parallel test for the SPMC Lev-Chase workstealing queue. The queue
is built to be wait-free even when there are multiple consumers, and
the parallel selftest provides a signal on whether the queue behaves
correctly when stress tested.
To support the test, this patch includes a test harness for parallel
selftests. The spmc selftest acts as an example of the naming and other
conventions expected by the harness.
Emil Tsalapatis [Fri, 5 Jun 2026 22:20:19 +0000 (18:20 -0400)]
selftests/bpf: libarena: Add spmc queue data structure
Expand libarena with a single producer multiple consumer deque data
structure. This is a single producer, multiple consumer lockless structure
that permits efficient work stealing. The structure is a Lev-Chase queue,
so it is lock-free and wait-free.
The data structure exposes three main calls. two of them are available to
the thread owning the queue and one available to all threads in the program:
spmc_owner_push(): Push an item to the top of the queue.
spmc_owner_pop(): Pop an item from the top of the queue.
spmc_steal(): Steal a thread from the bottom of the queue from
any thread.
Note that the queue is not really FIFO for all consumers, since
non-owners of the queue can only work steal from the bottom.
Emil Tsalapatis [Fri, 5 Jun 2026 22:20:18 +0000 (18:20 -0400)]
selftests/bpf: libarena: Add rbtree data structure
Add a native red-black tree data structure to libarena.
The data structure supports multiple APIs (key-value based,
node based) with which users can query and modify it. The
tree uses the libarena memory allocator to manage its data.
Sean Young [Fri, 5 Jun 2026 15:14:16 +0000 (16:14 +0100)]
selftests/bpf: Fix test_lirc test
Since commit 68a99f6a0ebf ("media: lirc: report ir receiver overflow"),
the rc-loopback driver does not accept edges over 50ms, as these are
never seen in real life ir protocols. Fix this.
====================
Add validation for bpf_set_retval helper
From: Xu Kuohai <xukuohai@huawei.com>
The bpf_set_retval() helper is used by cgroup BPF programs to set the
return value of the kernel hook. The argument type for this helper is
ARG_ANYTHING. This allows setting a positive value, which no cgroup
hook expects and can cause issues, such as the kernel panic reported
in [1].
This series adds validation for the argument of the bpf_set_retval()
helper.
For BPF_LSM_CGROUP, the same validation as BPF_LSM_MAC is enforced,
i.e. validate the argument against the LSM hook specific range, which
is returned by bpf_lsm_get_retval_range().
For all other cgroup program types, restrict the argument to
[-MAX_ERRNO, 0], which matches the kernel convention of 0 for success
and negative errno for error.
BPF_CGROUP_GETSOCKOPT is an exception from this restriction, since valid
getsockopt implementations may return positive values (e.g. optlen), as
allowed by commit c4dcfdd406aa ("bpf: Move getsockopt retval to struct
bpf_cg_run_ctx").
v5:
- Use resolve_prog_type(env->prog) instead of env->prog->type for prog type checks
- Target bpf-next tree
v4: https://lore.kernel.org/bpf/20260604130458.617765-1-xukuohai@huaweicloud.com
- Remove the return value limit for BPF_CGROUP_GETSOCKOPT type
- Refine the range of return value of bpf_get_retval helper
v3: https://lore.kernel.org/bpf/20260530101239.590395-1-xukuohai@huaweicloud.com/
- Mark R1 as precise to prevent validation bypass via branch pruning (sashiko)
v2: https://lore.kernel.org/bpf/20260530055557.549474-1-xukuohai@huaweicloud.com/
- Extend validation from LSM cgroup BPF type to all cgroup BPF types (sashiko)
Xu Kuohai [Fri, 5 Jun 2026 14:02:42 +0000 (14:02 +0000)]
bpf: Add validation for bpf_set_retval argument
The bpf_set_retval() helper is used by cgroup BPF programs to set the
return value of the target hook. The argument type for this helper is
ARG_ANYTHING. This allows setting a positive value, which no cgroup
hook expects and can cause issues, such as:
- BPF_LSM_CGROUP: a positive value from bpf_lsm_socket_create bypasses
the err < 0 check in __sock_create(), leaving the socket object
unallocated. The positive return value is then propagated to the
syscall entry __sys_socket(), which also bypasses the IS_ERR() guard
and ultimately causes a NULL pointer dereference.
- BPF_CGROUP_DEVICE: a positive value can be returned through cgroup
device bpf prog -> devcgroup_check_permission() -> bdev_permission()
-> bdev_file_open_by_dev(), where ERR_PTR(positive) produces a pointer
that IS_ERR() does not catch, leading to a wild pointer dereference.
- BPF_CGROUP_SOCK: a positive value can be returned through cgroup sock
bpf prog -> __cgroup_bpf_run_filter_sk() -> inet_create() ->
__sock_create(), where inet_create() frees the newly allocated sk
via sk_common_release() and sets sock->sk = NULL on the non-zero
return, but __sock_create() only checks err < 0 for cleanup, so a
positive retval bypasses cleanup and returns a socket with NULL sk
to userspace, triggering a NULL pointer dereference on subsequent
socket operations.
- BPF_CGROUP_SYSCTL: a positive value can be returned through the cgroup
bpf prog -> __cgroup_bpf_run_filter_sysctl() -> proc_sys_call_handler(),
where a non-zero return bypasses the normal sysctl proc_handler and is
returned directly to userspace as return value of read() or write()
syscall.
So add validation for the argument of the bpf_set_retval() helper.
For BPF_LSM_CGROUP, enforce the LSM hook specific range returned by
bpf_lsm_get_retval_range().
For all other cgroup program types, restrict the argument to
[-MAX_ERRNO, 0], which matches the kernel convention of 0 for success
and negative errno for error.
BPF_CGROUP_GETSOCKOPT is an exception, since valid getsockopt
implementations may return positive values, as allowed by commit c4dcfdd406aa ("bpf: Move getsockopt retval to struct bpf_cg_run_ctx").
Also refine the return value range of bpf_get_retval() so that
values returned by bpf_get_retval() can be passed directly to
bpf_set_retval() without extra manual bounds checking.
Xu Kuohai [Fri, 5 Jun 2026 14:02:41 +0000 (14:02 +0000)]
selftests/bpf: Restrict bpf_set_retval argument in sk_bypass_prot_mem
Test sk_bypass_prot_mem passes an unchecked value as argument to helper
bpf_set_retval(). The argument can be outside the valid range enforced
by the strict retval validation added in the next patch.
Restrict the argument to -EFAULT when it is outside the valid range, so
the test will not be rejected by the verifier when retval validation
is enforced.
====================
bpf: fix sysctl new-value handling in __cgroup_bpf_run_filter_sysctl
This series fixes three bugs in the sysctl write-buffer replacement path
of __cgroup_bpf_run_filter_sysctl(). It resolves a kvzalloc()/kfree()
mismatch, adds a missing NUL terminator to the replacement string, and
updates a stale return value check to safely restore the replacement
functionality.
Patch Summary:
- patch 1 NUL-terminates the replaced sysctl value
- patch 2 uses kvfree() for the replaced sysctl write buffer
- patch 3 restores sysctl new-value replacement
Changelog:
v2 -> v3:
- reordered patches 1 and 2
- added the missing Reviewed-by/Acked-by tags to patches 2 and 3
- fixed the incorrect Fixes tag in patch 3
- simplified the dynamic test logs in patch 1 and 2, and updated
titles
====================
Dawei Feng [Wed, 3 Jun 2026 10:53:17 +0000 (18:53 +0800)]
bpf: Restore sysctl new-value from 1 to 0
Commit 4e63acdff864 ("bpf: Introduce bpf_sysctl_{get,set}_new_value
helpers") changed the success return value to 0, but failed to update the
corresponding check in __cgroup_bpf_run_filter_sysctl(). Since
bpf_prog_run_array_cg() now returns 0 on success, the legacy ret == 1
condition is never satisfied. As a result, the modified value is ignored,
and bpf_sysctl_set_new_value() fails to replace the write buffer.
Fix this by checking for a return value of 0 instead, so cgroup/sysctl
programs can correctly replace the pending sysctl buffer.
This bug was discovered during a manual code review. Tested via a
cgroup/sysctl BPF reproducer overriding writes to a target sysctl.
Pre-fix, bpf_sysctl_set_new_value("foo") was silently ignored: the write
returned 8192 and the value remained "600". Post-fix, the BPF replacement
buffer properly propagates: the write returns 3 and the value updates to
"foo".
Dawei Feng [Wed, 3 Jun 2026 10:53:16 +0000 (18:53 +0800)]
bpf: use kvfree() for replaced sysctl write buffer
proc_sys_call_handler() allocates its temporary sysctl buffer with
kvzalloc() and passes it to __cgroup_bpf_run_filter_sysctl(). Since
kvzalloc() may fall back to vmalloc() for large allocations, freeing
that buffer with kfree() is wrong and can corrupt memory.
Use kvfree() to safely handle both kmalloc and kvzalloc()/vmalloc
allocations.
The bug was first flagged by an experimental analysis tool we are
developing for kernel memory-management bugs while analyzing
v6.13-rc1. The tool is still under development and is not yet publicly
available. Manual inspection confirms that the bug is still
present in v7.1-rc5.
Reproduced the bug based on v7.1-rc4 in a QEMU x86_64 guest booted with
KASAN and CONFIG_FAILSLAB enabled. To exercise the replacement path, the
test tree also included the accompanying fix for the stale ret == 1
check in __cgroup_bpf_run_filter_sysctl(). The reproducer confines
failslab injections to the proc_sys_call_handler() range, uses
stacktrace-depth=32, and injects fail-nth=1 while writing 8191 bytes to
/proc/sys/kernel/domainname from a task in the target cgroup. Under
that setup, fail-nth=1 triggered the fault:
Dawei Feng [Wed, 3 Jun 2026 10:53:15 +0000 (18:53 +0800)]
bpf: NUL-terminate replaced sysctl value
When writing to sysctls, proc_sys_call_handler() guarantees that the
buffer passed to proc handlers is NUL-terminated. If
bpf_sysctl_set_new_value() replaces the pending sysctl value, it can
hand a replacement buffer directly to proc handlers. However, the
helper currently copies only buf_len bytes into that buffer without
appending a NUL terminator, leaving downstream parsers vulnerable to
out-of-bounds access.
Fix this by appending a '\0' after the replaced value to restore the
expected sysctl semantics. Since the helper already rejects buf_len
greater than PAGE_SIZE - 1, there is always room for the extra byte.
Reproduced in a QEMU x86_64 guest booted with KASAN while exercising
the sysctl replacement path with a cgroup/sysctl BPF program. The
reproducer targets `/proc/sys/net/core/flow_limit_cpu_bitmap`, fills
the original user write buffer with non-zero bytes, and overrides the
sysctl value so the replacement buffer lacks a terminating NUL. Under
that setup, the pre-fix kernel reported:
BUG: KASAN: slab-out-of-bounds in strnchrnul+0x72/0x90
Read of size 1 at addr ffff88800de57000 by task repro_patch3/66
CPU: 0 UID: 0 PID: 66 Comm: repro_patch3 Not tainted 7.1.0-rc3-00269-g8370ca1f87cc #6 PREEMPT(lazy)
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.15.0-1 04/01/2014
Call Trace:
<TASK>
dump_stack_lvl+0x68/0xa0
print_report+0xcb/0x5e0
? __virt_addr_valid+0x21d/0x3f0
? strnchrnul+0x72/0x90
? strnchrnul+0x72/0x90
kasan_report+0xca/0x100
? strnchrnul+0x72/0x90
strnchrnul+0x72/0x90
bitmap_parse+0x37/0x2e0
flow_limit_cpu_sysctl+0xc6/0x840
? __pfx_flow_limit_cpu_sysctl+0x10/0x10
? __kvmalloc_node_noprof+0x5ba/0x870
proc_sys_call_handler+0x31d/0x480
? __pfx_proc_sys_call_handler+0x10/0x10
? selinux_file_permission+0x39f/0x500
? lock_is_held_type+0x9e/0x120
vfs_write+0x98e/0x1000
...
</TASK>
The buggy address is located 0 bytes to the right of
allocated 4096-byte region [ffff88800de56000, ffff88800de57000)
With this fix applied, rerunning the same sysctl-targeted path yields
no corresponding KASAN reports.
====================
bpf: Update transport_header when encapsulating UDP tunnel in lwt
Currently, bpf_lwt_push_ip_encap() does not update skb->transport_header.
When a driver, e.g. ice, reuses the stale skb->transport_header to
offload checksum computation to NIC hardware, VxLAN packets encapsulated
by bpf_lwt_push_encap() helper may be dropped due to incorrect checksum.
Update skb->transport_header in bpf_lwt_push_ip_encap() whenever the
encapsulated packet uses UDP, so checksum offload works correctly.
Changes:
v3 -> v4:
* Address comments from Emil:
* Make the logic of skb_set_transport_header() clearer in patch #1.
* Fold the code of fexit_lwt_push_ip_encap() into test_lwt_ip_encap.c in
patch #2.
* Resolve assorted issues of test in patch #2.
* v3: https://lore.kernel.org/bpf/20260601150203.20352-1-leon.hwang@linux.dev/
v2 -> v3:
* Drop patch #1 and #2 of v2 that aim to resolve potential issues
reported by sashiko (per Alexei).
* Check target IP version and UDP tunnel in test (per sashiko).
* v2: https://lore.kernel.org/bpf/20260529151351.69911-1-leon.hwang@linux.dev/
v1 -> v2:
* Address sashiko's reviews:
* Fix TOCTOU issue in lwt to avoid changing hdr after checks.
* Add check iph->ihl < 5 in lwt to avoid infinite-loop in MIPS driver.
* Update comment style in selftests with BPF comment style.
* v1: https://lore.kernel.org/bpf/20260525142650.2569-1-leon.hwang@linux.dev/
====================
The unexpected offsets are: outer encap headers
(IPv4: iphdr+udp+vxlan+eth = 50 bytes, IPv6: ipv6hdr+udp+vxlan+eth = 70 bytes)
plus the inner IP header (20 or 40 bytes), because without the fix
transport_header still points at the inner transport layer instead of the
outer UDP header.
Leon Hwang [Tue, 2 Jun 2026 15:09:30 +0000 (23:09 +0800)]
bpf: Update transport_header when encapsulating UDP tunnel in lwt
Currently, bpf_lwt_push_ip_encap() does not update skb->transport_header.
When a driver, e.g. ice, reuses the stale skb->transport_header to
offload checksum computation to NIC hardware, VxLAN packets encapsulated
by bpf_lwt_push_encap() helper may be dropped due to incorrect checksum.
Update skb->transport_header in bpf_lwt_push_ip_encap() whenever the
encapsulated packet uses UDP, so checksum offload works correctly.
Fixes: 52f278774e79 ("bpf: implement BPF_LWT_ENCAP_IP mode in bpf_lwt_push_encap") Cc: Leon Hwang <leon.huangfu@shopee.com> Signed-off-by: Leon Hwang <leon.hwang@linux.dev> Link: https://lore.kernel.org/r/20260602150931.49629-2-leon.hwang@linux.dev Signed-off-by: Alexei Starovoitov <ast@kernel.org>
====================
RISC-V JIT support for bpf_get_current_task/_btf
These two patches add support for the bpf_get_current_task and
bpf_get_current_task_btf kfuncs in RISC-V JIT and add a selftest.
The first patch adds support for cpu and feature detection on
the JIT disassembly helper function as RISC-V JITed code was not
being disassembled using `LLVMCreateDisasm` as it was missing the
"+c" CPU feature and JITed code contained RISC-V Compressed (C)
Extension. This patch generalizes that to detect CPU features and
enables testing on more RISC-V JIT work ahead.
The second patch, which actually adds this support has been benchmarked
on QEMU RISC-V and shows significant improvements.
It was benchmarked using a simple loop inside a bpf program that ran
bpf_get_current_task() and execution time was measured. It used
bpf_prog_test_run_opts() to repeatedly trigger the BPF program. The loop
ran in 1 second intervals and it kept firing bpf_prog_test_run_opts() as
fast as possible until a second had elapsed and then reported statistics.
====================
Varun R Mallya [Tue, 2 Jun 2026 20:58:47 +0000 (02:28 +0530)]
bpf, riscv: inline bpf_get_current_task() and bpf_get_current_task_btf()
On RISC-V, the current task pointer is stored in the thread pointer
register (tp). Emit a single `mv a5, tp` instead of a full helper
call for BPF_FUNC_get_current_task and BPF_FUNC_get_current_task_btf.
Register bpf_jit_inlines_helper_call() entries for both helpers so the
verifier treats them as inlined, and add the expected `mv a5, tp`
annotation to the riscv64 selftests.
The following show changes before and after this patch.
Varun R Mallya [Tue, 2 Jun 2026 20:58:46 +0000 (02:28 +0530)]
selftests/bpf: use host CPU features in JIT disassembler
Pass the host CPU name and feature string to
LLVMCreateDisasmCPUFeatures() instead of using LLVMCreateDisasm(), so
the disassembler correctly decodes CPU-specific instructions and
extensions such as RISC-V compressed and vector instructions.
====================
bpf: Check tail zero of bpf_map_info and bpf_prog_info
Check the tail bytes of bpf_map_info and bpf_prog_info due to padding
when getting map info and prog info via BPF_OBJ_GET_INFO_BY_FD, which
was discussed in the thread
"bpf: Check tail zero of bpf_common_attr using offsetofend" [1].
Leon Hwang [Fri, 5 Jun 2026 15:52:49 +0000 (23:52 +0800)]
selftests/bpf: Add tests to verify checking padding bytes for bpf_[map,prog]_info
Add two tests to verify that the tail padding 4 bytes of struct
bpf_map_info and bpf_prog_info are checked in syscall.c using
bpf_check_uarg_tail_zero().
/* size: 232, cachelines: 4, members: 38 */
/* sum members: 224 */
/* sum bitfield members: 1 bits, bit holes: 1, sum bit holes: 31 bits */
/* padding: 4 */
/* forced alignments: 9 */
/* last cacheline: 40 bytes */
} __attribute__((__aligned__(8)));
If a future kernel extension adds a new 4-byte field, older userspace
programs allocating this structure on the stack might inadvertently pass
uninitialized stack garbage into the new field, permanently breaking
backward compatibility. -- sashiko [1]
Fix it by changing sizeof(info) to
offsetofend(struct bpf_prog_info, attach_btf_id).
And, add "__u32 :32" to the tail of struct bpf_prog_info.
If a future kernel extension adds a new 4-byte field, older userspace
programs allocating this structure on the stack might inadvertently pass
uninitialized stack garbage into the new field, permanently breaking
backward compatibility. -- sashiko [1]
Fix it by changing sizeof(info) to
offsetofend(struct bpf_map_info, hash_size).
And, add "__u32 :32" to the tail of struct bpf_map_info.
====================
selftests/bpf: Tolerate partial builds across kernel configs
Currently the BPF selftests can only be built by using the minimum kernel
configuration defined in tools/testing/selftests/bpf/config*. This poses a
problem in distribution kernels that may have some of the flags disabled or
set as module. For example, we have been running the tests regularly in
openSUSE Tumbleweed [1] [2] but to work around this fact we created a
special package [3] that build the tests against an auxiliary vmlinux with
the BPF Kconfig. We keep a list of known issues that may happen due to,
amongst other things, configuration mismatches [4] [5].
The maintenance of this package is far from ideal, especially for
enterprise kernels. The goal of this series is to enable the common usecase
of running the following in any system:
As an example, the following script targeting a minimal config can be used
for testing:
```sh
make defconfig
scripts/config --file .config \
--enable DEBUG_INFO_DWARF_TOOLCHAIN_DEFAULT \
--enable DEBUG_INFO_BTF \
--enable BPF_SYSCALL \
--enable BPF_JIT
make olddefconfig
make -j$(nproc)
make -j$(nproc) -C tools/testing/selftests install \
SKIP_TARGETS= \
TARGETS=bpf \
BPF_STRICT_BUILD=0
```
This produces a test_progs binary with 592 subtests, against the total of
727. Many of them will still fail or be skipped at runtime due to lack of
symbols, but at least there will be a clear way of building the tests.
[1]: https://openqa.opensuse.org/tests/5811715
[2]: https://openqa.opensuse.org/tests/5811730
[3]: https://src.opensuse.org/rmarliere/kselftests
[4]: https://github.com/openSUSE/kernel-qe/blob/main/kselftests_known_issues.yaml
[5]: https://openqa.opensuse.org/tests/5811730/logfile?filename=run_kselftests-config_mismatches.txt
---
Changes in v12:
- Rebase from 11 to 8 commits: split the test_kmods KDIR patch into a
pure KDIR fix (no PERMISSIVE) and a separate tolerance patch; squash
the BPF_STRICT_BUILD toggle with its first PERMISSIVE user; squash
the four test_progs partial-build patches into one; move the install
fix to immediately follow the first PERMISSIVE user
- Link to v11: https://patch.msgid.link/20260430-selftests-bpf_misconfig-v11-0-e11f7a8c4fdc@suse.com
Changes in v11:
- Gate the BTFIDS pretty-print on $(filter 1,$(V)) so V=0 and V=2 still
print the marker; only V=1 suppresses it (patch 6)
- Use asm volatile ("") in the uprobe_multi_func_{1,2,3} weak stubs to
match the strong definitions in prog_tests/uprobe_multi_test.c
(patch 10)
- Link to v10: https://patch.msgid.link/20260430-selftests-bpf_misconfig-v10-0-cd302a31af16@suse.com
Changes in v10:
- Drop $(wildcard $^) from the bench link recipe so cc fails cleanly on
the first missing input and the || fallback emits one SKIP-LINK line
instead of a wall of undefined-reference errors; also makes make -n
accurate (patch 9)
- Include <alloca.h> in testing_helpers.c for alloca() (patch 10)
- Link to v9: https://patch.msgid.link/20260429-selftests-bpf_misconfig-v9-0-c311f06b4791@suse.com
Changes in v9:
- Also pass KBUILD_OUTPUT=$(KMOD_O_VALID) when invoking kbuild so an
inherited command-line KBUILD_OUTPUT cannot disagree with O= (patch 2)
- Note BPFOBJ remains a normal prereq intentionally (patch 5)
- Sub-shell isolate cd/CC and use $@ for $(RM); gate the BTFIDS
pretty-print on $(V) so verbose mode does not double-print (patch 6)
- Restrict permissive compile-failure tolerance and partial-link to
test_progs%; runners with strong cross-object references (test_maps)
keep strict semantics (patches 6 and 8)
- Link to v8: https://patch.msgid.link/20260428-selftests-bpf_misconfig-v8-0-bf02cf97dbcb@suse.com
Changes in v8:
- In permissive mode, keep source changes to already-built tests
triggering relinks, while using recipe-time $(wildcard ...) to pick
up fresh .test.o files without duplicate linker inputs (patch 8)
- Resolve relative O=/KBUILD_OUTPUT before recursing into test_kmods,
and only treat it as a kernel build dir when it contains
Module.symvers (patch 2)
- Note in the commit message that PERMISSIVE-mode bisecting here needs
the next two patches (patch 6)
- Clarify in the commit message that order-only skeleton prereqs do not
break the new-skel-via-local-header case (patch 5)
- Exclude not-built tests from the success count so summary and JSON
report them only as skipped (patch 7)
- Tighten commit messages for accuracy and clarity
- Link to v7: https://patch.msgid.link/20260416-selftests-bpf_misconfig-v7-0-a078e18012e4@suse.com
Changes in v7:
- Use $(abspath) for KMOD_O so relative O= paths resolve correctly
when make -C changes directory (patch 2)
- Guard make clean against missing KDIR unconditionally; there is
nothing to clean when kernel headers are absent (patch 2)
- Drop explicit $(TRUNNER_TEST_OBJS) from linker filter in strict mode;
$^ already contains them as normal prerequisites (patch 8)
- Link to v6: https://patch.msgid.link/20260416-selftests-bpf_misconfig-v6-0-7efeab504af1@suse.com
Changes in v6:
- Add --ignore-missing-args to -extras rsync so out-of-tree permissive
builds do not abort when .ko files are absent (patch 2)
- Use $(abspath) for KMOD_O so relative O= paths resolve correctly
when make -C changes directory (patch 2)
- Guard make clean against missing KDIR unconditionally so cleaning
does not abort when kernel headers are absent (patch 2)
- Remove stale skeleton headers in early-exit paths when .bpf.o is
missing on incremental builds (patch 3)
- Fix strict-mode skeleton rules: use && before temp file cleanup so
bpftool failures are not masked by rm -f exit code (patch 3)
- Track test filter selection separately from not_built so -t/-n flags
are respected for unbuilt tests (patch 7)
- Make TRUNNER_TEST_OBJS order-only only in permissive mode, preserving
incremental relinking in strict builds (patch 8)
- Drop explicit $(TRUNNER_TEST_OBJS) from linker filter in strict mode;
they are already in $^ as normal prerequisites (patch 8)
- Reorder: move skip-unbuilt-tests patch before partial-linking patch
for bisectability
- Link to v5: https://patch.msgid.link/20260415-selftests-bpf_misconfig-v5-0-03d0a52a898a@suse.com
Changes in v5:
- Add BPF_STRICT_BUILD toggle as patch 1 so every subsequent patch
gates tolerance behind PERMISSIVE from the start, making the series
bisectable with strict-by-default at every point
- Fix O= commit message; make parent cp conditional (patch 2)
- Tolerate linked skeleton failures (patch 3)
- Skip feature detection for emit_tests (patch 4)
- Clarify bench is all-or-nothing in commit message (patch 8)
- Move stack_mprotect() to testing_helpers.c, drop weak stubs (patch 9)
- Report not-built tests as "SKIP (not built)" in output (patch 10)
- Drop overly broad 2>/dev/null || true from install rsync; rely solely
on --ignore-missing-args which already handles absent files (patch 11)
- Link to v4: https://patch.msgid.link/20260406-selftests-bpf_misconfig-v4-0-9914f50efdf7@suse.com
Changes in v4:
- Drop the test_kmods kselftest module flow patch: lib.mk gen_mods_dir
invokes $(MAKE) -C $(TEST_GEN_MODS_DIR) without forwarding
RESOLVE_BTFIDS, breaking ASAN and GCC BPF CI builds (Makefile.modfinal
cannot find resolve_btfids in the kbuild output tree)
- Link to v3:
https://patch.msgid.link/20260406-selftests-bpf_misconfig-v3-0-587a1114263c@suse.com
Changes in v3:
- Split test_kmods patch into two: fix KDIR handling (O= passthrough,
EXTRA_CFLAGS/EXTRA_LDFLAGS clearing) and wire into lib.mk via
TEST_GEN_MODS_DIR
- Pass O= through to the kernel module build so artifacts land in the
output tree, not the source tree
- Clear EXTRA_CFLAGS and EXTRA_LDFLAGS when invoking the kernel build to
prevent host flags (e.g. -static) leaking into module compilation
- Replace the bespoke test_kmods pattern rule with lib.mk module
infrastructure (TEST_GEN_MODS_DIR); lib.mk now drives build and clean
lifecycle
- Make the .ko copy step resilient: emit SKIP instead of failing when a
module is absent
- Expand the uprobe weak stub comment in bpf_cookie.c to explain why
noinline is required
- Link to v2:
https://patch.msgid.link/20260403-selftests-bpf_misconfig-v2-0-f06700380a9d@suse.com
Changes in v2:
- Skip test_kmods build/clean when KDIR directory does not exist
- Use `Module.symvers` instead of `.config` for in-tree detection
- Fix skeleton order-only prereqs commit message
- Guard BTFIDS step when .test.o is absent
- Add `__weak stack_mprotect()` stubs in `bpf_cookie.c` and `iters.c`
- Link to v1:
https://patch.msgid.link/20260401-selftests-bpf_misconfig-v1-0-3ae42c0af76f@suse.com
Assisted-by: {codex,claude} Tested-by: Alan Maguire <alan.maguire@oracle.com> Signed-off-by: Ricardo B. Marliere <rbm@suse.com>
---
Ricardo B. Marlière (11):
selftests/bpf: Add BPF_STRICT_BUILD toggle
selftests/bpf: Fix test_kmods KDIR to honor O= and distro kernels
selftests/bpf: Tolerate BPF and skeleton generation failures
selftests/bpf: Avoid rebuilds when running emit_tests
selftests/bpf: Make skeleton headers order-only prerequisites of .test.d
selftests/bpf: Tolerate test file compilation failures
selftests/bpf: Skip tests whose objects were not built
selftests/bpf: Allow test_progs to link with a partial object set
selftests/bpf: Tolerate benchmark build failures
selftests/bpf: Provide weak definitions for cross-test functions
selftests/bpf: Tolerate missing files during install
selftests/bpf: Tolerate missing files during install
With partial builds, some TEST_GEN_FILES entries can be absent at install
time. rsync treats missing source arguments as fatal and aborts kselftest
installation.
Override INSTALL_SINGLE_RULE in selftests/bpf to use --ignore-missing-args,
while keeping the existing bpf-specific INSTALL_RULE extension logic. Also
add --ignore-missing-args to the TEST_INST_SUBDIRS rsync loop so that
subdirectories with no .bpf.o files (e.g. when a test runner flavor was
skipped) do not abort installation.
Note that the INSTALL_SINGLE_RULE override applies globally to all file
categories including static source files (TEST_PROGS, TEST_FILES). These
are version-controlled and should always be present, so the practical risk
is negligible.
selftests/bpf: Provide weak definitions for cross-test functions
Some test files reference functions defined in other translation units that
may not be compiled when skeletons are missing. Replace forward
declarations of uprobe_multi_func_{1,2,3}() with weak no-op stubs so the
linker resolves them regardless of which objects are present.
The stub bodies are `asm volatile ("")` rather than empty, matching the
shape of the strong definitions in prog_tests/uprobe_multi_test.c. This
keeps the weak and strong sides on the same footing for the optimiser
(noinline + asm-barrier), which is the form upstream already relies on
for these functions.
Move stack_mprotect() from test_lsm.c into testing_helpers.c so it is
always available. The previous weak-stub approach returned 0, which would
cause callers expecting -1/EPERM to fail their assertions
deterministically. Having the real implementation in a shared utility
avoids this problem entirely.
Include <alloca.h> for alloca() so the build does not rely on glibc's
implicit declaration via <stdlib.h>.
Benchmark objects depend on skeletons that may be missing when some BPF
programs fail to build. In that case, benchmark object compilation or final
bench linking should not abort the full selftests/bpf build.
Keep both steps non-fatal, emit SKIP-BENCH or SKIP-LINK, and remove failed
outputs so stale objects or binaries are not reused by later incremental
builds. Note that because bench.c statically references every benchmark via
extern symbols, partial linking is not possible: if any single benchmark
object fails, the entire bench binary is skipped. This is by design -- the
error handler catches all compilation failures including genuine ones, but
those are caught by full-config CI runs.
selftests/bpf: Allow test_progs to link with a partial object set
When individual test files are skipped due to compilation failures, their
.test.o files are absent. The linker step currently lists all expected
.test.o files as explicit prerequisites, so make considers any missing one
an error.
In permissive mode, declare the test objects that already exist on disk
(via parse-time $(wildcard ...)) as normal prerequisites of the binary so
that modifications to a test source still trigger a relink, and keep the
full TRUNNER_TEST_OBJS list as order-only prerequisites so that initial
fresh builds still produce them and missing objects do not abort the link.
The recipe filter is split per mode: in permissive mode it combines a
recipe-time $(wildcard ...) (which catches objects freshly produced via
the order-only path on a fresh build) with $(filter-out
$(TRUNNER_TEST_OBJS),$^) (which keeps the non-test inputs from $^ but
drops the parse-time wildcard duplicates). This avoids passing the same
.test.o twice to the linker while still presenting test objects before
libbpf.a so that GNU ld, which scans static archives left-to-right, pulls
in archive members referenced exclusively by test objects (e.g.
ring_buffer__new from ringbuf.c). In default (strict) mode the recipe
remains the simple $(filter %.a %.o,$^) since TRUNNER_TEST_OBJS is part
of $^ exactly once.
Gate the partial-link behavior on $(if $(filter test_progs%,$1),...) so
it only applies to test_progs and its flavors. test_maps and similar
runners using strong cross-object references would link-fail with a
partial set and intentionally retain strict link semantics.
Note: adding a brand-new test_*.c file in permissive mode requires
removing the binary (or a clean rebuild) before the new test is linked
in, because the parse-time $(wildcard ...) is evaluated when the Makefile
is read and will not yet see the new .test.o. This is acceptable since
permissive mode targets tolerant CI builds rather than incremental
development.
selftests/bpf: Skip tests whose objects were not built
When both run_test and run_serial_test are NULL (because the corresponding
.test.o was not compiled), mark the test as not built instead of fatally
aborting.
Report these tests as "SKIP (not built)" in per-test output and include
them in the skip count so they remain visible in CI results and JSON
output. The summary line shows the not-built count when nonzero:
Summary: 50/55 PASSED, 5 SKIPPED (3 not built), 0 FAILED
Tests filtered out by -t/-n remain invisible as before; only genuinely
unbuilt tests are surfaced.
selftests/bpf: Tolerate test file compilation failures
Individual test files may fail to compile when headers or kernel features
required by that test are absent. Currently this aborts the entire build.
Make the per-test compilation non-fatal: remove the output object on
failure and print a SKIP-TEST marker to stderr. Guard the BTFIDS
post-processing step so it is skipped when the object file is absent. The
linker step will later ignore absent objects, allowing the remaining tests
to build and run.
Group cd and CC in a sub-shell so a cd failure cannot leak into the
error-handling branch and operate in the original working directory; use
$@ (absolute path) for $(RM) so it cannot match an unrelated file there.
Replace the $(call msg,...) in the BTFIDS block with a plain printf
(the msg macro expands to @printf, which is a make-recipe construct and
is invalid inside a shell if-then-fi body) and gate the printf on
$(filter 1,$(V)) so verbose mode (V=1) does not double-print the line
that the recipe shell already echoes; non-verbose modes (V unset, V=0,
V=2, ...) still print the BTFIDS marker, matching the convention of the
shared msg macro.
Restrict tolerance to test_progs and its flavors via an inlined
$(if $(filter test_progs%,$1),$(if $(PERMISSIVE),...)) check: runners
with strong cross-object references (e.g. test_maps) would link-fail
with a partial object set, so they keep strict semantics even when
BPF_STRICT_BUILD=0. The check is inlined rather than stored in a helper
variable so $1 is substituted at $(call) time and the per-runner result
is baked into each recipe.
Note on bisectability: this change is gated entirely behind PERMISSIVE
for test_progs%, so default builds (BPF_STRICT_BUILD!=0) compile and
run identically at every commit in the series. Bisecting in PERMISSIVE
mode at this commit still requires the next two patches ("selftests/bpf:
Skip tests whose objects were not built" and "selftests/bpf: Allow
test_progs to link with a partial object set") to avoid the linker
rejecting missing objects and the runtime aborting on NULL function
pointers.
selftests/bpf: Make skeleton headers order-only prerequisites of .test.d
The .test.d dependency files are generated by the C preprocessor and list
the headers each test file actually #includes. Skeleton headers appear in
those generated lists, so the .test.o -> .skel.h dependency is already
tracked by the .d file content.
Making skeletons order-only prerequisites of .test.d means that a missing
or skipped skeleton does not prevent .test.d generation, and regenerating a
skeleton does not force .test.d to be recreated. This avoids unnecessary
recompilation and, more importantly, avoids build errors when a skeleton
was intentionally skipped due to a BPF compilation failure.
$$(BPFOBJ) is intentionally kept as a normal prerequisite: a libbpf
rebuild legitimately invalidates .test.d, since libbpf header changes
can affect the headers .test.o sees. Only the skeleton headers are
moved to order-only.
Note that adding a new BPF skeleton via a modified existing local header
still works correctly: GNU make builds order-only prerequisites that do
not exist (the order-only qualifier only suppresses timestamp-driven
rebuilds, not existence-driven builds), so a brand-new .skel.h listed in
TRUNNER_BPF_SKELS is generated even when .test.d is otherwise up to date.
The modified local header invalidates .test.o through the previously
included .d content, forcing a recompile that regenerates .test.d with
the new .skel.h dependency captured by gcc -MMD.
selftests/bpf: Avoid rebuilds when running emit_tests
emit_tests is used while installing selftests to generate the kselftest
list. Pulling in .d files for this goal can trigger BPF rebuild rules and
mix build output into list generation.
Skip dependency file inclusion for emit_tests, like clean goals, so list
generation stays side-effect free. Also add emit_tests to
NON_CHECK_FEAT_TARGETS so that feature detection is skipped; without this,
Makefile.feature's $(info) output leaks into stdout and corrupts the test
list captured by the top-level selftests Makefile.
selftests/bpf: Tolerate BPF and skeleton generation failures
Some BPF programs cannot be built on distro kernels because required BTF
types or features are missing. A single failure currently aborts the
selftests/bpf build.
Make BPF object and skeleton generation best effort in permissive mode:
emit SKIP-BPF or SKIP-SKEL to stderr, remove failed outputs so downstream
rules can detect absence, and continue with remaining tests. Apply the same
tolerance to linked skeletons (TRUNNER_BPF_SKELS_LINKED), which depend on
multiple .bpf.o files and abort the build when any dependency is missing.
Note that progress messages (GEN-SKEL, LINK-BPF) are also redirected to
stderr as a side effect of rewriting the recipes into single-shell
pipelines; the $(call msg,...) macro is a make-recipe construct that cannot
be used inside an &&-chained shell command sequence.
selftests/bpf: Fix test_kmods KDIR to honor O= and distro kernels
test_kmods/Makefile always pointed KDIR at the kernel source tree root,
ignoring O= and KBUILD_OUTPUT. On distro kernels where the source tree has
not been built, the Makefile had no fallback and would fail
unconditionally.
When O= or KBUILD_OUTPUT is set and points at a prepared kernel build
directory (one containing Module.symvers), pass it through so kbuild can
locate the correct build infrastructure (scripts, Kconfig, etc.). Note
that the module artifacts themselves still land in the M= directory,
which is test_kmods/; O= only controls where kbuild finds its build
infrastructure. Fall back to /lib/modules/$(uname -r)/build when neither
an explicit valid build directory nor an in-tree Module.symvers is
present.
A selftests-only O= value (one that does not contain Module.symvers, e.g.
a private output directory) is intentionally not treated as a kernel
build directory. Without this guard, a user invoking
"make -C tools/testing/selftests/bpf O=/tmp/out" would have test_kmods
try to use /tmp/out as the kernel build dir and fail.
The parent bpf/Makefile resolves O= and KBUILD_OUTPUT to absolute paths
before invoking the test_kmods sub-make. Without this, $(abspath ...)
inside test_kmods/Makefile would resolve relative paths against the
sub-make's CWD (test_kmods/) rather than the user's invocation directory.
When O= is passed to kbuild, also pass KBUILD_OUTPUT=$(KMOD_O_VALID)
explicitly. The parent invocation lifts KBUILD_OUTPUT into MAKEFLAGS as
a command-line variable, which would otherwise suppress kbuild's own
"KBUILD_OUTPUT := $(O)" assignment and cause it to use the inherited
KBUILD_OUTPUT instead of the validated O=.
Guard both all and clean against a missing KDIR so the step is silently
skipped rather than fatal. Make the parent Makefile's cp conditional so it
does not abort when modules were not built.
Distro kernels often lack BTF types or kernel features required by some BPF
selftests, causing the build to abort on the first failure and preventing
the remaining tests from running.
Add BPF_STRICT_BUILD (default 1) to control build failure tolerance. When
set to 0, the PERMISSIVE make variable is assigned a non-empty value that
subsequent Makefile rules use to make individual build steps non-fatal.
When set to 1 (the default), the build fails on any error, preserving the
existing behavior for CI and direct builds.
Users can opt in to permissive mode on the command line:
make -C tools/testing/selftests \
TARGETS=bpf SKIP_TARGETS= BPF_STRICT_BUILD=0
Kaitao Cheng [Fri, 5 Jun 2026 09:41:43 +0000 (17:41 +0800)]
bpf: Clear rb node linkage when freeing bpf_rb_root
bpf_rb_root_free() detaches the root by copying the current rb_root_cached
and then replacing the live root with RB_ROOT_CACHED. It then walks the
copied root and drops each object contained in the tree.
This leaves the rb node state intact while dropping the object. If the
object is refcounted and survives the drop, its bpf_rb_node_kern still
contains an owner pointer to the freed root and stale rb tree linkage. If
a later bpf_rb_root allocation reuses the same address, bpf_rbtree_remove()
can incorrectly pass the owner check and call rb_erase_cached() on a node
whose rb pointers belong to the old tree.
Mirror the list draining behavior by marking nodes as busy while the root
is being detached, then clear the rb node and release the owner before
dropping the containing object. This makes surviving nodes unowned and
safe to reject from remove or accept for a later add.
The main patchset refactoring object relationship tracking in the
verifier has landed and this is a followup that addresses the remaining
feedback in v6 [0].
Amery Hung [Fri, 5 Jun 2026 20:20:56 +0000 (13:20 -0700)]
selftests/bpf: Use bpf_dynptr_slice() to read file dynptr in leak test
use_file_dynptr_slice_after_put_file() reads the dynptr via
bpf_dynptr_data(), which always returns NULL for a read-only file
dynptr, making the example confusing. Switch to bpf_dynptr_slice(), the
correct read API for file dynptrs, and read (rather than write) the slice
since it is read-only. The test still fails as expected.
Amery Hung [Fri, 5 Jun 2026 20:20:55 +0000 (13:20 -0700)]
bpf: Remove WARN_ON_ONCE in check_ids()
check_ids() warned when it ran out of idmap slots, assuming this was
impossible because the slots are bounded by the number of registers and
stack slots. That assumption no longer holds: referenced dynptrs acquire
an intermediate reference that lives in refs[] but is not backed by any
register or stack slot [0], so a program can accumulate more reference
ids than the idmap can hold and exhaust it.
Exhaustion is fine for verification correctness. check_ids() already
returns false, which makes the states compare as not equivalent and
prevents unsound pruning. The only effect of the WARN_ON_ONCE() is log
noise, or a panic under panic_on_warn. Drop the warning and keep
returning false.
Amery Hung [Fri, 5 Jun 2026 20:20:54 +0000 (13:20 -0700)]
bpf: Compare parent_id in refsafe() for REF_TYPE_PTR
refsafe() compared each reference's id and type but not its parent_id,
so two states whose PTR references differ only in the parent object they
were derived from could be wrongly treated as equivalent and pruned. Fix
it by checking parent_id too.
Amery Hung [Fri, 5 Jun 2026 20:20:53 +0000 (13:20 -0700)]
bpf: Check acquire_reference() error for "__ref" struct_ops arguments
When acquiring references for struct_ops program arguments tagged with
"__ref", the return value of acquire_reference() was stored directly
into u32 ctx_arg_info[i].ref_id without checking for failure.
acquire_reference() returns -ENOMEM when acquire_reference_state() fails
to allocate, so the error was silently stored as a ref_id instead of
aborting verification. Fix it by checking the return.
Amery Hung [Fri, 5 Jun 2026 20:20:52 +0000 (13:20 -0700)]
bpf: Fix dead error check on acquire_reference() in check_kfunc_call
acquire_reference() returns a signed int that may be a negative errno
but was converted to unsigned, which makes the subsequent error check
deadcode. Fix it by declaring 'id' as int so the error path is taken
correctly.
Ian Rogers [Sun, 31 May 2026 01:07:50 +0000 (18:07 -0700)]
bpftool: Restrict feature tests during bootstrap compilation
When the perf build executes 'make -C ../bpf/bpftool bootstrap', bpftool's
Makefile unconditionally evaluated feature checks for llvm, libcap, libbfd,
and disassembler libraries because the bootstrap target was not exempted.
Since the bootstrap bpftool strictly compiles minimal AST parsing and C
code generation logic without linking LLVM or disassembler libraries, these
feature check sub-makes are completely redundant.
Exempt the bootstrap target from non-essential feature tests to eliminate
unneeded sub-make fork overhead during Kbuild startup.
Tested-by: James Clark <james.clark@linaro.org> Assisted-by: Gemini:gemini-3.1-pro-preview Signed-off-by: Ian Rogers <irogers@google.com> Acked-by: Quentin Monnet <qmo@kernel.org> Link: https://lore.kernel.org/r/20260531010750.525160-1-irogers@google.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Mykyta Yatsenko [Wed, 3 Jun 2026 14:39:15 +0000 (07:39 -0700)]
selftests/bpf: Fix flaky file_reader test
file_reader/on_open_expect_fault test expects page fault
when reading pages from the test harness executable.
It is not guaranteed that those are paged out, even
after madvise(MADV_PAGEOUT).
Relax the condition in the test to succeed with both
0 and -EFAULT returned.
Tejun Heo [Mon, 1 Jun 2026 18:37:28 +0000 (08:37 -1000)]
bpf: Replace scratch PTE atomically when allocating arena pages
apply_range_set_cb() maps the pages for a new arena allocation and returned
-EBUSY when the target PTE was already populated. Kernel-fault recovery
leaves the per-arena scratch page in unallocated arena PTEs, so a later
bpf_arena_alloc_pages() over such a page hits that -EBUSY, and every
subsequent allocation of it fails the same way. Allocation must install the
real page over scratch instead.
Overwriting the scratch PTE in place is a valid->valid change, which arm64
forbids without break-before-make. Route through an invalid entry instead:
ptep_try_set() fills only a none slot, so the PTE goes scratch->none->page.
On finding scratch, clear it and flush_tlb_before_set() before retrying. The
new flush_tlb_before_set() is a no-op except on arches like arm64 that need
the break-before-make TLB invalidate. The loop also copes with a concurrent
fault re-scratching the slot.
Arches without ptep_try_set() never install the scratch page, so keep the
must-be-empty check and set_pte_at() for them.
Zhao Zhang [Tue, 2 Jun 2026 08:43:33 +0000 (16:43 +0800)]
bpf: Reject fragmented frames in devmap
Devmap broadcast redirects clone the packet for all but the last
destination.
For native XDP, that clone path copies only the linear xdp_frame data,
while fragmented frames keep skb_shared_info in tailroom outside the
linear area. Cloning such a frame leaves XDP_FLAGS_HAS_FRAGS set but
without valid frag metadata, and the later free path can interpret
uninitialized tail data as skb_shared_info, leading to an out-of-bounds
access during frame return.
Reject fragmented native XDP frames in dev_map_enqueue_clone().
Add the same restriction to the generic XDP clone path in
dev_map_redirect_clone(). Generic XDP represents fragmented packets as
nonlinear skbs, and rejecting them here keeps clone-based broadcast
support aligned between native and generic XDP.
Song Chen [Wed, 3 Jun 2026 09:19:10 +0000 (17:19 +0800)]
bpf: Reject registration of duplicated kfunc
Search for duplicated kfunc in btf_vmlinux and btf_modules
before a kernel module attempts to register a kfunc.
If kfunc would shadow existing kfunc then pr_err() and
reject module loading.
Emil Tsalapatis [Thu, 4 Jun 2026 18:42:52 +0000 (14:42 -0400)]
MAINTAINERS: BPF: Add self as reviewer and run parse_maintainers.pl
Add myself as a reviewer for the BPF subsystem. While at it, run
./scripts/parse_maintainers.pl --order and reorder the BPF-related
entries in the file accordingly.
Signed-off-by: Emil Tsalapatis <emil@etsalapatis.com> Acked-by: Eduard Zingerman <eddyz87@gmail.com> Acked-by: Kumar Kartikeya Dwivedi <memxor@gmail.com> Acked-by: Jiri Olsa <jolsa@kernel.org> Link: https://lore.kernel.org/r/20260604184252.9917-1-emil@etsalapatis.com Signed-off-by: Alexei Starovoitov <ast@kernel.org>
This patch series introduces BPF_MAP_TYPE_RHASH, a new hash map type that
leverages the kernel's rhashtable to provide resizable hash map for BPF.
The existing BPF_MAP_TYPE_HASH uses a fixed number of buckets determined at
map creation time. While this works well for many use cases, it presents
challenges when:
1. The number of elements is unknown at creation time
2. The element count varies significantly during runtime
3. Memory efficiency is important (over-provisioning wastes memory,
under-provisioning hurts performance)
BPF_MAP_TYPE_RHASH addresses these issues by using rhashtable, which
automatically grows and shrinks based on load factor.
The implementation wraps the kernel's rhashtable with BPF map operations:
- Uses bpf_mem_alloc for RCU-safe memory management
- Supports all standard map operations (lookup, update, delete, get_next_key)
- Supports batch operations (lookup_batch, lookup_and_delete_batch)
- Supports BPF iterators for traversal
- Supports BPF_F_LOCK for spin locks in values
- Requires BPF_F_NO_PREALLOC flag (elements allocated on demand)
- In-place updates for improved performance.
- max_entries serves as a hard limit, not bucket count
- Uses bit_spin_lock() + local_irq_save() for bucket locking,
similar to existing BPF hashmap's raw_spin_lock_irqsave(), insertions and
deletes may fail.
- Iterations are best-effort, if resize, insertions, deletions take place
concurrently, iterations may visit same elements multiple times or skip
elements.
- Lock out insertions, when running special fields destructor to guarantee
its completion.
The series includes comprehensive tests:
- Basic operations in test_maps (lookup, update, delete, get_next_key)
- BPF program tests for lookup/update/delete semantics
- Seq file tests
Signed-off-by: Mykyta Yatsenko <yatsenko@meta.com>
---
Update implementation
---------------------
Current implementation of the BPF_MAP_TYPE_RHASH does not provide
the same strong guarantees on the values consistency under concurrent
reads/writes as BPF_MAP_TYPE_HASH.
BPF_MAP_TYPE_HASH allocates a new element and atomically swaps the
pointer. BPF_MAP_TYPE_RHASH does memcpy in place with no lock held.
rhash trades consistency for speed, concurrent readers can observe
partially updated data. Two concurrent writers to the same key can
also interleave, producing mixed values. This is similar to arraymap
update implementation, including handling of the special fields.
As a solution, user may use BPF_F_LOCK to guarantee consistent reads
and write serialization.
Pattern:
* Small map (1K): htab wins for 8 / 32 byte keys by 10-20 %
because the preallocated bucket array fits in L1. Equalises
at 256 byte keys.
* Large map (1M): rhtab wins everywhere, up to 4x at high load
factor with 8 byte keys.
* Higher load factor amplifies rhtab's lead: rhtab grows the
bucket array; htab stays at user-declared max.
2. FULL UPDATE (M events/sec per producer, -p 7)
htab per-producer:
20.33 22.02 19.27 23.61 24.18 23.17 21.07
mean 21.94 range 19.27 - 24.18
rhtab per-producer:
133.51 129.47 74.52 129.29 102.26 129.98 107.64
mean 115.24 range 74.52 - 133.51
speedup (mean): 5.25x (+425 %)
In-place memcpy avoids the per-update alloc + RCU pointer swap
that htab pays.
3. MEMORY (overwrite, -p 8, no --preallocated)
value_size | htab ops/s | rhtab ops/s | htab mem | rhtab mem
-----------+-------------+-------------+----------+----------
32 B | 122.87 k/s | 133.04 k/s | 2.47 MiB | 2.49 MiB
4096 B | 64.43 k/s | 65.38 k/s | 6.74 MiB | 6.44 MiB
rhtab/htab : +8 % ops, +0.8 % mem (32 B)
+1 % ops, -4 % mem (4096 B)
SUMMARY
* Small / well-fitting map: htab is faster (cache-friendly
fixed bucket array), but only by ~10-20 %.
* Large / high-load-factor map: rhtab is dramatically faster
(1.2x to 4x) because rhashtable resizes to keep the load
factor sane while htab stays stuck at user-declared max.
* Update-heavy workloads: rhtab is ~5x faster per producer
via in-place memcpy.
* Memory benchmark: effectively on par
---
Changes in v7:
- rhashtable_next_key: move into lib/rhashtable.c, drop params argument
(Herbert).
- rhashtable_next_key: kdoc clarifies that behavior on tables with
duplicate keys is undefined (sashiko).
- rhashtable: include Herbert's "Use irq work for shrinking" patch so
__rhashtable_remove_fast_one() can fire the shrink path from NMI
context (Herbert).
- hashtab: fix u32 multiply overflow in __rhtab_map_lookup_and_delete_batch
copy_to_user; cast total to size_t before multiplying by key_size /
value_size (sashiko, bot+bpf-ci).
- hashtab: allow kptr/refcount fields in rhtab values (same model as
array map).
- Link to v6: https://patch.msgid.link/20260602-rhash-v6-0-1bfd35a4184f@meta.com
Changes in v6:
- rhashtable_next_key: advance past duplicate keys in the main bucket
chain to avoid an infinite loop when there are duplicate keys
(sashiko).
- rhashtable_next_key: return ERR_PTR(-EOPNOTSUPP) on rhltable (sashiko).
- rhashtable: selftest pre-sizes the table to avoid concurrent rehash
triggering spurious failures (sashiko).
- hashtab: real rhtab_map_mem_usage in the basic commit; move
bpf_map_free_internal_structs from rhtab_free_elem into the
special-fields commit where it does meaningful work (bot+bpf-ci).
- bpf_iter (seq_file): switch to rhashtable_walk_* for stronger
coverage under concurrent rehash; get_next_key and batch keep
rhashtable_next_key (sashiko).
- iter ops: rhtab_map_get_next_key adds IS_ERR check
before dereferencing the element pointer (sashiko).
- iter ops: bpf_each_rhash_elem removes cond_resched() (sashiko).
- iter ops: batch returns -EAGAIN (not -ENOENT) on cursor delete,
so userspace can distinguish lost cursor from end-of-iteration
and restart from NULL (sashiko).
- Link to v5: https://patch.msgid.link/20260528-rhash-v5-0-7205191b6c57@meta.com
Changes in v5:
- rhashtable_next_key: add kdoc WARNING to highlight lack of rehash
detection and unbounded iteration (Herbert).
- rhashtable: selftest now checks IS_ERR() before PTR_ERR comparison
on the missing-key path (bot+bpf-ci).
- hashtab: drop dead stub bodies and unused map_ops registrations
from the basic commit; iteration commit adds bodies, structs, and
registrations together. .map_get_next_key keeps a stub registration
in the basic commit because the syscall dispatcher does not
NULL-check it; iteration commit replaces the stub body with the
real implementation (bot+bpf-ci).
- hashtab: fix batch cursor advancement. v4 stashed the lookahead
element key but then resumed via next_key(cursor), skipping that
element across batch boundaries and orphaning it on
lookup_and_delete_batch. v5 stashes the lookahead key and looks
it up directly on the next batch entry (bot+bpf-ci, sashiko v3).
- hashtab: document torn-read race in rhtab_map_update_existing,
matching arraymap semantics (bot+bpf-ci).
- Link to v4: https://patch.msgid.link/20260513-rhash-v4-0-dd3d541ccb0b@meta.com
Changes in v4:
- rhashtable: introduce rhashtable_next_key(), drop walker-based
iteration for BPF (also drops earlier rhashtable_walk_enter_from()
proposal).
- map_extra: presize hint via lower 32 bits (nelem_hint), capped at
U16_MAX.
- Automatic shrinking enabled (was missing despite being advertised).
- Reject key_size > U16_MAX (rhashtable_params.key_len is u16).
- Replace irqs_disabled() guard with bpf_disable_instrumentation around
bucket-lock paths: closes same-CPU NMI tracing recursion without
rejecting legitimate IRQ-context callers.
- lookup_and_delete reordered: unlink before copy to avoid populating
user buffer on concurrent-unlink -ENOENT.
- update_existing reordered: copy then free_fields, matching arraymap.
- Word-sized key fast path (sizeof(long) bytes), inlined hashfn/cmpfn
via static-const rhashtable_params; works on both 32-bit and 64-bit.
- check_and_init_map_value() on insert (zero special-field bytes from
recycled bpf_mem_alloc memory; previously bpf_spin_lock could read
garbage and qspinlock would deadlock).
- BPF_SPIN_LOCK / BPF_RES_SPIN_LOCK allowlist moved to the special-
fields commit so each commit is bisect-safe.
- Link to v3: https://patch.msgid.link/20260424-rhash-v3-0-d0fa0ce4379b@meta.com
Changes in v3:
- Squash all commits implementing basic functions into one (Alexei)
- Remove selftests that were not necessary (Alexei)
- Resize detection for kernel full iterations, error out on resize (Alexei)
- Remove second lookup in get_next_key() (Emil)
- __acquires(RCU)/__releases(RCU) on seq_start/seq_stop (Emil)
- Use bpf_map_check_op_flags() where it makes sense (Leon)
- Benchmarks refresh, experiment with alternative hash functions
- Rely on iterator invalidation during rehash to handle table resizes:
fail on resize where we fully iterate on table inside kernel, dont fail on
resize where iteration goes through userspace. Exception -
rhtab_map_free_internal_structs() should be just safe to iterate fully
in kernel, no risk of infinite loop, because no user holding reference.
- Handle special fields during in-place updates (Emil, sashiko)
- Link to v2: https://lore.kernel.org/all/20260408-rhash-v2-0-3b3675da1f6e@meta.com/
Changes in v2:
- Added benchmarks
- Reworked all functions that walk the rhashtable, use walk API, instead
of directly accessing tbl and future_tbl
- Added rhashtable_walk_enter_from() into rhashtable to support O(1)
iteration continuations
- Link to v1: https://lore.kernel.org/r/20260205-rhash-v1-0-30dd6d63c462@meta.com
Pattern:
* Small map (1K): htab wins for 8 / 32 byte keys by 10-20%
* Large map (1M): rhtab wins everywhere, up to 4x at high load
factor with 8 byte keys.
* Higher load factor amplifies rhtab's lead: rhtab grows the
bucket array; htab stays at user-declared max.
2. FULL UPDATE (M events/sec per producer)
htab per-producer:
20.33 22.02 19.27 23.61 24.18 23.17 21.07
mean 21.94 range 19.27 - 24.18
rhtab per-producer:
133.51 129.47 74.52 129.29 102.26 129.98 107.64
mean 115.24 range 74.52 - 133.51
speedup (mean): 5.25x (+425 %)
In-place memcpy avoids the per-update alloc + RCU pointer swap
that htab pays.
3. MEMORY
value_size | htab ops/s | rhtab ops/s | htab mem | rhtab mem
-----------+-------------+-------------+----------+----------
32 B | 122.87 k/s | 133.04 k/s | 2.47 MiB | 2.49 MiB
4096 B | 64.43 k/s | 65.38 k/s | 6.74 MiB | 6.44 MiB
rhtab/htab : +8 % ops, +0.8 % mem (32 B)
+1 % ops, -4 % mem (4096 B)
Throughput effectively tied
SUMMARY
* Small / well-fitting map: htab is faster (cache-friendly
fixed bucket array), but only by ~10-20 %.
* Large / high-load-factor map: rhtab is dramatically faster
(1.2x to 4x) because rhashtable resizes to keep the load
factor sane while htab stays stuck at user-declared max.
* Update-heavy workloads: rhtab is ~5x faster per producer
via in-place memcpy.
* Memory benchmark: effectively on par.
Mykyta Yatsenko [Fri, 5 Jun 2026 11:41:26 +0000 (04:41 -0700)]
selftests/bpf: Add basic tests for resizable hash map
Test basic map operations (lookup, update, delete) for
BPF_MAP_TYPE_RHASH including boundary conditions like duplicate
key insertion and deletion of nonexistent keys.
Mykyta Yatsenko [Fri, 5 Jun 2026 11:41:24 +0000 (04:41 -0700)]
bpf: Optimize word-sized keys for resizable hashtable
Specialize the lookup/update/delete paths for keys whose size matches
sizeof(long) (4 bytes on 32-bit, 8 bytes on 64-bit). A static-const
rhashtable_params lets the compiler inline a custom XOR-fold hashfn and
a single-word equality cmpfn, eliminating the indirect jhash dispatch.
The same hashfn and cmpfn are installed into rhashtable's stored params
at rhashtable_init time, so the rehash worker, slow-path inserts, and
rhashtable_next_key() all agree with the inlined fast paths.
The seq_file BPF iterator uses rhashtable_walk_* and is unaffected.
Mykyta Yatsenko [Fri, 5 Jun 2026 11:41:23 +0000 (04:41 -0700)]
bpf: Allow special fields in resizable hashtab
Add support for timers, workqueues, task work, spin locks and kptrs.
Without this, users needing deferred callbacks, BPF_F_LOCK, or
refcounted kernel pointers in a dynamically-sized map have no option -
fixed-size htab is the only map supporting these field types.
Resizable hashtab should offer the same capability.
kptr semantics under in-place updates are identical to array map.
Properly clean up BTF record fields on element delete and map
teardown by wiring up bpf_obj_free_fields through a memory allocator
destructor, matching the pattern used by htab for non-prealloc maps.
Mykyta Yatsenko [Fri, 5 Jun 2026 11:41:22 +0000 (04:41 -0700)]
bpf: Implement iteration ops for resizable hashtab
Implement get_next_key, batch lookup/lookup-and-delete, for_each_map_elem
callback, and the seq_file BPF iterator for BPF_MAP_TYPE_RHASH.
get_next_key() and batch use rhashtable_next_key() — stateless,
matches the syscall UAPI shape (no kernel-side iterator state).
get_next_key falls back to the first key when prev_key was
concurrently deleted (matches htab semantics). Batch reports
cursor loss as -EAGAIN so userspace can distinguish it from
end-of-iteration (-ENOENT) and restart from NULL.
The seq_file BPF iterator uses rhashtable_walk_* instead. It runs
only from read() syscall context, so the walker's spin_lock is
safe, and seq_file's per-fd state lets the walker handle rehash
correctly (retry on -EAGAIN) for stronger coverage than the
stateless API can provide.
Mykyta Yatsenko [Fri, 5 Jun 2026 11:41:21 +0000 (04:41 -0700)]
bpf: Implement resizable hashmap basic functions
Use rhashtable_lookup_likely() for lookups, rhashtable_remove_fast()
for deletes, and rhashtable_lookup_get_insert_fast() for inserts.
Updates modify values in place under RCU rather than allocating a
new element and swapping the pointer (as regular htab does). This
trades read consistency for performance: concurrent readers may
see partial updates. BPF_F_LOCK support and special-field
handling (timers, kptrs, etc.) follow in a later commit.
Initialize rhashtable with bpf_mem_alloc element cache. Require
BPF_F_NO_PREALLOC. Limit max_entries to 2^31. Free elements via
rhashtable_free_and_destroy().
Mykyta Yatsenko [Fri, 5 Jun 2026 11:41:18 +0000 (04:41 -0700)]
rhashtable: Add rhashtable_next_key() API
Introduce a simpler iteration mechanism for rhashtable that lets
the caller continue from an arbitrary position by supplying the
previous key, without the per-iterator state of the
rhashtable_walk_* API.
Caller holds RCU; passes NULL prev_key for the first element or
the previously returned key to advance. Walks tbl->future_tbl
chain so in-flight rehashes are observed.
Best-effort: in case of concurrent resize, provides no guarantees:
- may produce duplicate elements
- may skip any amount of elements
- termination of the loop is not guaranteed in case of
sustained rehash. Callers are advised to bound loop externally
or avoid inserting new elements during such loop.
Returns ERR_PTR(-ENOENT) if prev_key is not found.
Behavior on tables with duplicate keys is undefined.
rhltable is not supported — returns ERR_PTR(-EOPNOTSUPP).
selftests/bpf: ignore call depth accounting for retbleed in verifier tests
When running the selftests on a retbleed-affected platform (eg:
Skylake), with call depth accounting enabled
(CONFIG_CALL_DEPTH_TRACKING=y) _and_ with retbleed=stuff, some verifier
selftests fail to validate the jited instructions. For example:
Those affected selftests allways fail on some call instruction: this
failure is due to the JIT compiler emitting call depth accounting for
retbleed mitigation (see x86_call_depth_emit_accounting calls in
bpf_jit_comp.c), resulting in an additional instruction being inserted
in front of every call instruction, similar to this one:
sarq $0x5, %gs:-0x39882741(%rip)
Fix those selftests by allowing them to ignore this possibly present
call depth accounting instruction.
zap_vma_range() requires the owning mm's mmap_lock to be held.
Taking mmap_read_lock under arena->lock would AB-BA against
arena_vm_close() and arena_map_mmap(), both of which run with
mmap_write_lock held and then acquire arena->lock. Instead drop
arena->lock, mmget_not_zero() the vma's mm, take mmap_read_lock, and
re-resolve the vma via find_vma() since it may have been unmapped or
replaced while waiting.
Track processed vmls with a per-call generation in vml->zap_gen and
serialize zap_pages() callers with a new arena->zap_mutex so
concurrent callers on different uaddr ranges do not mark each other's
vmls processed before the zap is done.
Matt Bobrowski [Wed, 3 Jun 2026 20:18:22 +0000 (20:18 +0000)]
bpf: clean up btf_scan_decl_tags()
Refactor the newly introduced btf_scan_decl_tags() to improve
readability and maintainability. The current implementation uses a
manual if-else chain and a magic number offset to strip the "arg:"
prefix from declaration tags.
Replace the if-else logic with a table-driven approach using a static
const array. This separates the tag data from the scanning logic, making
the helper more extensible for future tags. Additionally, replace the
magic number '4' with a sizeof-based calculation on the prefix string to
ensure the offset remains synchronized with the search key.
Finally, optimize the loop by moving the is_global check to the top of
the block. This allows the verifier to fail-fast on static subprograms
without performing unnecessary BTF string and type lookups.
Daniel Borkmann [Wed, 3 Jun 2026 21:16:58 +0000 (23:16 +0200)]
selftests/bpf: Test signed loader error paths
The positive path for signed BPF loaders is covered today by the
signed lskels (fentry_test, fexit_test, atomics).
But the runtime metadata check the generated loader performs (libbpf
gen_loader's emit_signature_match), the map content hash it relies
on, the load-time signature, and the immutability invariants of its
metadata map are not yet covered.
Thus, add a new, extensive test suite which drives libbpf's gen_loader
(bpf_object__gen_loader, gen_hash=true), the same machinery which
bpftool uses for signed light skeletons, and exercise corner cases
so that we can assert this in BPF CI:
map_excl exercises exclusive-map binding (allowed/denied), map-in-map
and map iterator rejection. It does not cover the create-time validation
of excl_prog_hash: the kernel only accepts a SHA-256-sized hash and
requires the pointer and size to be consistent.
Add map_excl_create_validation to check the rejected combinations:
Woojin Ji [Wed, 3 Jun 2026 00:33:39 +0000 (09:33 +0900)]
bpftool: Use libbpf error code for flow dissector query
bpf_prog_query() returns a negative errno on failure.
query_flow_dissector() currently closes the namespace fd and then reads
errno to decide whether -EINVAL means that the running kernel does not
support flow dissector queries.
That errno check controls behavior, not just diagnostics: -EINVAL is
handled as a non-fatal old-kernel case, while any other error makes bpftool
net fail.
The namespace fd is opened read-only, so close() is not expected to
commonly fail in normal use. Still, the BPF_PROG_QUERY error is already
available in err, and reading errno after an intervening close() is
fragile. If close() does change errno, the compatibility branch may be
based on close()'s error instead of the BPF_PROG_QUERY result.
This was reproduced with an LD_PRELOAD fault injector that forced
BPF_PROG_QUERY for BPF_FLOW_DISSECTOR to fail with EINVAL and then
forced close() on the netns fd to fail with EIO. The unpatched bpftool
reported "can't query prog: Input/output error". With this change, the
same injected failure is handled as the intended non-fatal EINVAL
compatibility case.
Use the libbpf-returned error code instead. Keep the existing errno reset
in the non-fatal path to preserve batch mode behavior. The success path
is unchanged.
Fixes: 7f0c57fec80f ("bpftool: show flow_dissector attachment status") Signed-off-by: Woojin Ji <random6.xyz@gmail.com> Signed-off-by: Andrii Nakryiko <andrii@kernel.org> Acked-by: Leon Hwang <leon.hwang@linux.dev> Acked-by: Yonghong Song <yonghong.song@linux.dev> Acked-by: Quentin Monnet <qmo@kernel.org> Link: https://lore.kernel.org/bpf/20260603003339.33791-1-random6.xyz@gmail.com Assisted-by: ChatGPT:gpt-5.5
Another small follow-up from the sashiko findings about signed loaders.
In particular, closing the gap to reject exclusive maps in iterators.
====================
Daniel Borkmann [Tue, 2 Jun 2026 13:30:52 +0000 (15:30 +0200)]
selftests/bpf: Test that exclusive maps are rejected as iter targets
Add a subtest to map_excl that creates an exclusive map and verifies a
bpf_map_elem iterator cannot be attached to it, which would otherwise
let an unrelated program read and overwrite the map's contents through
the iterator's writable value buffer.
sashiko complained that 38498c0ebacd ("selftests/bpf: Adjust verifier_map_ptr
for the map's excl field") would slightly decrease the test coverage given
before the test was against the verifier rejecting the ops pointer. Recover
the old test with the right offsets and add the existing one as an additional
test case.
# LDLIBS=-static PKG_CONFIG='pkg-config --static' ./vmtest.sh -- ./test_progs -t verifier_map_ptr
[ 1.672932] bpf_testmod: module verification failed: signature and/or required key missing - tainting kernel
#637/1 verifier_map_ptr/bpf_map_ptr: read with negative offset rejected:OK
#637/2 verifier_map_ptr/bpf_map_ptr: read with negative offset rejected @unpriv:OK
#637/3 verifier_map_ptr/bpf_map_ptr: write rejected:OK
#637/4 verifier_map_ptr/bpf_map_ptr: write rejected @unpriv:OK
#637/5 verifier_map_ptr/bpf_map_ptr: read non-existent field rejected:OK
#637/6 verifier_map_ptr/bpf_map_ptr: read non-existent field rejected @unpriv:OK
#637/7 verifier_map_ptr/bpf_map_ptr: read beyond excl field rejected:OK
#637/8 verifier_map_ptr/bpf_map_ptr: read beyond excl field rejected @unpriv:OK
#637/9 verifier_map_ptr/bpf_map_ptr: read ops field accepted:OK
#637/10 verifier_map_ptr/bpf_map_ptr: read ops field accepted @unpriv:OK
#637/11 verifier_map_ptr/bpf_map_ptr: r = 0, map_ptr = map_ptr + r:OK
#637/12 verifier_map_ptr/bpf_map_ptr: r = 0, map_ptr = map_ptr + r @unpriv:OK
#637/13 verifier_map_ptr/bpf_map_ptr: r = 0, r = r + map_ptr:OK
#637/14 verifier_map_ptr/bpf_map_ptr: r = 0, r = r + map_ptr @unpriv:OK
#637 verifier_map_ptr:OK
[...]
Summary: 2/20 PASSED, 0 SKIPPED, 0 FAILED
Daniel Borkmann [Tue, 2 Jun 2026 13:30:50 +0000 (15:30 +0200)]
libbpf: Guard add_data() against size overflow
add_data() computes size8 = roundup(size, 8) and then hands size8 to
realloc_data_buf() before doing memcpy(gen->data_cur, data, size) with
the original size. A wrapped size8 passes through the realloc_data_buf()
INT32_MAX check. Harden this against overflow, though not realistic to
happen in practice.
Daniel Borkmann [Tue, 2 Jun 2026 13:30:49 +0000 (15:30 +0200)]
bpf: Reject exclusive maps for bpf_map_elem iterators
Exclusive maps (aka excl_prog_hash) are meant to be reachable only
from the single program whose hash matches. This is enforced by
check_map_prog_compatibility() when the map is referenced from a
program such as signed BPF loaders.
A bpf_map_elem iterator, however, binds its target map at attach
time in bpf_iter_attach_map() instead of referencing it from the
program, so the exclusivity check is never reached. On top of that,
the iterator exposes the map value as a writable buffer.
bpf: fix UAF by restoring RCU-delayed inode freeing in bpffs
commit 4f375ade6aa9 ("bpf: Avoid RCU context warning when unpinning
htab with internal structs") moved inode cleanup from ->free_inode()
into ->destroy_inode() to avoid sleeping in RCU context when calling
bpf_any_put(). However this removed the RCU delay on freeing the
inode itself and the cached symlink body (i_link), both of which
can be accessed by RCU pathwalk (pick_link, may_lookup etc.).
This causes a use-after-free when a concurrent unlinkat() drops the
last inode reference and destroy_inode() frees the inode immediately,
while another task is still walking the path in RCU mode and reads
inode->i_opflags (offset +2) inside current_time() -> is_mgtime().
KASAN reports:
BUG: KASAN: slab-use-after-free in is_mgtime include/linux/fs.h:2313
Read of size 2 at addr ffff8880407e4282 (offset +2 = i_opflags)
The rules (per Al Viro):
->destroy_inode() called immediately, can sleep, use for blocking
cleanup e.g. bpf_any_put()
->free_inode() called after RCU grace period, use for freeing
inode and anything RCU-accessible e.g. i_link
Fix: split the two concerns properly:
- keep bpf_any_put() in bpf_destroy_inode() since it is blocking
and needs to run promptly
- introduce bpf_free_inode() to handle kfree(i_link) and
free_inode_nonrcu() with proper RCU delay, preventing the UAF
====================
Minimize annotations for arena programs
BPF programs must currently include code to address two limitations
of function signatures that include arena types. First, arena arguments
must be annotated with __arg_arena in the function signature in addition
to __arena. Second, it is currently not allowed to return an arena pointer
from a subprog, even though it is safe to do so. These limitations require
extra annotations and typecasts respectively, and have proven sources of
confusion to programmers.
The patchset improves arena-related function signatures in two ways.
First, it removes the need for __arg_arena in function signatures.
Second, it allows subprogs to directly return arena pointers to their
caller.
To do this we add a new type tag to the existing __arena annotation.
The annotation is currently an alias for __attribute__((address_space(1))),
which is not discoverable from BTF alone and so cannot be used to
determine whether a pointer variable is an arena pointer during
verification. With the new type tag, we can determine whether
either the arguments and or the return value of a function belong
in an arena.
We test the new code by modifying libarena to take advantage of these
relaxed limitations.
- Added Acks by Eduard
- Complete the __arg_arena removal by removing them from htab (Alexei)
- Add a test in verifier_arena_globals1.c to confirm the new __arena attribute
works as expected in function argument and return types
- Reject type tags on non-pointer types, currently only possible in handcrafted
BTF (Eduard)
- Undo inaccurate change on verifier comment (AI)
- Fix error return value for invalid BTF return types during BTF parsing (Eduard)
- Rebased to fix conflict
- Removed the typedef foo * foo_t typedefs. Those were necessary to avoid
annotating each instance of the type with __arena. The new version of the
patch instead removes typedefs and uses __arena everywhere directly (see
patch 4/5 for more details).
- Reorganized the patchset to frontload all kernel-side changes and place
the libarena changes at the end.
====================
Emil Tsalapatis [Tue, 2 Jun 2026 00:41:20 +0000 (20:41 -0400)]
selftests/bpf: Add tests for the new type-tag based __arena identifier
Add selftests that combine the new type-based __arena identifier with
the volatile qualifier both in functions' arguments and return values.
This way we test both that they are recognized as arena arguments and
that they are not sensitive to the position they are placed in the type
compared to other qualifiers.
Emil Tsalapatis [Tue, 2 Jun 2026 00:41:19 +0000 (20:41 -0400)]
selftests/bpf: libarena: Directly return arena pointers from functions
Now that the __arena annotation includes a BTF type tag, and the
verifier can identify arena pointers at BTF loading time, return
arena pointers as their true type instead of casting to u64. Remove the
preprocessor typecast wrappers used to hide this from the caller.
Emil Tsalapatis [Tue, 2 Jun 2026 00:41:18 +0000 (20:41 -0400)]
selftests/bpf: Remove __arg_arena from the codebase
Now that BPF __arg_arena has been subsumed by __arena, remove
__arg_arena from the codebase. This way the user has one fewer
annotation to worry about.
To remove __arg_arena we remove the typedefs we were previously
using to minimize __arena annotations. This is because __arena
now also includes a BTF type tag, which is ignored for non-pointer
types. As a result, we cannot capture the whole __arena annotation
inside a typedef and need to directly annotate the pointer type when
declaring the variable.
The extra verbosity is worth it because the use of the __arena tag
is intuitive to the programmer and removes the __arg_arena tag that
has been a consistent source of confusion for users. The typedefs
can be reintroduced later (without __arg_arena) once compilers start
supporting BTF type tags for non-pointer types.
Emil Tsalapatis [Tue, 2 Jun 2026 00:41:17 +0000 (20:41 -0400)]
bpf: Allow subprogs to return arena pointers
BPF subprogs currently only return void or scalar values. However,
it is also safe to return arena pointers between subprogs in the same
BPF program: Arena pointers are guaranteed to be safe for both programs
at any point. Expand the verifier to permit returning an arena pointer
to the caller.
The main subprog is still not allowed to return an arena pointer because
arena pointers are internal to the BPF program, and the return values
permitted for each main subprog depend on the program type anyway.
Emil Tsalapatis [Tue, 2 Jun 2026 00:41:16 +0000 (20:41 -0400)]
verifier: parse BTF type tags for function arguments
The BTF parsing logic for function arguments goes through
the arguments' decl tags, but does not go into their type
tags. Add type tag parsing for function arguments.
Emil Tsalapatis [Tue, 2 Jun 2026 00:41:15 +0000 (20:41 -0400)]
selftests/bpf: libarena: Add "arena" BTF type tag to __arena qualifier
The arena qualifier currently designates its associated type
as belonging to address space 1. This property affects code
generation, but is not reflected in the BTF information of
the function.
This lack of information at the BTF level prevents us from
returning arena pointers from global subprograms. Subprogs
cannot return any data structure more complex than a scalar,
so pointers to structs are rejected as a return type. We
have no way of marking the return type as a pointer to an
arena, which is safe provided the two subprogs have the same
arena.
Expand the __arena qualifier to also attach a BTF type tag
to the type. This lets us determine whether a variable belongs
to an arena from its type alone through BTF parsing.
Follow-up fixes for the signed loader, includes also the recent
sashiko findings.
v1->v2:
- Fixed up verifier_map_ptr selftest
- Added patch 1/2/6/7 with a new map-in-map fix and a
redundant hash_buf memcpy cleanup as well as selftests
====================
Daniel Borkmann [Mon, 1 Jun 2026 15:02:48 +0000 (17:02 +0200)]
selftests/bpf: Test that exclusive maps are rejected in map-in-map
Add a subtest to map_excl that verifies an exclusive map (created with
excl_prog_hash) cannot be used in a map-of-maps, covering both kernel
enforcement points: i) the inner-map template at map-of-maps creation
and, ii) the element inserted into an existing map-of-maps.
KP Singh [Mon, 1 Jun 2026 15:02:47 +0000 (17:02 +0200)]
selftests/bpf: Adjust verifier_map_ptr for the map's excl field
Adding the u32 excl field at offset 32 of struct bpf_map right after the
sha[SHA256_DIGEST_SIZE] hash shifts the ops pointer from offset 32 to 40.
Therefore, fix up the test case.
Daniel Borkmann [Mon, 1 Jun 2026 15:02:46 +0000 (17:02 +0200)]
libbpf: Skip max_entries override on signed loaders
bpf_gen__map_create() lets the host-supplied loader ctx override a
map's max_entries at runtime (map_desc[idx].max_entries, when non-zero).
This is how the light skeleton sizes maps to the target machine, but
it happens after emit_signature_match() and is covered by neither the
signed loader instructions nor the hashed blob.
For a signed loader this means an untrusted host can re-dimension the
program's maps, outside what the signature attests to. Gate the override
on gen_hash so signed loaders use the signer-provided max_entries baked
into the blob.
Daniel Borkmann [Mon, 1 Jun 2026 15:02:45 +0000 (17:02 +0200)]
libbpf: Skip initial_value override on signed loaders
bpf_gen__map_update_elem() emits code that, when the host-supplied
loader ctx provides a non-NULL map_desc[idx].initial_value, overwrites
the blob value with bytes read from the host (bpf_copy_from_user /
bpf_probe_read_kernel) before the BPF_MAP_UPDATE_ELEM that populates
the program's .data/.rodata/.bss maps.
This override runs after emit_signature_match() has validated map->sha[],
and initial_value is part of neither the signed loader instructions nor
the hashed data blob. For a signed loader this lets an untrusted host
substitute global-variable contents into a program whose code carries
a valid signature, thus weakening what the signature attests to.
The blob already contains the signer-provided value (added via add_data()
and covered by the embedded, signed hash), so simply skip emitting the
override for signed loaders (gen_hash). Runtime initialization stays
available for the unsigned light-skeleton path as before. The jump
offsets within the override block are internal to it, so guarding the
whole block leaves them unchanged.
KP Singh [Mon, 1 Jun 2026 15:02:44 +0000 (17:02 +0200)]
libbpf: Reject non-exclusive metadata maps in the signed loader
The loader verifies map->sha against the metadata hash in its
instructions. map->sha is calculated when BPF_OBJ_GET_INFO_BY_FD is
called on the frozen map.
While the map is frozen, the /signed loader/ must also ensure the map
is exclusive, as, without exclusivity (which a hostile host could just
omit when loading the loader), another BPF program with map access can
mutate the contents afterwards, so the check passes on stale data.
With the extra check as part of the signed loader, it now refuses to
move on with map->sha validation if the host set it up wrongly.
Fixes: fb2b0e290147 ("libbpf: Update light skeleton for signing") Signed-off-by: KP Singh <kpsingh@kernel.org> Co-developed-by: Daniel Borkmann <daniel@iogearbox.net> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Link: https://lore.kernel.org/r/20260601150248.394863-4-daniel@iogearbox.net Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Daniel Borkmann [Mon, 1 Jun 2026 15:02:43 +0000 (17:02 +0200)]
bpf: Drop redundant hash_buf from map_get_hash operation
bpf_map_get_info_by_fd() is the only caller of the ->map_get_hash
and always invokes it with hash_buf == map->sha and hash_buf_size
of SHA256_DIGEST_SIZE. array_map_get_hash() in turn lets sha256()
write the digest directly into that buffer (map->sha) and then
performs a trailing memcpy(), which evaluates to memcpy(map->sha,
map->sha, 32): a redundant self-copy. The hash_buf_size argument
was never used at all. Simplify this a bit, no functional change.
Daniel Borkmann [Mon, 1 Jun 2026 15:02:42 +0000 (17:02 +0200)]
bpf: Reject exclusive maps as inner maps in map-in-map
An exclusive map (created with excl_prog_hash) is bound to a single
program by hash: check_map_prog_compatibility() refuses to load any
program whose digest does not match map->excl_prog_sha. That check
only runs for maps a program references directly, i.e. its used_maps.
A map reached at runtime through a map-of-maps is never in used_maps,
and bpf_map_meta_equal() does not consider excl_prog_sha, so an
exclusive map can be inserted into a non-exclusive outer map and
then looked up and mutated by an unrelated program, bypassing the
exclusivity guarantee.
For the signed loader this defeats the metadata map exclusivity check
added in the signed loader: the cached map->sha[] is validated against
the signed hash while another program on a hostile host rewrites the
frozen map's contents through the outer map.
This patchset cleans up dynptr handling, refactors object relationship
tracking in the verifier by introducing parent_id and folding ref_obj_id
into id, and fixes dynptr use-after-free bugs where file/skb dynptrs
are not invalidated when the parent referenced object is freed.
* Motivation *
In BPF qdisc programs, an skb can be freed through kfuncs. However,
since dynptr does not track the parent referenced object (e.g., skb),
the verifier does not invalidate the dynptr after the skb is freed,
resulting in use-after-free. The same issue also affects file dynptr.
The figure below shows the current state of object tracking. The
verifier tracks objects using three fields: id for nullness tracking,
ref_obj_id for lifetime tracking, and dynptr_id for tracking the parent
dynptr of a slice (PTR_TO_MEM only). While dynptr_id links slices to
their parent dynptr, there is no field that links a dynptr back to its
parent skb. When the skb is freed via release_reference(ref_obj_id=1),
only objects with ref_obj_id=1 are invalidated. Since skb dynptr is
non-referenced (ref_obj_id=0), the dynptr and its derived slices remain
accessible.
Current: object (id, ref_obj_id, dynptr_id)
id = unique id of the object (for nullness tracking)
ref_obj_id = id of the referenced object (for lifetime tracking)
dynptr_id = id of the parent dynptr (only for PTR_TO_MEM slices)
skb (0,1,0)
^^
! No link from dynptr to skb !
|+------------------------------+
| bpf_dynptr_clone |
dynptr A (2,0,0) dynptr C (4,0,0)
^ ^
bpf_dynptr_slice | |
| |
slice B (3,0,2) slice D (5,0,4)
* Why not simply use ref_obj_id to track the parent? *
A natural first approach is to link dynptr to its parent by sharing
the parent's ref_obj_id and propagating it to slices. Now, releasing
the skb via release_reference(ref_obj_id=1) correctly invalidates all
derived objects.
Attempted fix: share parent's ref_obj_id
skb (0,1,0)
^^
||
|+------------------------------+
| bpf_dynptr_clone |
dynptr A (2,1,0) dynptr C (4,1,0)
^ ^
bpf_dynptr_slice | |
| |
slice B (3,1,2) slice D (5,1,4)
However, this approach does not generalize to all dynptr types.
Referenced dynptrs such as file dynptr acquire their own ref_obj_id to
track the dynptr's lifetime. Since ref_obj_id is already used for the
dynptr's own reference, it cannot also be used to point to the parent
file object. While it is possible to add specialized handling for
individual dynptr types [0], it adds complexity and does not generalize.
An alternative approach is to avoid introducing a new field and instead
repurpose ref_obj_id as parent_id by folding lifetime tracking into id
[1]. In this design, each object is represented as (id, ref_obj_id)
where id is used for both nullness and lifetime tracking, and ref_obj_id
tracks the parent object's id.
Attempted: object (id, ref_obj_id)
id = id of the object (for nullness and lifetime tracking)
ref_obj_id = id of the parent object
' = id is referenced
skb (1',0)
^^
||
bpf_dynptr_from_skb |+------------------------------+
| bpf_dynptr_clone(A, C) |
dynptr A (2,1') dynptr C (4,1')
^ ^
bpf_dynptr_slice | |
| |
slice B (3,2) slice D (5,4)
However, this design cannot express the relationship between referenced
socket pointers and their casted counterparts. After pointer casting,
the original and casted pointers need the same lifetime (same ref_obj_id
in the current design) but different nullness (different id). The casted
pointer may be NULL even if the original is valid. With id serving as
the only field for both nullness and lifetime, and ref_obj_id repurposed
as parent, there is no way to express "different identity, same
lifetime."
Referenced socket pointer (expressed using current design):
C = ptr_casting_function(A)
ptr A (1,1,0) ptr C (2,1,0)
^ ^
| |
ptr C may be NULL even if ptr A is valid
but they have the same lifetime
* New Design: parent_id with branch splitting and intermediate reference *
The patchset folds ref_obj_id into id and adds parent_id to
bpf_reg_state (patch 5). A child object's parent_id points to the
parent object's id. This replaces the PTR_TO_MEM-specific dynptr_id.
Whether a register is referenced is determined by checking if its id
appears in the reference array via reg_is_referenced() rather than
reading a dedicated ref_obj_id field.
Pointer casting:
The challenge with pointer casting is that a cast result may be NULL
even when the source is valid, requiring distinct identity but shared
lifetime. This is solved using branch splitting: when a helper like
bpf_sk_fullsock() is called with a referenced pointer, the verifier
pushes an explicit NULL branch and assigns the cast result the same id
as the source. Since the cast may return NULL for a non-NULL input, the
NULL case is explored as a separate verifier branch. This allows
releasing any of the original or cast pointers to invalidate all others,
while avoiding the need for a separate tracking mechanism.
Referenced dynptrs:
The challenge with referenced dynptrs is that clones of a referenced
dynptr have the same lifetime but different identities. When a
referenced dynptr is overwritten, only slices derived from it will be
invalidated. To solve this, the verifier creates an intermediate
reference. This reference serves as a shared lifetime anchor for the
dynptr and all its clones. All clones share the same parent_id but get
unique ids for independent slice tracking. Releasing a referenced dynptr
releases the intermediate reference, which in turn invalidates all
clones and their derived slices. If the parent object is released while
the intermediate reference still exists, it is reported as a leaked
reference.
Release cascading:
When releasing an object, release_reference() performs a stack-based DFS
to invalidate all descendants. It walks the object tree via parent_id
links, invalidating registers and dynptr stack slots. Child references
encountered during traversal are reported as leaked references.
parent_id is also added to bpf_reference_state to enable intermediate
reference. When acquiring a reference, a parent_id can be specified to
link the new reference to an existing one (e.g., file dynptr's
intermediate reference has parent_id linking to the file's reference).
Final: object (id, parent_id)
id = unique id of the object (for nullness and lifetime
tracking)
parent_id = id of the parent object (for object relationship
tracking)
I = intermediate reference serving as lifetime anchor in
acquired_refs
' = id is referenced (appears in reference array)
skb (1',0)
^^
||
bpf_dynptr_from_skb |+------------------------------+
| bpf_dynptr_clone(A, C) |
dynptr A (2,1') dynptr C (4,1')
^ ^
bpf_dynptr_slice | |
| |
slice B (3,2) slice D (5,4)
* Preserving reg->id after null-check *
For parent_id tracking to work, child objects need to refer to the
parent's id. This requires two preparatory changes: assigning reg->id
when reading referenced kptrs from program context (patch 3), and
preserving reg->id of pointer objects after null-check (patch 4).
Previously, null-check would clear reg->id, making it impossible for
children to reference the parent afterward. The latter causes a slight
increase in verified states for some programs. One selftest object
sees +19 states (+5.01%). For Meta BPF objects, the increase is
also minor, with the largest being +34 states (+3.63%).
* Object relationship in different scenarios (for reference) *
The figures below show how the final design handles all four
combinations of referenced/non-referenced dynptr with
referenced/non-referenced parent.
(1) Non-referenced dynptr with referenced parent (e.g., skb in Qdisc):
skb (1',0)
^^
||
bpf_dynptr_from_skb |+------------------------------+
| bpf_dynptr_clone(A, C) |
dynptr A (2,1') dynptr C (4,1')
dynptr A and C live independently
(2) Non-referenced dynptr with non-referenced parent (e.g., skb in TC,
always valid):
bpf_dynptr_from_skb
bpf_dynptr_clone(A, C)
dynptr A (1,0) dynptr C (2,0)
dynptr A and C live independently
(3) Referenced dynptr with referenced parent:
file (1',0)
^
bpf_dynptr_from_file |
I (2',1') <-- intermediate reference
^^
||
|+-------------------------------+
| bpf_dynptr_clone(A, C) |
dynptr A (3,2') dynptr C (4,2')
dynptr A and C have the same lifetime
Releasing either dynptr releases I, invalidating both.
Releasing file (1') detects I as a leaked reference.
(4) Referenced dynptr with non-referenced parent:
bpf_ringbuf_reserve_dynptr
I (1',0) <-- intermediate reference
^^
||
|+--------------------------------+
| bpf_dynptr_clone(A, C) |
dynptr A (2,1') dynptr C (3,1')
v5 -> v6
- Squash "bpf: Fold ref_obj_id into id and introduce virtual references"
(v5 patch 9) into "bpf: Refactor object relationship tracking and
fix dynptr UAF bug" (now patch 5). ref_obj_id is removed in the same
patch that introduces parent_id, eliminating the intermediate state
where both coexist (Eduard)
- Drop virtual references for pointer casting. Instead, cast results
reuse the source pointer's id and use branch splitting to explore
the NULL case as a separate verifier branch. This avoids adding
virtual reference infrastructure for a case that can be handled more
simply (Eduard, Andrii)
- Address nit from Eduard Link: https://lore.kernel.org/bpf/20260519181314.2731658-1-ameryhung@gmail.com/
v4 -> v5
- Add patch 9 folding ref_obj_id into id and introducing virtual
references for pointer casting and referenced dynptr clones (Eduard, Andrii)
- Add patch 10 fixing dynptr ref counting to scan all call frames
instead of only the current frame (Eduard)
- Add utility function validate_ref_obj() (Eduard) Link: https://lore.kernel.org/bpf/20260506142709.2298255-1-ameryhung@gmail.com/
v3 -> v4
- Add patch 1 clean up mark_stack_slot_obj_read() and callers
(to address v3 ignoring err returned from mark_dynptr_read) (Andrii)
- Fix release_reference() and move the logic allowing destroying a
referenced object when refcnt > 1 from
destroy_if_stack_slots_dynptr() to release_reference() (Mykyta)
- Add patch 7 introducing ref_obj_desc and unifying ref_obj handling
(to address Eduard's concern about unclear meta->{id,ref_obj_id}
initialization/use and confusing function arguments of
process_dynptr_func())
- Add patch 8 unifying release_regno handling so that bpf_kptr_xchg
also use release_reference() Link: https://lore.kernel.org/bpf/20260421221016.2967924-1-ameryhung@gmail.com/
v2 -> v3
- Rebase to bpf-next/master
- Update veristat numbers
- Update commit msg to explain multiple dropped checks (Mykyta, Andrii)
- Reuse idmap as idstack in release_reference() and check for
duplicate id (Mykyta, Andrii)
- Change to use RUN_TEST for qdisc dynptr selftest (Eduard) Link: https://lore.kernel.org/bpf/20260307064439.3247440-1-ameryhung@gmail.com/
v1 -> v2
- Redesign: Use object (id, ref_obj_id, parent_id) instead of
(id, ref_obj_id) as it cannot express ptr casting without
introducing specialized code to handle the case
- Use stack-based DFS to release objects to avoid recursion (Andrii)
- Keep reg->id after null check
- Add dynptr cleanup
- Fix dynptr kfunc arg type determination
- Add a file dynptr UAF selftest Link: https://lore.kernel.org/bpf/20260202214817.2853236-1-ameryhung@gmail.com/
---
====================
Amery Hung [Fri, 29 May 2026 01:49:35 +0000 (18:49 -0700)]
selftests/bpf: Test using file dynptr after the reference on file is dropped
File dynptr and slice should be invalidated when the parent file's
reference is dropped in the program. Without the verifier tracking
dyntpr's parent referenced object, the dynptr would continute to be
incorrectly used even if the underlying file is being tear down or gone.
Amery Hung [Fri, 29 May 2026 01:49:34 +0000 (18:49 -0700)]
selftests/bpf: Test using slice after invalidating dynptr clone
The parent object of a cloned dynptr is skb not the original dynptr.
Invalidate the original dynptr should not prevent the program from
using the slice derived from the cloned dynptr.
projects / thirdparty / linux.git / log
