From: Andrii Nakryiko Date: Thu, 26 Mar 2026 20:53:57 +0000 (-0700) Subject: Merge branch 'add-btf-layout-to-btf' X-Git-Url: http://git.ipfire.org/gitweb.cgi?a=commitdiff_plain;h=01504da43e67f823fbd22f4ec9e4469236a34b5c;p=thirdparty%2Fkernel%2Flinux.git Merge branch 'add-btf-layout-to-btf' Alan Maguire says: ==================== Add BTF layout to BTF Update struct btf_header to add a new "layout" section containing a description of how to parse the BTF kinds known about at BTF encoding time. This provides the opportunity for tools that might not know all of these kinds - as is the case when older tools run on more newly-generated BTF - to still parse the BTF provided, even if it cannot all be used. The ideas here were discussed at [1], with further discussion at [2]. Patches for pahole will enable the layout addition during BTF generation are at [3], but even absent these the addition of the layout feature in the final patch in this series should not break anything since such unknown features are simply ignored during pahole BTF generation. Separately tested sanitization of BTF location info with separate small series which simulates feature absence to support testing of features for older kernels; will follow up with that shortly. Changes since v15 [4]: - Fixed endian issues for layout section by swapping flags fields where needed (sashiko.dev, patch 2) - Fixed string size issue with swapped endian case, use btf->magic for comparison to determine endian mismatch (bpf review bot, sashiko.dev, patch 6) Changes since v14 [5]: - Fix potential overflow for swapped endian case (BPF review bot, patch 2) - Add global: keyword to libbpf.map (sashiko.dev, patch 4) - Fix endian issues in sanitization; we use the endian safe btf->hdr and check for endian mismatch between it and raw original BTF header to inform how we write the change str_off. Also fix potential truncation issues due to not including hdr->type_off (sashiko.dev, patch 6) - Fix issues with selftests raw BTF file interactions (sashiko.dev, patch 8) - Drop feature test test since it will be covered by another series Changes since v13: [6]: - add feature check/sanitization of BTF with layout info (Andrii, patch 6) - added feature check test for layout support (patch 9) Changes since v12: [7]: - add logging of layout off/len to kernel header logging (review bot, patch 6) - add mode to open() in selftest (review bot, patch 7) Changes since v11 [8]: - Revert unneeded changes to btf__new_empty() (Eduard, review bot, patch 4) - Reorder btf_parse_layout_sec() checks to ensure min size check occurs before multiple check (review bot, patch 6) Changes since v10 [9]: - deal with read overflow with small header (review bot, patch 2) - validate layout length is a multiple of sizeof(struct btf_layout) (review bot, patch 6) - fix comment style (Alexei, patches 4,7) - remove bpftool BTF metadata subcommands for now (Alexei) Changes since v9: [10]: - fix memcpy header size overrun (review bot, patch 2) - return size computation directly (Andrii, patch 333) - revert to original unknown kind logging (Alexei/review bot, patch 6) - gap-checking logic can be simplified now that we have 4-byte aligned types and layout together (patch 6) - fix naming of layout offset, unconditionally emit a layouts array in json (Quentin, review bot, patch 8) - fix metadata output in man page to include flags (review bot, patch 9) Changes since v8: [11]: - updated name from "kind_layout" to "layout" (Andrii) - moved layout info to inbetween types and strings since both types and layout info align on 4 bytes (Andrii) - use embedded btf_header (Eduard) - when consulting layout, fall back to base BTF if none found in split BTF; this will allow us to only encode layout info in vmlinux rather than repeating it for each module. Changes since v7: [12]: - Fixed comment style in UAPI headers (Mykyta, patch 1) - Simplify calcuation of header size using min() (Mykyta, patch 2) - simplify computation of bounds for kind (Mykyta, patch 3) - Added utility functions for updating type, string offsets when data is added; this simplifies the code and encapsulates such updates more clearly (patch 2) Changes since v6: [13]: - BPF review bot caught some memory leaks around freeing of kind layout; more importantly, it noted that we were breaking with the contiguous BTF representation for btf_new_empty_opts(). Doing so meant that freeing kind_layout could not be predicated on having btf->modifiable set, so adpoted the contiguous raw data layout for BTF to be consistent with type/string storage (patches 2,4) - Moved checks for kind overflow prior to referencing kinds to avoid any risk of overrun (patches 3, 8) - Tightened up kind layout header offset/len header validation to catch invalid combinations early in btf_parse_hdr() (patch 2) - Fixed selftest to verify calloc success (patch 7) Changes since v5: [14]: - removed flags field from kind layout; it is not really workable since we would have to define semantics of all possible future flags today to be usable. Instead stick to parsing only, which means each kind just needs the length of the singular and vlen-specified objects (Alexei) - added documentation for bpftool BTF metadata dump (Quentin, patch 9) Changes since v4: [15]: - removed CRC generation since it is not needed to handle modules built at different time than kernel; distilled base BTF supports this now - fixed up bpftool display of empty kind names, comment/documentation indentation (Quentin, patches 8, 9) Changes since v3 [16]: - fixed mismerge issues with kbuild changes for BTF generation (patches 9, 14) - fixed a few small issues in libbpf with kind layout representation (patches 2, 4) Changes since v2 [17]: - drop "optional" kind flag (Andrii, patch 1) - allocate "struct btf_header" for struct btf to ensure we can always access new fields (Andrii, patch 2) - use an internal BTF kind array in btf.c to simplify kind encoding (Andrii, patch 2) - drop use of kind layout information for in-kernel parsing, since the kernel needs to be strict in what it accepts (Andrii, patch 6) - added CRC verification for BTF objects and for matching with base object (Alexei, patches 7,8) - fixed bpftool json output (Quentin, patch 10) - added standalone module BTF support, tests (patches 13-17) Changes since RFC - Terminology change from meta -> kind_layout (Alexei and Andrii) - Simplify representation, removing meta header and just having kind layout section (Alexei) - Fixed bpftool to have JSON support, support prefix match, documented changes (Quentin) - Separated metadata opts into add_kind_layout and add_crc - Added additional positive/negative tests to cover basic unknown kind, one with an info_sz object following it and one with N elem_sz elements following it. - Updated pahole-flags to use help output rather than version to see if features are present [1] https://lore.kernel.org/bpf/CAEf4BzYjWHRdNNw4B=eOXOs_ONrDwrgX4bn=Nuc1g8JPFC34MA@mail.gmail.com/ [2] https://lore.kernel.org/bpf/20230531201936.1992188-1-alan.maguire@oracle.com/ [3] https://lore.kernel.org/dwarves/20260226085240.1908874-1-alan.maguire@oracle.com/ [4] https://lore.kernel.org/bpf/20260324174450.1570809-1-alan.maguire@oracle.com/ [5] https://lore.kernel.org/bpf/20260318132927.1142388-1-alan.maguire@oracle.com/ [6] https://lore.kernel.org/bpf/20260306113630.1281527-1-alan.maguire@oracle.com/ [7] https://lore.kernel.org/bpf/20260303182003.117483-1-alan.maguire@oracle.com/ [8] https://lore.kernel.org/bpf/20260302114059.3697879-1-alan.maguire@oracle.com/ [9] https://lore.kernel.org/bpf/20260227100426.2585191-1-alan.maguire@oracle.com/ [10] https://lore.kernel.org/bpf/20260226085624.1909682-1-alan.maguire@oracle.com/ [11] https://lore.kernel.org/bpf/20251215091730.1188790-1-alan.maguire@oracle.com/ [12] https://lore.kernel.org/dwarves/20251211164646.1219122-1-alan.maguire@oracle.com/ [13] https://lore.kernel.org/bpf/20251210203243.814529-1-alan.maguire@oracle.com/ [14] https://lore.kernel.org/bpf/20250528095743.791722-1-alan.maguire@oracle.com/ [15] https://lore.kernel.org/bpf/20231112124834.388735-1-alan.maguire@oracle.com/ [16] https://lore.kernel.org/bpf/20231110110304.63910-1-alan.maguire@oracle.com/ [17] https://lore.kernel.org/bpf/20230616171728.530116-1-alan.maguire@oracle.com/ ==================== Link: https://patch.msgid.link/20260326145444.2076244-1-alan.maguire@oracle.com Signed-off-by: Andrii Nakryiko --- 01504da43e67f823fbd22f4ec9e4469236a34b5c