Aaron Merey [Fri, 1 Mar 2024 00:46:09 +0000 (19:46 -0500)]
tests/run-getsrc-die.sh: Avoid using objcopy
run-getsrc-die.sh uses objcopy to remove .debug_aranges from some
testfiles. However depending how objcopy is built, it may fail to
recognize the format of these testfiles.
Avoid using objcopy and instead add versions of the testfiles to
the testsuite with .debug_aranges already stripped.
Aaron Merey [Mon, 26 Feb 2024 14:58:39 +0000 (09:58 -0500)]
Add __libdw_getdieranges
__libdw_getdieranges builds an aranges list by iterating over each
CU and recording each address range.
This function is an alternative to dwarf_getaranges. dwarf_getaranges
attempts to read address ranges from .debug_aranges, which might be
absent or incomplete.
This patch replaces dwarf_getaranges with __libdw_getdieranges in
dwarf_addrdie and dwfl_module_addrdie. The existing tests in
run-getsrc-die.sh are also rerun with .debug_aranges removed from
the testfiles.
This is because dwarf_decl_file calls dwarf_getsrclines to populate
cu->files. For normal units, cu->files is cached by dwarf_getsrclines
when it parses the line number information. However, for split units,
the line number information is parsed for the skeleton unit, then copied
to the split unit's cu->lines. Split units have their own file name
table, so cu->files is not copied.
The obvious solution is to use dwarf_getsrcfiles instead of relying on
implicit caching.
Also add a test case for dwarf_decl_file.
* libdw/dwarf_decl_file.c (dwarf_decl_file): Use
dwarf_getsrcfiles instead of dwarf_getsrclines.
* tests/Makefile.am (check_PROGRAMS): Add declfiles.
(TESTS): Add run-declfiles.sh.
(EXTRA_DIST): Add run-declfiles.sh.
(declfiles_LDADD): New variable.
* tests/declfiles.c: New test.
* tests/run-declfiles.sh: New test.
Mark Wielaard [Wed, 21 Feb 2024 21:19:32 +0000 (22:19 +0100)]
readelf: Use unsigned loop variables in handle_verneed and handle_verdef
Prevent signed underflow by changing loop variables to unsigned and
doing count checks before decrementing. This isn't really a bug, but
prevents UB detected by ubsan on fuzzed input. The bad (fuzzed) input
data does get detected anyway.
* src/readelf.c (handle_verneed): Use unsigned cnt, cnt2.
(handle_verdef): Likewise.
Annobin address ranges were always printed as if they were 64bit wide
because addr_size was set to twice the size. This was done because the
note description size should contain two addresses. Fix this by setting
the address size to just one address and then check that descsz is
twice that.
* libebl/eblobjnote.c (ebl_object_note): Set addr_size to one
ELF_T_ADDR. Check descsz equals two times addr_size.
Update the documentation of dwarf_cu_dwp_section_info to make clear
that the function only returns an error if the DWARF package file data
couldn't be read or an unknown section constant is provided. Missing
DWP information for a given CU isn't an error and will set both OFFSET
and SIZE to zero. It also makes sure the documentation is < 76 chars
wide.
Omar Sandoval [Wed, 6 Dec 2023 09:22:17 +0000 (01:22 -0800)]
libdw: Try .dwp file in __libdw_find_split_unit()
Try opening the file in the location suggested by the standard (the
skeleton file name + ".dwp") and looking up the unit in the package
index. The rest is similar to .dwo files, with slightly different
cleanup since a single Dwarf handle is shared.
* libdw/libdw_find_split_unit.c (try_dwp_file): New function.
(__libdw_find_split_unit): Call try_dwp_file.
* libdw/libdwP.h (Dwarf): Add dwp_dwarf and dwp_fd.
(__libdw_dwp_findcu_id): New declaration.
(__libdw_link_skel_split): Handle .debug_addr for dwp.
* libdw/libdw_begin_elf.c (dwarf_begin_elf): Initialize
result->dwp_fd.
* libdw/dwarf_end.c (dwarf_end): Free dwarf->dwp_dwarf and close
dwarf->dwp_fd.
(cu_free): Don't free split dbg if it is dwp_dwarf.
Omar Sandoval [Wed, 6 Dec 2023 09:22:16 +0000 (01:22 -0800)]
libdw: Parse DWARF package file index sections
The .debug_cu_index and .debug_tu_index sections in DWARF package files
are basically hash tables mapping a unit's 8 byte signature to an offset
and size in each section used by that unit [1]. Add support for parsing
and doing lookups in the index sections.
We look up a unit in the index when we intern it and cache its hash
table row in Dwarf_CU. Then, a new function, dwarf_cu_dwp_section_info,
can be used to look up the section offsets and sizes for a unit. This
will mostly be used internally in libdw, but it will also be needed in
static inline functions shared with eu-readelf. Additionally, making it
public it makes dwp support much easier for external tools that do their
own low-level parsing of DWARF information, like drgn [2].
* libdw/dwarf.h: Add DW_SECT_TYPES.
* libdw/libdwP.h (Dwarf): Add cu_index and tu_index.
(Dwarf_CU): Add dwp_row.
(Dwarf_Package_Index): New type.
(__libdw_dwp_find_unit): New declaration.
(dwarf_cu_dwp_section_info): New INTDECL.
Add DWARF_E_UNKNOWN_SECTION.
* libdw/Makefile.am (libdw_a_SOURCES): Add
dwarf_cu_dwp_section_info.c.
* libdw/dwarf_end.c (dwarf_end): Free dwarf->cu_index and
dwarf->tu_index.
* libdw/dwarf_error.c (errmsgs): Add DWARF_E_UNKNOWN_SECTION.
* libdw/libdw.h (dwarf_cu_dwp_section_info): New declaration.
* libdw/libdw.map (ELFUTILS_0.190): Add
dwarf_cu_dwp_section_info.
* libdw/libdw_findcu.c (__libdw_intern_next_unit): Call
__libdw_dwp_find_unit, and use it to adjust abbrev_offset and
assign newp->dwp_row.
* libdw/dwarf_cu_dwp_section_info.c: New file.
* tests/Makefile.am (check_PROGRAMS): Add cu-dwp-section-info.
(TESTS): Add run-cu-dwp-section-info.sh
(EXTRA_DIST): Add run-cu-dwp-section-info.sh and new test files.
(cu_dwp_section_info_LDADD): New variable.
* tests/cu-dwp-section-info.c: New test.
* tests/run-cu-dwp-section-info.sh: New test.
* tests/testfile-dwp-4-strict.bz2: New test file.
* tests/testfile-dwp-4-strict.dwp.bz2: New test file.
* tests/testfile-dwp-4.bz2: New test file.
* tests/testfile-dwp-4.dwp.bz2: New test file.
* tests/testfile-dwp-5.bz2: New test file.
* tests/testfile-dwp-5.dwp.bz2: New test file.
* tests/testfile-dwp.source: New file.
Completely replace the "fdcache" algorithms in debuginfod, which
manages files extracted from archives. Previous logic was a LRU queue
for files requested by users, and a separate LRU queue for prefetched
files found nearby the ones users requested. The code did not handle
annoying edge cases like infrequently accessed but very costly
extraction of files like fedora kernels' vdso.debug. In addition, the
queue was searched linearly for normal lookups. It was also
unceremoniously dropped at each groom cycle.
New code replaces this with an indexed datastructure for quick
lookups, and extra metadata for use during eviction decisions. Each
entry tracks size and such, but now also tracks how recently and how
many times it was requested, how long it took to originally extract.
The new code combines these quantities in a score, by which eviction
eligibility is ranked. Intuitively, the effect is to prefer to hoard
small / slow-to-access files, and prefer to jettison large / fast /
never accessed ones.
It's a tricky thing to balance. The parameters in this configuration
were tested by timing-accurate replaying a few days' worth of actual
traffic of the main fedora debuginfod server. The peer
debuginfod.stg.fedoraproject.org runs the new code. It shows good
performance, excellent use of the cache storage, and strong preference
to hold onto those vdso.debug files. But who knows, it might need
tweaking later. The new code adds more prometheus metrics to make it
possible to grok the effectiveness of the few remaining
fdcache-related options.
Patch includes doc updates and NEWS. The changes are invisible to the
testsuite (except with respect to the new metrics). Code changes are
focused on all the member functions of class libarchive_fdcache, and
their callers. Unused parameters are removed, with previous command
line options hidden/accepted/ignored. Some other minor error-path
tempfile-gc was fixed in the extraction paths.
Aaron Merey [Sat, 10 Feb 2024 02:10:19 +0000 (21:10 -0500)]
Handle DW_AT_decl_file 0
Modify dwarf_decl_file to support DW_AT_decl_file with value 0.
Because of inconsistencies in the DWARF 5 spec, it is ambiguous whether
DW_AT_decl_file value 0 is a valid .debug_line file table index for the
main source file or if it means that there is no source file specified.
dwarf_decl_file interprets DW_AT_decl_file 0 as meaning no source file
is specified. This works with DWARF 5 produced by gcc, which duplicates
the main source file name at index 0 and 1 of the file table and avoids
using DW_AT_decl_file 0.
However clang uses DW_AT_decl_file 0 for the main source index with no
duplication at another index. In this case dwarf_decl_file will be
unable to find the file name of the main file.
This patch changes dwarf_decl_file to treat DW_AT_decl_file 0 as a normal
index into the file table, allowing it to work with DWARF 5 debuginfo
produced by clang.
As for earlier DWARF versions which exclusively use DW_AT_decl_file 0
to indicate that no source file is specified, dwarf_decl_file will now
return the name "???" if called on a DIE with DW_AT_decl_file 0.
Frank Ch. Eigler [Mon, 12 Feb 2024 15:03:02 +0000 (10:03 -0500)]
debuginfod.8 man page: tweak -U explanation
In debian bug #1063768, smcv noted that the man page was
out of date with respect to the tool debuginfod actually
uses for -U. Update the man page to fix the mismatch.
Reported-By: Simon McVittie <smcv@collabora.com> Signed-off-By: Frank Ch. Eigler <fche@redhat.com>
Aaron Merey [Mon, 22 Jan 2024 00:44:34 +0000 (19:44 -0500)]
unstrip: Call adjust_relocs no more than once per section.
During symtab merging, adjust_relocs might be called multiple times on
some SHT_REL/SHT_RELA sections. In these cases it is possible for a
relocation's symbol index to be correctly mapped from X to Y during the
first call to adjust_relocs but then wrongly remapped from Y to Z during
the second call.
Fix this by adjusting relocation symbol indices just once per section.
Also add stable sorting for symbols during symtab merging so that the
symbol order in the output file's symtab does not depend on undefined
behaviour in qsort.
Note that adjust_relocs still might be called a second time on a section
during add_new_section_symbols. However since add_new_section_symbols
generates its own distinct symbol index map, this should not trigger the
bug described above.
Mark Wielaard [Tue, 6 Feb 2024 11:34:51 +0000 (12:34 +0100)]
srcfiles: Fix --enable-gcov (BUILD_STATIC) build
When configuring with --enable-gcov we build most things static.
Including libdebuginfod. The src Makefile was only setup for a
shared library build of libdebuginfod.so. Fix this by providing
a static libdebuginfod in case of BUILD_STATIC.
This fixes the builder.sourceware.org elfutils-snapshots-coverage
and provides fresh coverage reports again at
https://snapshots.sourceware.org/elfutils/coverage/latest/
* Makefile.am (BUILD_STATIC): Provide libdebuginfod.a
Housam Alamour [Tue, 6 Feb 2024 00:18:05 +0000 (19:18 -0500)]
PR 30991: srcfiles tarball feature
* srcfiles.cxx: Introduce new --zip option that places all the
source files associated with a specified dwarf/elf file
into a zip file and sends it to stdout. Files may be
fetched from debuginfod (if applicable) or locally as
a backup.
Added -b option to disable the backup of checking
for files locally in -z mode.
* run-srcfiles-self.sh: Added test-case for the new zip
feature that archives the source files of the srcfiles
tool and checks archive integrity. An additional test
ensures that if debuginfod is enabled, the files are
fetched and archived properly while maintaing integrity.
* debuginfod-subr.sh: On very slow/remote storage, it can
take O(minute) to finish indexing the entire elfutils
build tree, so a wait_ready4 shell function is one
way to let a longer debuginfod wait operation work.
* srcfiles.1, NEWS: Added documentation for the new zip feature.
* configure.ac: Simplify check for libarchive for srcfiles.cxx
by integrating it into the same check for debuginfod.
* Makefile.am: build with local copy of debuginfod-client.
Mark Wielaard [Thu, 1 Feb 2024 13:56:18 +0000 (14:56 +0100)]
libelf: Treat elf_memory as if using ELF_C_READ_MMAP
An Elf handle created through elf_memory was treated as if opened with
ELF_C_READ. Which means libelf believed it had read the memory itself
and could simply write to it if it wanted (because it wasn't mmaped
directly on top of a file). This causes issues when that memory was
actually read-only. Work around this by pretending the memory was
actually read with ELF_C_READ_MMAP (so directly readable, but not
writable).
Add extra tests to elfgetzdata to check using elf_memory with
read-only memory works as expected.
* libelf/elf_memory.c (elf_memory): Call
__libelf_read_mmaped_file with ELF_C_READ_MMAP.
* tests/elfgetzdata.c (main): Add new "mem" option.
* tests/run-elfgetzdata.sh: Also run all tests with new
"mem" option.
Mark Wielaard [Sun, 21 Jan 2024 19:54:39 +0000 (20:54 +0100)]
libdwfl: Add some extra space to buffer to read kernel image header
GCC 14 notices we play some tricks with the array into which we try
to read the kernel image header.
image-header.c: In function ‘__libdw_image_header’:
image-header.c:77:18: error: array subscript -496 is outside array bounds of ‘char[96]’ [-Werror=array-bounds=]
77 | header = header_buffer - H_START;
| ^
image-header.c:67:12: note: at offset -496 into object ‘header_buffer’ of size 96
67 | char header_buffer[H_READ_SIZE];
| ^~~~~~~~~~~~~
GCC is correct. The new header pointer is before the actually buffer we
want to read from. Later in the code we "correct" the address again by
adding the "offset" off the elements we want to read. Such pointer
arithmetic is technically invalid. Make it valid by making the buffer
a little bigger, so all pointer arithmetic stays inside the header_buffer.
This does waste 496 bytes on the stack at the front of the buffer that
is never used.
* libdwfl/image-header.c (__libdw_image_header): Add H_START
to header_buffer size and return
Frank Ch. Eigler [Tue, 16 Jan 2024 02:21:24 +0000 (21:21 -0500)]
PR31248: debuginfod_find_*: lseek to the front on returned fds
Previous code could return a fd that had its file-offset left at the
place where libcurl last used it, namely at the end of the freshly
downloaded file. Not good if a client just wants to read it right
away! We now ensure (and document) that the fd is pointed to the
beginning of the file.
tests: fix build against upcoming `gcc-14` (`-Werror=calloc-transposed-args`)
`gcc-14` added a new `-Wcalloc-transposed-args` warning recently. It
detected minor infelicity in `calloc()` API usage in `elfutils`:
elfstrmerge.c: In function 'main':
elfstrmerge.c:450:32: error:
'calloc' sizes specified with 'sizeof' in the earlier argument and not in the later argument [-Werror=calloc-transposed-args]
450 | newscnbufs = calloc (sizeof (void *), newshnums);
| ^~~~
elfstrmerge.c:450:32: note: earlier argument should specify number of elements, later size of each element
Signed-off-by: Sergei Trofimovich <slyich@gmail.com>
Khem Raj [Sun, 10 Dec 2023 20:20:33 +0000 (12:20 -0800)]
Add helper function for basename
musl does not provide GNU version of basename and lately have removed
the definiton from string.h [1] which exposes this problem. It can be
made to work by providing a local implementation of basename which
implements the GNU basename behavior, this makes it work across C
libraries which have POSIX implementation only.
Mark Wielaard [Tue, 12 Dec 2023 09:43:49 +0000 (10:43 +0100)]
tests: Don't redirect output to /dev/null in run-native-test.sh
By redirecting all output to /dev/null in run-native-test.sh the
run-native-test.sh.log file will be empty on failures. This makes
it hard to figure out what went wrong.
Aleksei Vetrov [Thu, 23 Nov 2023 15:31:47 +0000 (15:31 +0000)]
libelf: check decompressed ZSTD size
Decompression functions like __libelf_decompress_zlib check that
decompressed data has the same size as it was declared in the header
(size_out argument). The same check is now added to
__libelf_decompress_zstd to make sure that the whole allocated buffer is
initialized.
* libelf/elf_compress.c (__libelf_decompress_zstd): Use return value
of ZSTD_decompress to check that decompressed data size is the
same as size_out of the buffer that was allocated.
It is possible for segments of different shared libaries to be interleaved
in memory such that the segments of one library are located in between
non-contiguous segments of another library.
For example, this can be seen with firefox on RHEL 7.9 where multiple
shared libraries could be mapped in between ld-2.17.so segments:
dwfl_segment_report_module did not account for the possibility of
interleaving non-contiguous segments, resulting in premature closure
of modules as well as failing to report modules.
Fix this by removing segment skipping in dwfl_segment_report_module.
When dwfl_segment_report_module reported a module, it would return
the index of the segment immediately following the end address of the
current module. Since there's a chance that other modules might fall
within this address range, dwfl_segment_report_module instead returns
the index of the next segment.
This patch also fixes premature module closure that can occur in
dwfl_segment_report_module when interleaving non-contiguous segments
are found. Previously modules with start and end addresses that overlap
with the current segment would have their build-ids compared with the
current segment's build-id. If there was a mismatch, that module would
be closed. Avoid closing modules in this case when mismatching build-ids
correspond to distinct modules.
Aleksei Vetrov [Mon, 20 Nov 2023 17:44:48 +0000 (17:44 +0000)]
tests: Add test for duplicate entries in archive
Test dwfl-report-offline-memory against an archive that contains
non-relocatable ELFs with the same name and contents.
* tests/test-ar-duplicates.a.bz2: New test file.
* tests/run-dwfl-report-offline-memory.sh: Test new
test-ar-duplicates.a.bz2.
* tests/Makefile.am (EXTRA_DIST): Add test-ar-duplicates.a.bz2.
Aleksei Vetrov [Mon, 20 Nov 2023 17:44:47 +0000 (17:44 +0000)]
libdwfl: handle duplicate ELFs when reporting archives
When archive is processed in process_archive (libdwfl/offline.c), it
creates an Elf object for each archive member. Then in
process_archive_member it calls process_file to create a Dwfl_Module
through __libdwfl_report_elf.
The ownership of the Elf object is expected to be:
* either transfered to the Dwfl_Module, if __libdwfl_report_elf returns
not NULL;
* or handled at the end of process_archive_member by calling elf_end.
Moreover, Elf object is expected to be alive, if __libdwfl_report_elf
returns not NULL, because at the end of process_archive_member it
advances to the next member through the elf_next call.
The problem happens when __libdwfl_report_elf encounters Elf with the
same name and content as it seen before. In that case dwfl_report_module
will reuse existing Dwfl_Module object. This leads to a codepath that
calls elf_end on the Elf object, while returning not NULL, breaking the
elf_next call to the next member.
The fix is to destroy m->main.elf instead and put the new Elf object in
the already existing Dwfl_Module.
* libdwfl/dwfl_report_elf.c (__libdwfl_report_elf): Replace Elf in
the Dwfl_Module in case of duplicate modules to prolong its
lifetime for subsequent processing.
Mark Wielaard [Wed, 15 Nov 2023 16:33:42 +0000 (17:33 +0100)]
readelf: Don't print average number of tests when no tests are done
If the symbol hash table only contains lenght zero chains, no lookup
tests need to be done and eu-readelf -I would print out bogus numbers
for the number of tests that were successful/unsuccessful.
e.g. for an "empty" program like
int main() {}
eu-readelf -I would print:
Histogram for bucket list length in section [ 5] '.gnu.hash' (total of 1 bucket):
Addr: 0x00000000004003c0 Offset: 0x0003c0 Link to section: [ 6] '.dynsym'
Symbol Bias: 1
Bitmask Size: 8 bytes 0% bits set 2nd hash shift: 0
Length Number % of total Coverage
0 1 100.0%
Average number of tests: successful lookup: -nan
unsuccessful lookup: 0.000000
Only print out the Average number of tests when there were actual
tests to do.
run-debuginfod-response-headers.sh does occassionally fail because
it might scan an rpm more than once. Try to fix this by making sure
all files that debuginfod is supposed to scan are ready before the
server starts. And to explicitly wait till the first scan is ready
and done before testing 'scanned_files_total{source=".rpm archive"}'
instead of sending an kill -USR1.
Aleksei Vetrov [Thu, 16 Nov 2023 21:29:22 +0000 (21:29 +0000)]
libdw: check offset dwarf_formstring in all cases
This check was initially added to test if offset overflows the safe
prefix where any string will be null-terminated. However the check
was placed in a wrong place and didn't cover all `attrp->form` cases.
* libdw/dwarf_formstring.c (dwarf_formstring): Move offset check
right before returning the result.
Mark Wielaard [Tue, 14 Nov 2023 20:34:50 +0000 (21:34 +0100)]
libelf: Fix elf_begin.c build on 32bit arches.
On 32bit architectures gcc produces an error:
elf_begin.c: In function ‘file_read_elf’:
elf_begin.c:495:30: error: cast to pointer from integer of different
size [-Werror=int-to-pointer-cast]
elf->state.elf64.shdr = (Elf64_Shdr *) (ehdr + e_shoff);
^
This is because we are adding an uintptr to an Elf64_Off which
promotes the result to a 64bit value. Fix this by casting the
e_shoff to an ptrdiff_t. This is fine since the mmap of the file
would have failed if it didn't fit in the 32bit address space
and we check that e_shoff fits inside the image.
* libelf/elf_begin.c (file_read_elf): Cast e_shoff to ptrdiff_t
before adding to ehdr.
Suggested-by: Paul Pluzhnikov <ppluzhnikov@google.com> Signed-off-by: Mark Wielaard <mark@klomp.org>
Paul Pluzhnikov [Mon, 13 Nov 2023 22:40:46 +0000 (22:40 +0000)]
Fix computations with (potentially) NULL pointer
When map_address is NULL, computing map_address+offset is technically
undefined behavior, and triggers Clang/LLVM warning when using
-fsanitize=pointer-overflow.
Fix this by using uintptr_t to perform computations.
Signed-off-by: Shahriar "Nafi" Rouf <nafi@google.com> Signed-off-by: Paul Pluzhnikov <ppluzhnikov@google.com>
Mark Wielaard [Wed, 8 Nov 2023 12:50:01 +0000 (13:50 +0100)]
libelf: Fix possible memory leak in elf_getdata_rawchunk
If the rawchunk is not properly aligned we'll create a new buffer
that is correctly aligned and put the data in that new buffer with
memcpy or the conversion function. In such cases the rawchunk leaks
because the new buffer is put into the Elf_Data_Chunk.
* libelf/elf_getdata_rawchunk.c (elf_getdata_rawchunk):
Call free on the rawchunk if new buffer was allocated.
Mark Wielaard [Mon, 13 Nov 2023 21:38:10 +0000 (22:38 +0100)]
backends: Fix arm_machine_flag_name version string.
arm_machine_flag_name checks the version byte and if not zero returns
a version string. There are only 5 versions defined. So check the
version byte is not larger.
* backends/arm_machineflagname.c (arm_machine_flag_name):
Check version <= 0, otherwise return NULL.
Mark Wielaard [Fri, 3 Nov 2023 14:05:57 +0000 (15:05 +0100)]
The default branch is now 'main'
Updated the HEAD symbolic reference.
Updated gitsigure checkref.
Updated hooks/post-receive irkerhook
Updated buildbot elfutils_[snapshots_]scheduler and elfutils_gitpoller
If you still have a checkout that refers to master and you do a git
pull you might get:
Your configuration specifies to merge with the ref
'refs/heads/master' from the remote, but no such ref was
fetched.
Make sure to rename your branch to main and set the new upstream:
libelf/elf_newscn.c: fix build failure against gcc-14 (-Walloc-size)
`gcc-14` adde a new -Walloc-size warning that makes sure that size of an
individual element matches size of a pointed type:
https://gcc.gnu.org/PR71219
`elfutils` triggers is on `calloc()` call where member size is sued as
`1`.
elf_newscn.c: In function `elf_newscn`:
elf_newscn.c:97:12: error: allocation of insufficient size «1» for type «Elf_ScnList» with size «16» [-Werror=alloc-size]
97 | newp = calloc (sizeof (Elf_ScnList)
| ^
The change swaps arguments to pass larger value as a member size.
Signed-off-by: Sergei Trofimovich <slyich@gmail.com>
Aaron Merey [Tue, 31 Oct 2023 20:00:27 +0000 (16:00 -0400)]
readelf: Support .gdb_index version 9
Version 9 adds a "shortcut table" to the index. The shortcut table contains
the name and language of the main function, if it exists.
A testcase added in this patch uses an executable written with Fortran.
This is because gdb does not currently populate the shortcut table of
C/C++ programs (see sourceware PR30996).
libdw and libdwfl currently save the path of the directory containing
the ELF file to use when searching for alt and dwo files. To search for
dwp files, we need the file name too. Add an elfpath field to Dwarf,
and set the debugdir field from it. Also update libdwfl to set elfpath
and debugdir.
libdw: Recognize .debug_[ct]u_index sections in dwarf_elf_begin
DWARF package (.dwp) files have a .debug_cu_index section and,
optionally, a .debug_tu_index section. Add them to the list of DWARF
sections.
Unfortunately, it's not that simple: the other debug sections in a dwp
file have names ending with .dwo, which confuses the checks introduced
by commit 5b21e70216b8 ("libdw: dwarf_elf_begin should use either plain,
dwo or lto DWARF sections."). So, we also have to special case
.debug_cu_index and .debug_tu_index in scn_dwarf_type and check_section
to treat them as TYPE_DWO sections.
PR 30000: debuginfod-find should have a source-list verb
* seclines.cxx: Introduce new tool that compiles a list of source
files associated with a specified dwarf/elf file. This
compilation relies on searching the dwarf debug information,
which can be automatically retrieved via debuginfod using
libdwfl functions when required. The target file can
encompass various types, such as an executable, a coredump,
a running process, or the currently executing kernel. The
source file names are rendered as unique entries and then
displayed on the standard output.
Frank Ch. Eigler [Tue, 10 Oct 2023 20:21:00 +0000 (16:21 -0400)]
PR30962: debuginfod: full paths for X-DEBUGINFOD-FILE/ARCHIVE response headers
Previous code was inconsistent in offering basename versus full
pathname for these headers. The documentation was not explicit on
this issue. We now simplify by always passing full names back, and
document this in the debuginfod.8 man page, along with pointers to
how to use proxy front-end servers to strip them if needed.
Mark Wielaard [Fri, 6 Oct 2023 11:56:55 +0000 (13:56 +0200)]
libdw: Skip zero entries in aranges
An address/length entry of two zeros is supposed to mark the end of a
table. But in some cases a producer might leave zero entries in the
table (for example when using gcc -ffunction-sections -gc-sections).
Since we know the lenght of the table we can just skip such entries
and continue to the end.
* libdw/dwarf_getaranges.c (dwarf_getaranges): Calculate endp.
When seeing two zero values, check we are at endp.
Mark Wielaard [Fri, 6 Oct 2023 14:25:56 +0000 (16:25 +0200)]
strip: Clean up a bit more after errors
In some cases (late) errors would exit early without cleanup. Set
result to 1 (failure) in those cases and fallthrough. That way we
cleanup and might provide more hints to the user about what was wrong
with the file.
* src/strip.c (handle_elf): Set result to 1 and fallthrough
instead of return 1.
libdw: Handle split DWARF in dwarf_macro_getsrcfiles
Macro information references file names from the line number information
table, which is tricky in split DWARF for a couple of reasons.
First, the line number information for a macro unit comes from the
.debug_line.dwo section in the split file, not the .debug_line section
in the skeleton file. This was not specified in the GNU DebugFission
design document [1] or the DWARF 5 standard, but it is how GCC and Clang
behave in practice and was clarified in DWARF standard issue
200602.1 [2] for the upcoming DWARF 6 standard.
dwarf_macro_getsrcfiles uses the line number information from whichever
Dwarf handle it was passed. This is error-prone, since the most natural
thing to do is to pass the skeleton Dwarf handle. Fix this by storing
the appropriate Dwarf handle in Dwarf_Macro_Op_Table and using that one.
Second, for .debug_macinfo.dwo in GNU DebugFission (generated by gcc
-gdwarf-4 -gstrict-dwarf -gsplit-dwarf), the offset into .debug_line.dwo
is implicitly 0. Again, this isn't in any specification, but it's how
GCC behaves in practice (Clang never generates macro information for
DWARF 4 split DWARF). Make get_macinfo_table default to 0 for split
DWARF when it can't find DW_AT_stmt_list.
Dwarf_Macro_Op_Table::is_64bit conflates the address size and the offset
size: for .debug_macinfo, it is initialized based on the compilation
unit's address size, but for .debug_macro, it is initialized based on
the macro unit's offset size. is_64bit is used to determine the address
size to pass to __libdw_getsrclines. For a 64-bit architecture using
DWARF 5 with 32-bit offsets (the common case), this fails because
read_srclines checks that the given address size matches the address
size from the line number program header.
Fix it by splitting is_64bit into separate address_size and offset_size
members.
libdw: Handle other string forms in dwarf_macro_param2
dwarf_getmacros handles the additional macro string forms added by DWARF
5, but dwarf_macro_param2 doesn't. Update it with the list of all
string forms allowed in .debug_macro. In particular, GCC and Clang
generate DW_MACRO_define_strx and DW_MACRO_undef_strx, which
dwarf_macro_param2 couldn't handle.
libdw: Handle DW_AT_ranges in split DWARF 5 skeleton in dwarf_ranges
When commit 879f3a4f99df ("libdw: Handle .debug_rnglists in
dwarf_ranges.") added support for split DWARF 5 in 2018, GCC put all
range lists for split DWARF in the .debug_rnglists section of the
skeleton file (similarly to GNU DebugFission, which puts all range lists
in .debug_ranges in the skeleton file).
In 2021, after a discussion on the dwarf-discuss mailing list [1], GCC
changed this to match Clang's behavior. Now, ranges are in
.debug_rnglists.dwo in the split file, _except_ for one: the skeleton
unit DIE has a DW_AT_ranges attribute, and its ranges are in
.debug_rnglists in the skeleton file. See GCC commit 4b33c5aaab9e
("dwarf2out: Fix up ranges for -gdwarf-5 -gsplit-dwarf [PR99490]") and
the Issue 210310.1 clarifying the DWARF standard [2].
Unfortunately, this confuses dwarf_ranges, which always uses
.debug_rnglists.dwo if it exists. Fix it by special casing the unit
DIE: its range lists should be in .debug_rnglists if that exists, and
.debug_rnglists.dwo otherwise.
If a DIE has no DW_AT_entry_pc attribute, dwarf_entrypc looks for
DW_AT_low_pc in that DIE. But for a split compilation unit DIE,
DW_AT_low_pc is in the corresponding skeleton DIE, so this fails.
dwarf_lowpc already handles this fallback, so use it instead.
PR30879: intermittent debuginfod crash with unhandled exception
Code inspection identified two places where sqlite_ps objects were
being created/used outside try/catch protection. This patch wraps or
replaces them.
* configure.ac: Look for glibc backtrace headers.
* debuginfod.cxx (scan): New function wrapped by a try/catch loop.
(sqlite_checkpoint_pb): Use non-exception-producing sqlite functions.
(main, my_terminate_handler): New terminate() handler.
Mark Wielaard [Thu, 7 Sep 2023 14:14:43 +0000 (16:14 +0200)]
libelf: tdelete dummy key if anything goes wrong setting up rawchunk
elf_getdata_rawchunk uses a binary search tree cache. If a rawchunk is
not yet in the cache we setup a new entry. But if anything went wrong
setting up the new rawchunk we would leave a NULL key in the
cache. This could blow up the next search. Fix this by removing the
(dummy) key from the cache on any failure.
* libelf/elf_getdata_rawchunk.c (elf_getdata_rawchunk): Don't
assign NULL to *found. Call tdelete if anything goes wrong.
Mark Wielaard [Sun, 3 Sep 2023 16:25:56 +0000 (18:25 +0200)]
tests: Fix system_elf_gelf_test build without system libelf.h
If there is no system libelf.h then the building of system-elf-gelf-test
fails with:
../libelf/gelf.h:32:10: fatal error: libelf.h: No such file or directory
This is because although the testcase includes the headers as
../libelf/libelf.h and ../libelf/gelf.h, gelf.h itself does an
Fix this by putting a copy of libelf.h in the build test directory
and using -I. for building system-elf-gelf-test.
* tests/Makefile.am (BUILT_SOURCES): New for libelf.h.
(CLEANFILES): Add libelf.h.
(libelf.h): New target that copies srdir libelf.h.
(system_elf_gelf_test_CPPFLAGS): Add -I.
* debuginfod-client.c (debuginfod_query_server): Use fstat(3)
of the file handle being downloaded into as the preferred
source of download progress.
Tested by hand, as the testsuite doesn't have enough machinery to
simulate compressed vs. uncompressed service. Hand testing with
(unmodified) fedora-38 gdb and debuginfod-find shows dramatically
improved progress displays: all have quantitative figures when
fetching from real (unmodified) upstream servers.
Mark Wielaard [Sun, 23 Jul 2023 21:14:31 +0000 (23:14 +0200)]
libelf, readelf, elflint: Add RELR support
Handle RELR as defined here:
https://groups.google.com/g/generic-abi/c/bX460iggiKg/m/YT2RrjpMAwAJ
Introduce new ELF_T_RELR Elf_Type and handle it for SHT_RELR. Check
various properties in elflint. Print RELR relocations in
readelf. Just the entries with -U. Just the addresses with -N. And
addresses plus symbol/offsets by default.
Also add a test to check that gelf.h works with the system elf.h.
* libebl/eblsectiontypename.c (ebl_section_type_name): Add RELR
to knownstype.
* libelf/elf32_updatenull.c (updatenull_wrlock): Handle
sh_entsize for SHT_RELR.
* libelf/gelf.h (GElf_Relr): New typedef for Elf64_Relr.
* libelf/gelf_fsize.c (__libelf_type_sizes): Add ELF_T_RELR.
* libelf/gelf_xlate.c (__elf_xfctstom): Likewise.
* libelf/gelf_xlate.h: Add RELR as FUNDAMENTAL.
* libelf/libelf.h (Elf_Type): Add ELF_T_RELR. Add RELR
defines/typedefs if undefined in system elf.h.
* libelf/libelfP.h: Define ELF32_FSZ_RELR and ELF64_FSZ_RELR.
* src/elflint.c (check_reloc_shdr): Check she_entsize for
ELF_T_RELR.
(check_relr): New function.
(check_dynamic): Handle DT_RELR.
(special_sections): Add SHT_RELR.
(check_sections): Call check_relr.
* src/readelf.c (print_relocs): Also accept a Dwfl_Module.
(handle_relocs_relr): New function.
(print_dwarf_addr): Make static and declare early.
(process_elf_file): Pass dwflmod to print_relocs.
(handle_dynamic): Handle DT_RELRSZ and DTRELRENT.
* system-elf-gelf-test.c: New test.
* Makefile.am (TESTS): Add system-elf-gelf-test.
(check_PROGRAMS): Likewise.
(system_elf_gelf_test_CPPFLAGS): New variable.
(system_elf_gelf_test_LDADD): Likewise.
Mark Wielaard [Mon, 28 Aug 2023 18:23:41 +0000 (20:23 +0200)]
libelf: Remove elf_scncnt from libelf.map
elf_scncnt was never implemented. It was probably an old name for
elf_getshnum (which was the deprecated name of the elf_getshdrnum
alias). Just remove it from the map file
Mark Wielaard [Sun, 30 Jul 2023 15:28:00 +0000 (17:28 +0200)]
libelf: Free and clear rawdata_base and zdata_base consistently
There could be a leak if a program called elf_strptr on a compressed
section, but the program never requests the (uncompressed) section data,
but does explicitly (re)compress that same section data.
Fix this by explicitly always freeing and clearing the zdata_base
and rawdata_base in __libelf_reset_rawdata and elf_compress. Also
clear zdata_base in elf_end so the pointer isn't indeterminate when
it is being used in a later comparison against rawdata_base.
* libelf/elf_compress.c (elf_compress): Explicitly free
zdata_base before clearing.
(__libelf_reset_rawdata): Free zdata_base if it isn't
(going to be) used for rawdata_base. Explicitly clear
rawdata_base and zdata_base after free.
* libelf/elf_end.c (elf_end): Clear zdata_base after free.
Sam James [Wed, 19 Jul 2023 23:16:38 +0000 (00:16 +0100)]
tests: run-lfs-symbols.sh needs gawk
With awk=mawk, I get:
```
FAIL: run-lfs-symbols.sh
========================
First sanity-check that LFS detection works.
checking ./testfile-nolfs
awk: line 3: syntax error at or near /
FAIL run-lfs-symbols.sh (exit status: 2)
```
* tests/run-lfs-symbols.sh: Call 'gawk' instead of 'awk'.
Mark Wielaard [Wed, 21 Jun 2023 16:05:12 +0000 (18:05 +0200)]
libelf: Replace list of elf_getdata_rawchunk results with a tree
elf_getdata_rawchunks did a linear search to see if a chunk was
already fetched. Replace this list with a binary search tree to make
lookup faster when a lot of Elf_Data_Chunk were created.
* libelf/libelfP.h (Elf_Data_Chunk): Remove next field.
(struct Elf): Change the rawchunks type from Elf_Data_Chunk *
to void *.
* elf_getdata_rawchunk.c (chunk_compare): New static function.
(elf_getdata_rawchunk): Use tsearch instead of a manual linked
list.
* elf_end.c (free_chunk): New static function.
(elf_end): Call tdestroy instead of walking linked list.
Mark Wielaard [Thu, 22 Jun 2023 12:45:56 +0000 (14:45 +0200)]
debuginfod: Fix formatting in debuginfod_config_cache
The formatting of debuginfod_config_cache in debuginfod-client.c was
slightly off making it hard to see the program logic. Make sure lines
are < 76 chars, and if { } else { } indentation follows GNU style.
Mark Wielaard [Sat, 17 Jun 2023 19:37:41 +0000 (21:37 +0200)]
tests: Split up run-strip-reloc.sh test in three subtests
This test can take a very long time under valgrind (on s390x).
Split it into three tests for kernel modules, self-test and ppc64.
That way the tests can run in parallel and each individual
test produces a result (so the buildbot won't time out).
Also change the comparison to elfcmp instead of doing
readelf -a on both and compare the textual output.
* tests/run-strip-reloc.sh: Split out generic part into...
* tests/strip-reloc-subr.sh: ...this file.
* tests/run-strip-reloc-ko.sh: New file.
* tests/run-strip-reloc-ppc64.sh: New file.
* tests/run-strip-reloc-self.sh: New file.
* tests/Makefile.am (TESTS): Remove run-strip-reloc.sh.
Add run-strip-reloc-ko.sh, run-strip-reloc-self.sh and
run-strip-reloc-ppc64.sh
(EXTRA_DIST): Remove run-strip-reloc.sh. Add
strip-reloc-subr.sh, run-strip-reloc-ko.sh,
run-strip-reloc-self.sh and run-strip-reloc-ppc64.sh.
Mark Wielaard [Sat, 17 Jun 2023 12:56:12 +0000 (14:56 +0200)]
tests: Really split run-copymany-sections.sh into separate tests
The previous commit 4fac1627c "tests: Split run-copymany-sections.sh
into separate tests" duplicated the tests because they were still
also done in the new test-copymany-subr.sh. Remove them really this
time. Also don't source test-subr.sh twice, test-copymany-subr.sh
already sources it, so the subtests don't have to also do it again.
Mark Wielaard [Sat, 17 Jun 2023 12:03:36 +0000 (14:03 +0200)]
tests: Split run-copymany-sections.sh into separate tests
This test can take a very long time under valgrind (on s390x).
Split it into four tests for big/little endian, 32/64 bit.
That way the tests can run in parallel and each individual
test produces a result (so the buildbot won't time out).
* tests/run-copymany-sections.sh: Delete and move
generic part into...
* tests/test-copymany-subr.sh: ... here.
* tests/run-copymany-be32.sh: New big endian 32 bit tests.
* tests/run-copymany-be64.sh: New big endian 64 bit tests.
* tests/run-copymany-le32.sh: New little endian 32 bit tests.
* tests/run-copymany-le64.sh: New little endian 64 bit tests.
* tests/Makefile.am (TESTS): Remove run-copymany-sections.sh,
add run-copymany-be32.sh, run-copymany-le32.sh,
run-copymany-be64.sh and run-copymany-le64.sh.
(EXTRA_DIST): Remove run-copymany-sections.sh. Add
test-copymany-subr.sh, run-copymany-be32.sh,
run-copymany-le32.sh, run-copymany-be64.sh and
run-copymany-le64.sh.
Mark Wielaard [Fri, 16 Jun 2023 23:40:38 +0000 (01:40 +0200)]
tests: Add a bit less and slightly smaller sections for addsections
The run-copymany-sections.sh testcase adds 64K sections (twice)
to a couple of times. Each section is just 6 bytes long, but each
data section is malloced and freed. That adds up. And is especially
slow when running under valgrind.
Reduce the number of sections added to 32K (twice) and make each
section data just one single zero byte.
* tests/addsections.c (add_sections): Don't strdup and free
the string ".extra", but just add the empty string.
* tests/run-copymany-sections.sh: Call addsections with
32768 instead of 65535.
Add a "--scan-checkpoint=NUM" option to debuginfod to control forced
synchronization & sqlite -wal checkpointing for the multithreaded
scanning process. In absence of this, a server that's busy with other
read & write operations can accumulate potentially large SQLITE WAL
temporary files. This option causes the server to take intermittent
quiescent breaks during scanning, during which the -wal file can be
processed and truncated.
debuginfod: PR30378: better compression for _files table
Split the _files table into two links into a new _fileparts table,
interning the dirname and basename of each file name string. This
reduces storage requirements for many thousands of almost-identical
long paths that are evident in large builds like kernels.
This is unfortunately a schema-breaking change, so requires reindexing
of the corpus.
While in the vicinity, the file scan work queue is changed from a
<set> to an <unordered_set>. The intent is that files be scanned in a
more random sequence instead of sorted. If they're sorted, then files
that contain errors will tend to be retried over and over again at the
next scan cycle, even at the expense of making progress on the other
files in the queue.
backends: Add frame pointer unwinding for LoongArch
If we don't find any debug information for a given frame, we usually
cannot unwind any further. However, the binary in question might have
been compiled with frame pointers, in which case we can look up the
well known frame pointer locations in the stack snapshot and use them
to bridge the frames without debug information.
Signed-off-by: Liwei Ge <geliwei@openanolis.org> Signed-off-by: Youling Tang <tangyouling@loongson.cn>
projects / thirdparty / elfutils.git / log
