readelf.c: Close dwfl if dwfl_report_offline fails
Within create_dwfl, if dwfl_begin is successful but dwfl_report_offline
fails, the dwfl * pointer being reported is reset to NULL without calling
dwfl_end, causing a memory leak.
Update create_dwfl to call dwfl_end in this case, preventing the leak.
elf_begin.c: Use relative offset in archive size check
Before creating an elf descriptor for an archive member, dup_elf
verifies that the size of an archive is large enough to contain the
member. This check uses the member's offset relative to the map_address
of the top-level archive containing the member.
This check can incorrectly fail when an archive contains another
archive as a member. For members of the inner archive, their offset
relative to the outer archive might be significantly larger than
the max_size of the inner archive. This appears as if the offset and
max_size values are inconsistent and the creation of the member's elf
descriptor is stopped unnecessarily.
Fix this by accounting for the inner archive's non-zero start_offset
when judging whether the member can fit within it.
Also perform this size check before creating a copy of the member's
Elf_Arhdr to prevent leaking the header's ar_name and ar_rawname if
the size check fails.
Mark Wielaard [Sat, 6 Sep 2025 10:39:15 +0000 (12:39 +0200)]
libelf: Check ELF parent size can contain ar member
Don't trust the ar header offset and size. When creating an Elf
descriptor for an ar member check the offset isn't past the end of the
containing Elf and don't use/set the member maximum_size larger than
the remaining size of the parent.
* libelf/elf_begin.c (dup_elf): Only call read_file if the
offset isn't past the end and with a maximum_size not too large.
If libdw_open_elf detects an invalid ELF file, it may attempt to
temporarily treat it as an ELF archive in order to check if there's
a valid ELF file following a header.
When doing this, the elf descriptor for the invalid file is given
the dummy state.ar.elf_ar_hdr.ar_name "libdwfl is faking you out".
Afterwards libdw_open_elf will call elf_end on the elf descriptor
for the invalid ELF file. elf_end will attempt to free the address
of the "libdwfl is faking you out" literal, causing an invalid free.
Fix this by setting the ar_name to NULL before libdw_open_elf calls
elf_end on the descriptor of the invalid ELF file.
When elf_getdata_rawchunk aquires a new chunk for the first time, it
inserts a stack-allocated dummy chunk into a search_tree with a rdlock
held. When the real chunk is prepared to replace the dummy chunk, the
rdlock is released and a wrlock is then held while replacing the
dummy with the real chunk.
Before the wrlock is held, other threads could incorrectly acquire the
dummy chunk as if it were a real chunk.
Fix this by holding a wrlock throughout elf_getdata_rawchunk.
__libdw_dieabbrev: Replace rwlock with __atomic builtins
__libdw_dieabbrev uses the abbrev_lock rwlock to synchronize access to the
Dwarf_Die abbrev field as well as its lazy loading. Calls to rwlock_wrlock
and unlock incur significant performance overhead even in single-threaded
cases.
This patch implements thread safety in __libdw_dieabbrev with GCC __atomic
builtins instead of an rwlock, improving performance. abbrev_lock has been
changed to a mutex and now synchronizes access to the Dwarf_CU field
last_abbrev_offset in __libdw_findabbrev.
libdw/
* dwarf_begin_elf.c (valid_p): Change type of abbrev_lock from
rwlock to mutex.
* dwarf_end.c (cu_free): Ditto.
* dwarf_tag.c (__libdw_findabbrev): Protect
cu->last_abbrev_offset with mutex.
* libdwP.h (struct Dwarf_CU): Change type of abbrev_lock from
rwlock to mutex.
(__libdw_dieabbrev): Use __atomic builtins instead of rwlock.
* libdw_findcu.c (__libdw_intern_next_unit): Change type of
abbrev_lock from rwlock to mutex.
Mark Wielaard [Sat, 30 Aug 2025 13:47:24 +0000 (15:47 +0200)]
elfclassify: Use (void) for no-argument static functions, not ()
In -std=gnu11 mode gcc will error out on nor argument () static
functions:
src/elfclassify.c:757:1: error: function declaration isn’t a prototype
[-Werror=strict-prototypes]
757 | check_checks ()
| ^~~~~~~~~~~~
src/elfclassify.c: In function ‘check_checks’:
src/elfclassify.c:757:1: error: old-style function definition
[-Werror=old-style-definition]
This isn't an issue in -std=gnu23 mode (the default with GCC 15):
A definition using () is not considered an old-style definition in
C23 mode, because it is equivalent to (void) in that case, but is
considered an old-style definition for older standards
So just always use (void) instead of () for an empty argument list.
* src/elfclassify.c (check_checks): Define as check_checks (void).
(check_ar_members): Define as check_ar_members (void).
There is --unstripped but that is whether the ELF file can be stripped
(further) by removing any .[z]debug_* sections or symbol table
(.symtab section). The --has-debug-sections option only checks
whether there are .[z]debug sections present.
* src/elfclassify.c (is_has_debug_sections): New function.
(enum classify_check): Add classify_has_debug_sections.
(check_checks): Add and print (if verbose)
classify_has_debug_sections value.
(main): Add ar-member to options. Update argp.doc to explain
different debug options (--unstripped, --has-debug-section and
--debug-only).
* tests/run-elfclassify.sh: Add various --has-debug-sections
tests.
Mark Wielaard [Tue, 26 Aug 2025 12:51:54 +0000 (14:51 +0200)]
elfclassify: Add --any-ar-member option to classify archive member
To determine whether an ELF archive contains a member of a particular
type add a new input option to eu-elfclassify --any-ar-member.
eu-elfclassify will match if a given file is an ELF archive and the
classification options apply to at least one of the members.
This can be used for example to check whether an ELF archive file
contains ELF members that can be stripped by using:
eu-elfclassify --ar-member --unstripped <ar-file>
* src/elfclassify.c (current_path): Drop const.
(flag_ar_member): New static bool.
(classify_flag_ar_member): new enum member.
(parse_opt): Handle classify_flag_ar_member.
(check_checks): New function taken from...
(process_current_path): ...here.
(check_ar_members): New function.
(main): Add ar-member to options. Update argp.doc.
* tests/run-elfclassify.sh: Add various --any-ar-member tests.
Aaron Merey [Mon, 11 Aug 2025 00:26:35 +0000 (20:26 -0400)]
elf_getarhdr: Replace per-archive Elf_Arhdr storage with per-member storage
Currently each archive descriptor maintains a single Elf_Arhdr for the
current archive member (as determined by elf_next or elf_rand) which is
returned by elf_getarhdr.
A single per-archive Elf_Arhdr is not ideal since elf_next and elf_rand
can invalidate an archive member's reference to its own Elf_Arhdr.
Avoid this possible invalidation by storing each Elf_Arhdr in its
archive member descriptor.
The existing Elf_Arhdr parsing and storage in the archive descriptor
internal state is left mostly untouched. When an archive member is
created with elf_begin it is given its own copy of its Elf_Arhdr from
the archive descriptor.
Also update tests/arextract.c with verification that each Elf_Arhdr
is distinct and remains valid after calls to elf_next that would
have previously invalidated the Elf_Arhdr.
Aaron Merey [Tue, 12 Aug 2025 14:39:58 +0000 (10:39 -0400)]
Makefile.am: Add errors to lcov ignore list
Recent versions of lcov may raise the following errors during
make coverage:
genhtml: ERROR: (corrupt) unable to read trace file 'elfutils.lcov':
genhtml: ERROR: (inconsistent) [...] line is hit but no branches on line have been evaluated.
Add "inconsistent,corrupt" to the --ignore-errors list so that
make coverage exits without error.
Aaron Merey [Tue, 5 Aug 2025 03:20:54 +0000 (23:20 -0400)]
src/readelf.c: Support concurrency for -w, --debug-dump
Implement concurrent execution of print_debug_* functions during handling
of -w, --debug-dump using libthread.a.
A new `-C, --concurrency=NUM` command line option controls the maximum
number of threads that may be used. This value defaults to the number of CPUs.
Job output is buffered and printed in the order that jobs were added to
the queue. This helps preserve the existing order of stdout output.
* src/readelf.c (default_concurrency): Function estimating the
maximum number of threads.
(parse_opt): Handle -C, --concurrency=NUM.
(do_job): Entry point function for threads.
(schedule_job): If thread safety is enabled, add job to the
job queue. Otherwise just run the job from the main thread.
(print_debug): Pass print_debug_* function pointers and
args to schedule_job. Also call run_jobs if thread safety
is enabled.
Aaron Merey [Tue, 5 Aug 2025 03:20:53 +0000 (23:20 -0400)]
src/readelf.c: Add support for print_debug_* output buffering
Safely handle stdout output during concurrent calls to print_debug_*
functions.
For any print_debug_* function and any function that could be called
from print_debug_* which also prints to stdout: add a FILE * argument
and replace all printf, puts, putchar with fprintf. All printing
to stdout will now be written to this FILE instead.
The FILE * is an interface to a per-thread dynamically-sized buffer.
libthread.a manages the allocation, printing and deallocation of
these buffers.
Aaron Merey [Tue, 5 Aug 2025 03:20:52 +0000 (23:20 -0400)]
src: Add threadlib library for parallel job execution
Add new internal static library libthread.a that provides infrastructure
for eu-* tools to run functions concurrently using pthreads.
threadlib.c manages per-job threads as well as per-job buffers for stdout
output. Output for each job is printed to stdout in the order that the
jobs were added to the job queue. This helps preserve the order of
output when parallelization is added to an eu-* tool.
threadlib.h declares functions add_job and run_jobs. Jobs are added to
a threadlib.c internal job queue using add_job. run_jobs concurrently
executes jobs in parallel.
eu-readelf now links against libthread.a when elfutils is configured
with --enable-thread-safety.
* src/Makefile.am: libthread.a is compiled and and linked with
readelf when USE_LOCKS is defined.
* src/threadlib.c: New file. Manages job creation, concurrent
execution and output handling.
* src/threadlib.h: New file. Declares functions add_job and
run_jobs.
Archives are usually not meant to be executable and hence should not
need the executable bit set. Drop it consistently to avoid unintential
(and potentially harmful) execution.
Note, no binary in this repository has been identified as harmful in any
way; this is just a drive-by change.
Suggested-by: Philipp Reichart <reichart@google.com> Signed-off-by: Matthias Maennich <maennich@google.com>
Aaron Merey [Sun, 10 Aug 2025 19:25:39 +0000 (15:25 -0400)]
debuginfod-find.c: Avoid leaving temp files in the cache
Add a SIGINT handler to debuginfod-find so that ctrl-c during
a download causes clean up to occur instead of immediately killing
the process and possibly leaving temp files in the client cache.
This change requires setting the client progressfn unconditionally
instead of only when --verbose is given. This in turn requires a small
change to a debuginfod-find testcase in run-debuginfod-artifact-running.sh
to account for output differences between the debuginfod-find progressfn
and the debuginfod_client default_progressfn which was previously used
when debuginfod-find lacked -v but had DEBUGINFOD_PROGRESS=1 set.
Aaron Merey [Sun, 10 Aug 2025 19:25:17 +0000 (15:25 -0400)]
debuginfod-client.c: Skip negative cache entry for cancelled downloads
The debuginfod client cache uses empty files to indicate that a download
was unsuccessful and should not be attempted again.
Commit 5527216460c61 skips the creation of empty files when a downloaded
is cancelled by the user. This works by setting
client->progressfn_cancel to true when progressfn returns a non-zero value.
Commit d47d93b1049ec appears to have accidentally removed this setting of
client->progressfn_cancel.
Restore setting client->progressfn_cancel to true when progressfn
returns a non-zero value. Also set client->progressfn_cancel to false
at the beginning of debuginfod_find_metadata to avoid previous
cancellations affecting the current metadata query.
elf_getaroff currently returns ELF_C_NULL (0) to indicate that an error
occured (ex. the Elf descriptor is not associated with an archive).
However elf_getaroff is intended to return -1 if an error occurs.
eu-ar assumes -1 indicates an error and other libelf implementations
use -1 to indicate an error in elf_getaroff.
Replace ELF_C_NULL with -1 as elf_getaroff's error return value.
Mark Wielaard [Thu, 26 Jun 2025 13:06:07 +0000 (15:06 +0200)]
config: Adjust AM_CPPFLAGS for srcdir and .. path includes
When building with clang and libc++ some standard headers might try
including <stack> even when no source file requests such include
directly. When building with srcdir == builddir this might clash in
the src dir since we then build a stack binary there and the src dir
also has srcfiles.cxx which might include some standard c++ headers.
Work around this by removing -I.. from AM_CPPFLAGS and replacing it
with -I(abs_top_builddir) where the <config.h> file can be found. And
use -iquote for srcdir so it doesn't get included in the search path
for the system <header> includes (only for "header" includes).
Note that DEFAULT_INCLUDES might add . and srcdir back. So
DEFAULT_INCLUDES is disabled explicitly in src/Makefile.am (where the
stack binary is build). We could also use the nostdinc automake option
to completely suppress that, but that needs more auditing of various
installed vs not-installed header files.
* config/eu.am (AM_CPPFLAGS): Use -iquote for $(srcdir) and
replace -I.. with -I$(abs_top_builddir).
* libdw/libdwP.h: Include "libdw.h" and "dwarf.h" instead of
the system headers <libdw.h> and <dwarf.h>.
Mark Wielaard [Sat, 21 Jun 2025 11:43:51 +0000 (13:43 +0200)]
libdwl: Add validate_strdata to limit Elf_Data d_size to valid strings.
dwfl_module_getsym returns the name of a symbol as found in the
corresponding (symbol) string section. Make sure all names are
correctly zero terminated by making sure the last valid index in a
section/segment Elf_Data contains a zero character.
* libdwfl/dwfl_module_getdwarf.c (validate_strdata): New
function taking Elf_Data and restricting d_size to last zero
char.
(translate_offs): Call validate_strdata.
(find_symtab): Likewise for both symstrdata and aux_symstrdata.
Mark Wielaard [Sat, 31 May 2025 22:16:51 +0000 (00:16 +0200)]
libdw: Make __libdw_fde_by_offset static in libdw/fde.c
__libdw_fde_by_offset is marked as an extern internal function in
libdw/cfi.h (a not public header file). But it is (now) only used in
libdw/fde.c, called from __libdw_find_fde. It was originally used for
dwarf_cfi_validate_fde, but that function was deleted and never made
public. So simplify things and make it static in libdw/fde.c.
* libdw/cfi.h (__libdw_fde_by_offset): Remove.
* libdw/fde.c (__libdw_fde_by_offset): Make static.
Mark Wielaard [Mon, 2 Jun 2025 23:50:07 +0000 (01:50 +0200)]
libcpu: riscv_disasm use 50 char mnebuf
Some "illegal" instructions can be up to 24 chars (192 bits), We'll
print this as 0x<48 hex chars>. So make sure the mnebuf is 50 chars
(no terminating zero is needed).
This shows up with _FORTIFY_SOURCE which would immediate terminate on
such "illegal" instructions. Without we just use a few extra bytes on
the stack (which aren't used afterwards, without any issue, even
though it is technically UB).
* libcpu/riscv_disasm.c (riscv_disasm): Extend char mnebuf
array to 50.
Aaron Merey [Mon, 2 Jun 2025 21:06:31 +0000 (17:06 -0400)]
libdw: Fix eu_search_tree TOCTOU bugs
eu_tfind is used to facilitate lazy loading throughout libdw.
If a result is not found via eu_tfind, work is done to load
the result and cache it in an eu_search_tree.
Some calls to eu_tfind allow for TOCTOU bugs. Multiple threads
might race to call eu_tfind on some result that hasn't yet been
cached. Multiple threads may then attempt to load the result
which might cause an unnecessary amount of memory to be allocated.
Additionally this memory may not get released when the associated
libdw data structure is freed.
Fix this by adding additional locking to ensure that only one
thread performs lazy loading.
One approach used in this patch is to preserve calls to eu_tfind
without additional locking, but when the result isn't found then
a lock is then used to synchronize access to the lazy loading code.
An extra eu_tfind call has been added at the start of these critical
section to synchronize verification that lazy loading should proceed.
Another approach used is to simply synchronize entire calls to
functions where lazy loading via eu_tfind might occur (__libdw_find_fde
and __libdw_intern_expression). In this case, new _nolock variants of
the eu_t* functions are used to avoid unnecessary double locking.
libdw/
* cfi.h (struct Dwarf_CFI_s): Declare new mutex.
* dwarf_begin_elf.c (valid_p): Initialize all locks for fake CUs.
* dwarf_cfi_addrframe.c (dwarf_cfi_addrframe): Place lock around
__libdw_find_fde.
* dwarf_end.c (cu_free): Deallocate all locks unconditionally,
whether or not the CU is fake.
* dwarf_frame_cfa.c (dwarf_frame_cfa): Place lock around
__libdw_intern_expression.
* dwarf_frame_register.c (dwarf_frame_register): Ditto.
* dwarf_getcfi.c (dwarf_getcfi): Initialize cfi lock.
* dwarf_getlocation.c (is_constant_offset): Synchronize access
to lazy loading section.
(getlocation): Place lock around __libdw_intern_expression.
* dwarf_getmacros.c (cache_op_table): Synchronize access to lazy
loading section.
* frame-cache.c (__libdw_destroy_frame_cache): Free Dwarf_CFI
mutex.
* libdwP.h (struct Dwarf): Update macro_lock comment.
(struct Dwarf_CU): Declare new mutex.
libdw_findcu.c (__libdw_intern_next_unit): Initialize
intern_lock.
(__libdw_findcu): Adjust locking so that the first eu_tfind
can be done without extra lock overhead.
Michal Sekletar [Mon, 2 Jun 2025 12:46:20 +0000 (14:46 +0200)]
libdwfl: resolve all paths relative to sysroot
Whenever possible, resolve all symlinks as if the sysroot path were a
chroot environment. This prevents potential interactions with files from
the host filesystem.
Signed-off-by: Michal Sekletar <msekleta@redhat.com>
Mark Wielaard [Tue, 6 May 2025 09:50:12 +0000 (11:50 +0200)]
tests: Create random test_dir name
The testsuite relies on there being no files in the test directory
after the test finishes. A test will fail if the test dir cannot be
removed. But the test dir isn't really random, it uses the pid of the
shell script that executes the test. On some of the buildbots that
execute a lot of tests it can happen that the pid number wraps around
and a pid of a previous pid is reused. To prevent that happening
generate a real random number (8 bytes) using od /dev/urandom and
xargs (to trim away spaces left by od).
* tests/test-subr.sh: Define test_name and random_number and use
those to define test_dir.
Missing a few pieces, but worth sharing as an RFC. My idea is to
ensure better test coverage for eu-stack and then
eu-stacktrace+libdwfl_stacktrace by running against a live process
with known content, stopped at a known location, and aggressively
scrubbing output that's known to vary from testrun to testrun.
This is a very basic preview of how that might look. If the approach
is sound, I hope to make it more sophisticated/reliable.
Unanswered questions:
- Scrub more data (e.g. libc symvers) from a more known program.
Scrub stack frame numbers to account for a case where extra frames
appear / are missing at the bottom of the stack?
- Something better than sed for the scrubbing?
- An equivalent eu-stacktrace test will require privileged perf_events
access for profiling data and therefore likely to be skipped by
default. How feasible is it to be enabled on the buildbots, though?
* tests/run-stack-live-test.sh: New test with wild and fuzzy
testrun_compare variant that scrubs inherently unpredictable parts of
the data. Needs to scrub even more.
* tests/Makefile.am (TESTS): Add run-stack-live-test.sh.
Mark Wielaard [Tue, 29 Apr 2025 21:19:47 +0000 (23:19 +0200)]
readelf: Pass around GElf_Ehdr instead of calling gelf_getehdr
handle_core_item calls gelf_getehdr for each item without checking if
the call succeeds. It should always succeed because process_elf_file
already checked gelf_getehdr returned a valid Ehdr before passing it
to handle_notes. Just pass the Ehdr down a couple more function calls.
* readelf.c (handle_core_item): Take const Gelf_Ehdr and use it.
(handle_core_items): Take const Gelf_Ehdr and pass it to
handle_core_item.
(handle_core_note): Take const Gelf_Ehdr and pass it to
handle_core_items.
(handle_notes_data): Pass ehdr to handle_core_note.
eu-stacktrace [12/12]: use dwflst_perf_sample_getframes
Remove the code from src/stacktrace.c that is now covered by
libdwfl_stacktrace/dwflst_perf_frame.c and
dwflst_perf_sample_getframes.
* src/stacktrace.c (show_memory_reads): Remove this verbose option as
the relevant code is inside libdwfl_stacktrace now.
(struct __sample_arg): Remove, handled by
libdwfl_stacktrace/dwflst_perf_frame.c.
(sample_next_thread): Ditto.
(sample_getthread): Ditto.
(copy_word_64): Ditto.
(copy_word_32): Ditto.
(elf_memory_read): Ditto.
(sample_memory_read): Ditto.
(sample_set_initial_registers): Ditto.
(sample_detach): Ditto.
(sample_thread_callbacks): Ditto.
(sysprof_find_dwfl): Now also return the Elf* so that it can be
passed to dwflst_perf_sample_getframes. Don't create sample_arg. Do
record sp in sui->last_sp. Don't dwfl_attach_state,
dwflst_perf_sample_getframes handles that now.
(sysprof_unwind_cb): Adapt to sysprof_find_dwfl changes,
now invoke dwflst_perf_sample_getframes instead of
dwfl_getthread_frames.
(parse_opt): Remove show_memory_reads.
(main): Remove show_memory_reads.
This is a new interface for unwinding that doesn't require the Dwfl to
be attached to a live process (via ptrace) or via corefile. Instead,
data from a perf_events stack sample is provided along with an Elf
struct used to identify the architecture. Based on code from
eu-stacktrace.
* libdwfl_stacktrace/libdwfl_stacktrace.h (dwflst_perf_sample_getframes):
New function.
* libdwfl_stacktrace/dwflst_perf_frame.c
(struct __libdwfl_stacktrace_perf_sample_info): New struct, based on
src/stacktrace.c struct sample_arg.
(sample_next_thread): New function, based on src/stacktrace.c.
(sample_getthread): Ditto.
(copy_word_64): New macro, based on src/stacktrace.c.
(copy_word_32): Ditto.
(copy_word): Ditto.
(elf_memory_read): New function, based on src/stacktrace.c.
(sample_memory_read): Ditto.
(sample_set_initial_registers): Ditto.
(sample_detach): Ditto.
(sample_thread_callbacks): New struct, set of callbacks based on
src/stacktrace.c sample_thread_callbacks.
(dwflst_perf_sample_getframes): New function, based on parts of
src/stacktrace.c sysprof_find_dwfl. If the Dwfl is not attached,
attaches it with sample_thread_callbacks and
__libdwfl_stacktrace_perf_sample_info. Populates the
__libdwfl_stacktrace_perf_sample_info with data from the stack
sample and calls dwfl_getthread_frames to unwind it using the
sample_thread_callbacks.
* libdw/libdw.map (ELFUTILS_0.193_EXPERIMENTAL): Add
dwflst_perf_sample_getframes.
eu-stacktrace [10/12]: use dwflst_tracker_find_pid
Initial minimal change to ensure dwflst_tracker_find_pid is
tested. For now, we keep the additional dwfltab implementation in
stacktrace.c, since it's being used to track statistics.
In future follow-ups, it will be good to switch to storing
eu-stacktrace statistics e.g. in dwfl->process->callbacks_arg. This
requires some additional design to keep the statistics from being lost
when a pid is reused and the corresponding processtracker table entry
is replaced.
* src/stacktrace.c (sysprof_init_dwfl): New function.
(sysprof_find_dwfl): Rename the existing sysprof_init_dwfl.
Also use dwflst_tracker_find_pid with callback.
(sysprof_unwind_cb): Rename the existing sysprof_init_dwfl.
New function that retrieves the Dwfl for a particular PID, or,
if the Dwfl is absent, creates it via a provided callback
and adds it to the table later, when the PID is confirmed
via dwfl_attach_state.
* libdwfl_stacktrace/libdwfl_stacktrace.h (dwflst_tracker_find_pid):
New function.
* libdwfl_stacktrace/dwfl_process_tracker.c (dwflst_tracker_find_pid):
New function; find a Dwfl in the dwfltab or create one using the
provided callback. The newly created Dwfl will be added to the
dwfltab automatically when its pid is confirmed by a call to
dwfl_attach_state.
* libdw/libdw.map: Add dwflst_tracker_find_pid.
libdwfl_stacktrace [8/12]: Dwfl* caching via Dwflst_Process_Tracker
The Dwflst_Process_Tracker also includes a dynamicsizehash cache which
maps process ids to Dwfl * (or rather, dwflst_tracker_dwfl_info *
allowing the table entry to be replaced). Dwfls created from
the tracker are automatically added to it, and removed on dwfl_end().
* libdwfl_stacktrace/libdwfl_stacktraceP.h (dwflst_tracker_dwfl_info):
New typedef, provides indirection to allow a dwfltab entry to be
invalidated.
(struct Dwflst_Process_Tracker): add dwfltab.
(__libdwfl_stacktrace_add_dwfl_to_tracker): New function.
(__libdwfl_stacktrace_remove_dwfl_from_tracker): New function.
* libdwfl_stacktrace/dwflst_process_tracker.c
(dwflst_tracker_begin): Init dwfltab.
(__libdwfl_stacktrace_add_dwfl_to_tracker): New function; add dwfl
to dwfltab.
(__libdwfl_stacktrace_remove_dwfl_from_tracker): New function;
invalidate dwfl entry, since dynamicsizehash doesn't support
outright deletion.
(dwflst_tracker_end): Clean up dwfltab. Lock and iterate the table
to free tracker->dwfltab.table items.
* libdwfl_stacktrace/dwflst_tracker_dwfltab.c: New file, instantiates
lib/dynamicsizehash_concurrent.c to store dwfltracker_dwfl_info
structs.
* libdwfl_stacktrace/dwflst_tracker_dwfltab.h: New file, ditto.
* libdwfl_stacktrace/Makefile.am
(libdwfl_stacktrace_a_SOURCES): Add dwflst_tracker_dwfltab.c.
(noinst_HEADERS): Add dwflst_tracker_dwfltab.h.
* libdwfl/dwfl_frame.c (dwfl_attach_state):
Call __libdwfl_stacktrace_add_dwfl_to_tracker.
* libdwfl/dwfl_end.c (dwfl_end): Add INTDEF.
Call __libdwfl_stacktrace_remove_dwfl_from_tracker.
* libdwfl/libdwflP.h (INTDECLs): Add dwfl_end.
eu-stacktrace [7/12]: use Dwflst_Process_Tracker for Elf * caching
* src/Makefile.am (AM_CPPFLAGS): Include headers from
../libdwfl_stacktrace.
* src/stacktrace.c (tracker): New global variable.
(sample_callbacks): Use dwflst_tracker_linux_proc_find_elf for
caching.
(sysprof_init_dwfl): Use dwflst_tracker_dwfl_begin.
(main): Initialize and clean up tracker.
libdwfl_stacktrace [6/12]: Elf* caching via dwflst_process_tracker
The Dwflst_Process_Tracker includes a dynamicsizehash cache which maps
file paths to Elf * (or rather, dwflst_tracker_elf_info * storing fd
and Elf *). We provide a dwflst_tracker_linux_proc_find_elf callback
which checks the cache for an already-loaded Elf * and, if missing,
populates the cache with the fd returned by dwfl_linux_proc_find_elf.
Later, open_elf updates the cache with the Elf * for that fd. The
commented asserts in dwflst_tracker_cache_elf still catch some cases
where a redundant Elf * is being created without checking the cache.
Since the Elf * outlasts the Dwfl that created it, we use the
(convenient, already-existing) reference count field in Elf * to
retain the data in the table. Then dwfl_end calling elf_end will
decrement the refcount cleanly, and dwflst_tracker_end will issue
another elf_end call.
* libdwfl_stacktrace/libdwfl_stacktrace.h
(dwflst_tracker_find_cached_elf): New function.
(dwflst_tracker_cache_elf): New function.
(dwflst_module_gettracker): New function, gives external users
a way to access Dwflst_Process_Tracker given a Dwfl_Module.
(dwflst_tracker_linux_proc_find_elf): New function, serves as a
cached version of the dwfl_linux_proc_find_elf callback.
* libdwfl_stacktrace/libdwfl_stacktraceP.h (dwflst_tracker_elf_info):
New struct typedef.
(struct Dwflst_Process_Tracker): Add dynamicsizehash table of
dwflst_tracker_elf_info structs + associated rwlock.
(INTDECLs): Add INTDECL for dwflst_tracker_find_cached_elf,
dwflst_tracker_cache_elf, dwflst_module_gettracker.
* libdwfl_stacktrace/dwflst_tracker_elftab.c: New file, instantiates
lib/dynamicsizehash_concurrent.c to store dwflst_tracker_elf_info
structs.
* libdwfl_stacktrace/dwflst_tracker_elftab.h: New file, ditto.
* libdwfl_stacktrace/libdwfl_stacktrace_next_prime.c: New file.
* libdwfl_stacktrace/dwflst_process_tracker.c (dwflst_tracker_begin):
Init elftab.
(dwflst_tracker_end): Clean up elftab. Lock and iterate the hash to
free tracker->elftab.table items.
* libdwfl_stacktrace/dwflst_tracker_find_elf.c: New file, implements a
find_elf callback that wraps dwfl_linux_proc_find_elf with
additional caching logic, and an API to access the
Dwflst_Process_Tracker Elf cache when implementing a custom find_elf
callback.
* libdwfl_stacktrace/Makefile.am (libdwfl_stacktrace_a_SOURCES): Add
dwflst_tracker_find_elf.c, dwflst_tracker_elftab.c,
libdwfl_stacktrace_next_prime.c.
(noinst_HEADERS): Add dwflst_tracker_elftab.h.
* libdw/libdw.map: Add dwflst_tracker_find_cached_elf,
dwflst_tracker_cache_elf,
dwflst_module_gettracker,
dwflst_tracker_linux_proc_find_elf.
* libdwfl/Makefile.am (AM_CPPFLAGS): Include headers from
../libdwfl_stacktrace.
* libdwfl/dwfl_module_getdwarf.c (open_elf): Cache file->elf in
Dwflst_Process_Tracker. Must be done here as
dwfl_linux_proc_find_elf opens an fd but does not yet create the
Elf *.
New data structure to coordinate caching Elf data among multiple Dwfl
structs attached to different processes. Meant to reduce the overhead
for profilers that use elfutils for unwinding.
The caching is well-justified, as the prior approach (e.g. in
eu-stacktrace, sysprof-live-unwinder) of creating a separate Dwfl per
process was wildly redundant, opening ~hundreds of file descriptors
just for each common library such as libc.so and leaking the data.
Initial patch just introduces the struct, to be filled in by the rest
of the patch series.
* libdwfl_stacktrace/libdwfl_stacktrace.h
(Dwflst_Process_Tracker): New struct.
(dwflst_tracker_begin): New function.
(dwflst_tracker_dwfl_begin): New function.
(dwflst_tracker_end): New function.
* libdw/libdw.map: Add new functions.
* libdwfl_stacktrace/libdwfl_stacktraceP.h
(struct Dwflst_Process_Tracker): New struct.
* libdwfl/libdwflP.h (Dwflst_Process_Tracker): Typedef forward decl.
(struct Dwfl): Add 'tracker' field.
* libdwfl_stacktrace/Makefile.am (libdwfl_stacktrace_a_SOURCES):
Add dwflst_process_tracker.c.
* libdwfl_stacktrace/dwflst_process_tracker.c: New file.
(dwflst_tracker_begin): Initialize the tracker.
(dwflst_tracker_dwfl_begin): Initialize Dwfl * with specified tracker.
(dwflst_tracker_end): Deallocate the tracker.
Subsequent patches in the series introduce a new library for
tracking/caching Elf structs across multiple processes and wrapping
the libebl perf register handling (since libebl is a private
interface).
In this patch, add the library and an interface to access the set of
registers that libdwfl needs to allow proper unwinding of stack sample
data. (Unwinding with a smaller set of registers can be attempted,
but will yield a truncated call chain in most cases.)
(Accessing the set of registers feels like an implementation detail,
but the profiler invoking perf_event_open and elfutils unwinding code
need to agree on the exact set of registers being requested. So it's
best for the profiler to ask elfutils for this detail.)
* libdwfl_stacktrace/Makefile.am: New file.
* libdwfl_stacktrace/libdwfl_stacktrace.h: New file, library for
tracking/caching Elf structs and unwinding across multiple
processes.
* libdwfl_stacktrace/libdwfl_stacktraceP.h: New file.
* libdwfl_stacktrace/dwflst_perf_frame.c: New file.
(dwflst_perf_sample_preferred_regs_mask): New function.
* libdw/libdw.map: Add dwflst_perf_sample_preferred_regs_mask.
* NEWS: Add NEWS item about libdwfl_stacktrace.
* configure.ac: Add libdwfl_stacktrace/Makefile.
* Makefile.am (SUBDIRS): Add libdwfl_stacktrace.
* libdw/Makefile.am (libdw_so_LIBS): Add libdwfl_stacktrace.
(libdwfl_stacktrace_objects): Add libdwfl_stacktrace.manifest.
(libdw_a_LIBADD): Add libdwfl_stacktrace_objects.
* config/elfutils.spec.in (%files devel): Add libdwfl_stacktrace.h.
libebl [3/12]: eu-stacktrace: use new register handling api
Dummy commit to show how the sample_set_initial_registers callback in
eu-stacktrace would use the proper libebl
ebl_set_initial_registers_sample function (if it were public).
* src/Makefile.am (stacktrace_LDADD): Add libebl.
* src/stacktrace.c (sample_registers_cb): New function,
though identical to pid_thread_state_registers_cb.
(sample_set_initial_registers): (XXX Invoke
ebl_set_initial_registers_sample instead of containing
platform-specific code directly. This is now commented out.
Patch12 in the series replaces with code in
libdwfl_stacktrace/dwflst_perf_frame.c.)
libebl [1/12]: api for perf register handling, start with x86_64
First patch of a series that reworks eu-stacktrace functionality
into a library interface for other profiling tools.
* * *
As it happens, Linux perf_events and DWARF can prescribe completely
different layouts for the register file, requiring non-obvious code
for translation. This makes sense to put in libebl if we want
profilers to handle perf sample data with elfutils.
Start with the x86_64/i386 implementation taken from eu-stacktrace. The
code has been generalized to accept other perf register masks besides
the 'preferred' one.
* backends/Makefile.am (i386_SRCS): Add i386_initreg_sample.c.
(x86_64_SRCS): Add x86_64_initreg_sample.c.
(noinst_HEADERS): Add libebl_PERF_FLAGS.h,
linux-perf-regs.c, x86_initreg_sample.c.
* backends/libebl_PERF_FLAGS.h: New file.
* backends/linux-perf-regs.c: New file.
(perf_sample_find_reg): Index into a perf register array whose
contents are described by regs_mask. The target index is the index
of the register in the enum defined by
linux arch/N/include/uapi/asm/perf_regs.h on arch N.
* backends/x86_initreg_sample.c: New file, implements a generalized
version of the eu-stacktrace register shuffling for x86-64/i386.
* backends/x86_64_initreg_sample.c: New file, specializes
x86_initreg_sample.c for x86-64.
* backends/i386_initreg_sample.c: New file, specializes
i386_initreg_sample.c for i386.
* backends/x86_64_init.c (x86_64_init): Add initialization for
set_initial_registers_sample, perf_frame_regs_mask,
sample_base_addr, sample_pc.
* backends/i386_init.c (i386_init): Add initialization for
set_initial_registers_sample, perf_frame_regs_mask,
sample_base_addr, sample_pc.
* libebl/Makefile.am (libebl_a_SOURCES): Add eblinitreg_sample.c.
* libebl/ebl-hooks.h (set_initial_registers_sample): New hook.
(sample_base_addr): Ditto.
(sample_pc): Ditto.
* libebl/eblinitreg_sample.c: New file, implements ebl interface to
set_initial_registers_sample, sample_base_addr, sample_pc
backend hooks.
* libebl/libebl.h (ebl_set_initial_registers_sample): New function.
(ebl_perf_frame_regs_mask): New function.
(ebl_sample_base_addr): New function.
(ebl_sample_pc): New function.
* libebl/libeblP.h (struct ebl): Add perf_frame_regs_mask field
giving the preferred register mask.
Mark Wielaard [Wed, 2 Apr 2025 11:35:31 +0000 (13:35 +0200)]
libdw: Add Nim language and dwarf_srclang tests
DW_LANG_Nim and DW_LNAME_Nim were added to the DWARF languages.
While adding them, and the default lower bounds, I noticed DW_LANG_V
and DW_LANG_Algol68 where missing in srclang_to_language an internal
helper function.
Testing through the public api is not that easy since you need a
Dwarf_Die with the right attributes. So this patch adds a way to
compile an individual source file with an optional main function that
can directly access the internal/static functions.
Note that it is almost generic. But even though using .SECONDEXPANSION
I couldn't figure out how to create the equivalent of a rule starting
with %_check$(EXEEXT) target. So for now the rule has to repeated for
every new _check TEST. And there needs to be a line to tell make dist
to not expect the fake source: nodist_src_check_SOURCES = src_check.c
The new test pointed out that there were a few more bugs with
DW_LANG_Dylan and DW_LNAME_Mojo. Also fix those.
* libdw/dwarf.h: Add DW_LANG_Nim and DW_LNAME_Nim.
* libdw/Makefile.am: Add check_PROGRAMS and TESTS for
dwarf_srclang_check.
* libdw/dwarf_default_lower_bound.c
(dwarf_default_lower_bound): Add DW_LANG_Nim.
(dwarf_language_lower_bound): Add DW_LNAME_Nim.
* libdw/dwarf_srclang.c (srclang_to_language): Handle
DW_LANG_Dylan, DW_LANG_V, DW_LANG_Algol68 and DW_LANG_Nim.
(language_to_srclang): Fix DW_LNAME_Mojo. Add DW_LNAME_Nim.
(test_lang): New function guarded by MAIN_CHECK.
(main): Likewise.
Mark Wielaard [Fri, 4 Apr 2025 11:50:04 +0000 (13:50 +0200)]
lib: Prevent double inclusion of config.h through system.h
Files that include both <config.h> and "system.h" might include
config.h twice. Prevent that by adding a guard check in system.h
before including config.h.
* lib/crc32.c: Use #ifdef HAVE_CONFIG_H instead of #if.
* lib/error.h: Check HAVE_CONFIG_H before including config.h.
* lib/system.h: Check both HAVE_CONFIG_H and whether
EU_CONFIG_H is defined before including config.h.
Suggested-by: Dmitry V. Levin <ldv@strace.io> Signed-off-by: Mark Wielaard <mark@klomp.org>
Michael Trapp [Thu, 27 Mar 2025 16:06:35 +0000 (17:06 +0100)]
debuginfod: add --listen-address option
Use MHD_OPTION_SOCK_ADDR to bind the http listen socket to a single address.
The address should be an IPv4 or IPv6 address configured on the system:
--listen-address=127.0.0.1
--listen-address=::1
--listen-address='LOCAL_IPv4|IPv6_ADDRESS'
As debuginfod does not include any security features, a listen on the
localhost address is sufficient for a HTTP/HTTPS reverse-proxy setup.
Signed-off-by: Michael Trapp <michael.trapp@sap.com>
* configure.ac: Add --enable-helgrind option.
* tests/Makefile.am: If USE_HELGRIND is true, then include
--tool=helgrind in the valgrind command that tests are run
under.
projects / thirdparty / elfutils.git / log
