Gives an idea of how much more flexibility is needed for the munging
to work across a variety of systems. Also concerned that elfutils
certainly runs on non-linux non-GLIBC systems too, worth a brief test
on musl and bsd?
Missing a few pieces, but worth sharing as an RFC. My idea is to
ensure better test coverage for eu-stack and then
eu-stacktrace+libdwfl_stacktrace by running against a live process
with known content, stopped at a known location, and aggressively
scrubbing output that's known to vary from testrun to testrun.
This is a very basic preview of how that might look. If the approach
is sound, I hope to make it more sophisticated/reliable.
Unanswered questions:
- Scrub more data (e.g. libc symvers) from a more known program.
Scrub stack frame numbers to account for a case where extra frames
appear / are missing at the bottom of the stack?
- Something better than sed for the scrubbing?
- An equivalent eu-stacktrace test will require privileged perf_events
access for profiling data and therefore likely to be skipped by
default. How feasible is it to be enabled on the buildbots, though?
* tests/run-stack-live-test.sh: New test with wild and fuzzy
testrun_compare variant that scrubs inherently unpredictable parts of
the data. Needs to scrub even more.
* tests/Makefile.am (TESTS): Add run-stack-live-test.sh.
eu-stacktrace [12/12]: use dwflst_perf_sample_getframes
Changes for v4:
- Separate out libdwfl_stacktrace, as requested.
Changes for v3:
- Fix initialization of elf in sysprof_init_dwfl
(previously triggered -Wmaybe-uninitialized).
* * *
Remove the code from src/stacktrace.c that is now covered by
libdwfl_stacktrace/dwflst_perf_frame.c and
dwflst_perf_sample_getframes.
* src/stacktrace.c (show_memory_reads): Remove this verbose option as
the relevant code is inside libdwfl_stacktrace now.
(struct __sample_arg): Remove, handled by
libdwfl_stacktrace/dwflst_perf_frame.c.
(sample_next_thread): Ditto.
(sample_getthread): Ditto.
(copy_word_64): Ditto.
(copy_word_32): Ditto.
(elf_memory_read): Ditto.
(sample_memory_read): Ditto.
(sample_set_initial_registers): Ditto.
(sample_detach): Ditto.
(sample_thread_callbacks): Ditto.
(sysprof_find_dwfl): Now also return the Elf* so that it can be
passed to dwflst_perf_sample_getframes. Don't create sample_arg. Do
record sp in sui->last_sp. Don't dwfl_attach_state,
dwflst_perf_sample_getframes handles that now.
(sysprof_unwind_cb): Adapt to sysprof_find_dwfl changes,
now invoke dwflst_perf_sample_getframes instead of
dwfl_getthread_frames.
(parse_opt): Remove show_memory_reads.
(main): Remove show_memory_reads.
- use const void *stack, not void *, to allow users
to pass a const stack sample
Changes for v2:
- use renamed __libdwfl_set_initial_registers_thread
- oops, should use provided sample_arg->perf_regs_mask
in sample_set_initial registers
* * *
This is a new interface for unwinding that doesn't require the Dwfl to
be attached to a live process (via ptrace) or via corefile. Instead,
data from a perf_events stack sample is provided along with an Elf
struct used to identify the architecture. Based on code from
eu-stacktrace.
* libdwfl_stacktrace/libdwfl_stacktrace.h (dwflst_perf_sample_getframes):
New function.
* libdwfl_stacktrace/dwflst_perf_frame.c
(struct __libdwfl_stacktrace_perf_sample_info): New struct, based on
src/stacktrace.c struct sample_arg.
(sample_next_thread): New function, based on src/stacktrace.c.
(sample_getthread): Ditto.
(copy_word_64): New macro, based on src/stacktrace.c.
(copy_word_32): Ditto.
(copy_word): Ditto.
(elf_memory_read): New function, based on src/stacktrace.c.
(sample_memory_read): Ditto.
(sample_set_initial_registers): Ditto.
(sample_detach): Ditto.
(sample_thread_callbacks): New struct, set of callbacks based on
src/stacktrace.c sample_thread_callbacks.
(dwflst_perf_sample_getframes): New function, based on parts of
src/stacktrace.c sysprof_find_dwfl. If the Dwfl is not attached,
attaches it with sample_thread_callbacks and
__libdwfl_stacktrace_perf_sample_info. Populates the
__libdwfl_stacktrace_perf_sample_info with data from the stack
sample and calls dwfl_getthread_frames to unwind it using the
sample_thread_callbacks.
* libdw/libdw.map (ELFUTILS_0.193): Add dwflst_perf_sample_getframes.
eu-stacktrace [10/12]: use dwflst_tracker_find_pid
Changes for v6:
- Minor fixes as requested.
Changes for v4:
- Separate out libdwfl_stacktrace, as requested.
* * *
Initial minimal change to ensure dwflst_tracker_find_pid is
tested. For now, we keep the additional dwfltab implementation in
stacktrace.c, since it's being used to track statistics.
In future follow-ups, it will be good to switch to storing
eu-stacktrace statistics e.g. in dwfl->process->callbacks_arg. This
requires some additional design to keep the statistics from being lost
when a pid is reused and the corresponding processtracker table entry
is replaced.
* src/stacktrace.c (sysprof_init_dwfl): New function.
(sysprof_find_dwfl): Rename the existing sysprof_init_dwfl.
Also use dwflst_tracker_find_pid with callback.
(sysprof_unwind_cb): Rename the existing sysprof_init_dwfl.
New function that retrieves the Dwfl for a particular PID, or,
if the Dwfl is absent, creates it via a provided callback
and adds it to the table later, when the PID is confirmed
via dwfl_attach_state.
* libdwfl_stacktrace/libdwfl_stacktrace.h (dwflst_tracker_find_pid):
New function.
* libdwfl_stacktrace/dwfl_process_tracker.c (dwflst_tracker_find_pid):
New function; find a Dwfl in the dwfltab or create one using the
provided callback. The newly created Dwfl will be added to the
dwfltab automatically when its pid is confirmed by a call to
dwfl_attach_state.
* libdw/libdw.map: Add dwflst_tracker_find_pid.
Serhei Makarov [Fri, 7 Mar 2025 17:41:40 +0000 (12:41 -0500)]
libdwfl_stacktrace [8/12]: Dwfl* caching via Dwflst_Process_Tracker
Changes for v6:
- Minor fixes as requested.
Changes for v4:
- Separate out libdwfl_stacktrace, as requested.
Changes for v3:
- Handle dwfl->process == NULL case in __libdwfl_remove_dwfl_from_tracker.
* * *
The Dwflst_Process_Tracker also includes a dynamicsizehash cache which
maps process ids to Dwfl * (or rather, dwflst_tracker_dwfl_info *
allowing the table entry to be replaced). Dwfls created from
the tracker are automatically added to it, and removed on dwfl_end().
* libdwfl_stacktrace/libdwfl_stacktraceP.h (dwflst_tracker_dwfl_info):
New typedef, provides indirection to allow a dwfltab entry to be
invalidated.
(struct Dwflst_Process_Tracker): add dwfltab.
(__libdwfl_stacktrace_add_dwfl_to_tracker): New function.
(__libdwfl_stacktrace_remove_dwfl_from_tracker): New function.
* libdwfl_stacktrace/dwflst_process_tracker.c
(dwflst_tracker_begin): Init dwfltab.
(__libdwfl_stacktrace_add_dwfl_to_tracker): New function; add dwfl
to dwfltab.
(__libdwfl_stacktrace_remove_dwfl_from_tracker): New function;
invalidate dwfl entry, since dynamicsizehash doesn't support
outright deletion.
(dwflst_tracker_end): Clean up dwfltab. Lock and iterate the table
to free tracker->dwfltab.table items.
* libdwfl_stacktrace/dwflst_tracker_dwfltab.c: New file, instantiates
lib/dynamicsizehash_concurrent.c to store dwfltracker_dwfl_info
structs.
* libdwfl_stacktrace/dwflst_tracker_dwfltab.h: New file, ditto.
* libdwfl_stacktrace/Makefile.am
(libdwfl_stacktrace_a_SOURCES): Add dwflst_tracker_dwfltab.c.
(noinst_HEADERS): Add dwflst_tracker_dwfltab.h.
* libdwfl/dwfl_frame.c (dwfl_attach_state):
Call __libdwfl_stacktrace_add_dwfl_to_tracker.
* libdwfl/dwfl_end.c (dwfl_end): Add INTDEF.
Call __libdwfl_stacktrace_remove_dwfl_from_tracker.
* libdwfl/libdwflP.h (INTDECLs): Add dwfl_end.
Serhei Makarov [Tue, 28 Jan 2025 17:27:34 +0000 (12:27 -0500)]
eu-stacktrace [7/12]: use Dwflst_Process_Tracker for Elf * caching
Changes for v4:
- Separate out libdwfl_stacktrace, as requested.
* * *
* src/Makefile.am (AM_CPPFLAGS): Include headers from
../libdwfl_stacktrace.
* src/stacktrace.c (tracker): New global variable.
(sample_callbacks): Use dwflst_tracker_linux_proc_find_elf for
caching.
(sysprof_init_dwfl): Use dwflst_tracker_dwfl_begin.
(main): Initialize and clean up tracker.
libdwfl_stacktrace [6/12]: Elf* caching via dwflst_process_tracker
Changes for v6:
- Minor fixes as requested.
Changes for v5:
- Bugfixes in dwflst_tracker_find_elf.c.
Changes for v4:
- Separate out libdwfl_stacktrace, as requested.
- dwfl_module_getdwarf.c now uses the dwflst_tracker_cache_elf()
interface instead of editing the elftab directly.
Changes for v3:
- Reworked elftab to incorporate dev/ino into the caching key
(to allow caching modules from different filesystems
e.g. a main filesystem and a container filesystem)
and guard more carefully against collisions.
- Add external API for implementing a custom
find_elf callback that reads/writes the cache.
Changes for v2:
- Add locking for elftab. This is needed in addition to the
intrinsic locking in dynamicsizehash_concurrent to avoid
having cache_elf expose an incomplete dwfltracker_elf_info*
entry to other threads while its data is being populated /
replaced.
- Tidy dwfl_process_tracker_find_elf.c into the main find_elf
callback and two functions to consider (in future) making into
a public api for custom cached callbacks.
* * *
The Dwflst_Process_Tracker includes a dynamicsizehash cache which maps
file paths to Elf * (or rather, dwflst_tracker_elf_info * storing fd
and Elf *). We provide a dwflst_tracker_linux_proc_find_elf callback
which checks the cache for an already-loaded Elf * and, if missing,
populates the cache with the fd returned by dwfl_linux_proc_find_elf.
Later, open_elf updates the cache with the Elf * for that fd. The
commented asserts in dwflst_tracker_cache_elf still catch some cases
where a redundant Elf * is being created without checking the cache.
Since the Elf * outlasts the Dwfl that created it, we use the
(convenient, already-existing) reference count field in Elf * to
retain the data in the table. Then dwfl_end calling elf_end will
decrement the refcount cleanly, and dwflst_tracker_end will issue
another elf_end call.
* libdwfl_stacktrace/libdwfl_stacktrace.h
(dwflst_tracker_find_cached_elf): New function.
(dwflst_tracker_cache_elf): New function.
(dwflst_module_gettracker): New function, gives external users
a way to access Dwflst_Process_Tracker given a Dwfl_Module.
(dwflst_tracker_linux_proc_find_elf): New function, serves as a
cached version of the dwfl_linux_proc_find_elf callback.
* libdwfl_stacktrace/libdwfl_stacktraceP.h (dwflst_tracker_elf_info):
New struct typedef.
(struct Dwflst_Process_Tracker): Add dynamicsizehash table of
dwflst_tracker_elf_info structs + associated rwlock.
(INTDECLs): Add INTDECL for dwflst_tracker_find_cached_elf,
dwflst_tracker_cache_elf, dwflst_module_gettracker.
* libdwfl_stacktrace/dwflst_tracker_elftab.c: New file, instantiates
lib/dynamicsizehash_concurrent.c to store dwflst_tracker_elf_info
structs.
* libdwfl_stacktrace/dwflst_tracker_elftab.h: New file, ditto.
* libdwfl_stacktrace/libdwfl_stacktrace_next_prime.c: New file.
* libdwfl_stacktrace/dwflst_process_tracker.c (dwflst_tracker_begin):
Init elftab.
(dwflst_tracker_end): Clean up elftab. Lock and iterate the hash to
free tracker->elftab.table items.
* libdwfl_stacktrace/dwflst_tracker_find_elf.c: New file, implements a
find_elf callback that wraps dwfl_linux_proc_find_elf with
additional caching logic, and an API to access the
Dwflst_Process_Tracker Elf cache when implementing a custom find_elf
callback.
* libdwfl_stacktrace/Makefile.am (libdwfl_stacktrace_a_SOURCES): Add
dwflst_tracker_find_elf.c, dwflst_tracker_elftab.c,
libdwfl_stacktrace_next_prime.c.
(noinst_HEADERS): Add dwflst_tracker_elftab.h.
* libdw/libdw.map: Add dwflst_tracker_find_cached_elf,
dwflst_tracker_cache_elf,
dwflst_module_gettracker,
dwflst_tracker_linux_proc_find_elf.
* libdwfl/Makefile.am (AM_CPPFLAGS): Include headers from
../libdwfl_stacktrace.
* libdwfl/dwfl_module_getdwarf.c (open_elf): Cache file->elf in
Dwflst_Process_Tracker. Must be done here as
dwfl_linux_proc_find_elf opens an fd but does not yet create the
Elf *.
New data structure to coordinate caching Elf data among multiple Dwfl
structs attached to different processes. Meant to reduce the overhead
for profilers that use elfutils for unwinding.
The caching is well-justified, as the prior approach (e.g. in
eu-stacktrace, sysprof-live-unwinder) of creating a separate Dwfl per
process was wildly redundant, opening ~hundreds of file descriptors
just for each common library such as libc.so and leaking the data.
Initial patch just introduces the struct, to be filled in by the rest
of the patch series.
* libdwfl_stacktrace/libdwfl_stacktrace.h
(Dwflst_Process_Tracker): New struct.
(dwflst_tracker_begin): New function.
(dwflst_tracker_dwfl_begin): New function.
(dwflst_tracker_end): New function.
* libdw/libdw.map: Add new functions.
* libdwfl_stacktrace/libdwfl_stacktraceP.h
(struct Dwflst_Process_Tracker): New struct.
* libdwfl/libdwflP.h (Dwflst_Process_Tracker): Typedef forward decl.
(struct Dwfl): Add 'tracker' field.
* libdwfl_stacktrace/Makefile.am (libdwfl_stacktrace_a_SOURCES):
Add dwflst_process_tracker.c.
* libdwfl_stacktrace/dwflst_process_tracker.c: New file.
(dwflst_tracker_begin): Initialize the tracker.
(dwflst_tracker_dwfl_begin): Initialize Dwfl * with specified tracker.
(dwflst_tracker_end): Deallocate the tracker.
- Separate ELFUTILS_0.193_EXPERIMENTAL namespace, mark library experimental.
Changes for v4:
- Separate out libdwfl_stacktrace, as requested.
Changes for v2:
- guard the linux/perf_events.h include properly with an #if defined __linux__
* * *
Subsequent patches in the series introduce a new library for
tracking/caching Elf structs across multiple processes and wrapping
the libebl perf register handling (since libebl is a private
interface).
In this patch, add the library and an interface to access the set of
registers that libdwfl needs to allow proper unwinding of stack sample
data. (Unwinding with a smaller set of registers can be attempted,
but will yield a truncated call chain in most cases.)
(Accessing the set of registers feels like an implementation detail,
but the profiler invoking perf_event_open and elfutils unwinding code
need to agree on the exact set of registers being requested. So it's
best for the profiler to ask elfutils for this detail.)
* libdwfl_stacktrace/Makefile.am: New file.
* libdwfl_stacktrace/libdwfl_stacktrace.h: New file, library for
tracking/caching Elf structs and unwinding across multiple
processes.
* libdwfl_stacktrace/libdwfl_stacktraceP.h: New file.
* libdwfl_stacktrace/dwflst_perf_frame.c: New file.
(dwflst_perf_sample_preferred_regs_mask): New function.
* libdw/libdw.map: Add dwflst_perf_sample_preferred_regs_mask.
* NEWS: Add NEWS item about libdwfl_stacktrace.
* configure.ac: Add libdwfl_stacktrace/Makefile.
* Makefile.am (SUBDIRS): Add libdwfl_stacktrace.
* libdw/Makefile.am (libdw_so_LIBS): Add libdwfl_stacktrace.
(libdwfl_stacktrace_objects): Add libdwfl_stacktrace.manifest.
(libdw_a_LIBADD): Add libdwfl_stacktrace_objects.
* config/elfutils.spec.in (%files devel): Add libdwfl_stacktrace.h.
Serhei Makarov [Thu, 5 Dec 2024 21:16:00 +0000 (16:16 -0500)]
libebl [3/12]: eu-stacktrace: use new register handling api
Changes for v4:
- Since __libdwfl_set_initial_registers_thread is now private to
libdwfl, the modified code in this patch has been disabled.
* * *
Dummy commit to show how the sample_set_initial_registers callback in
eu-stacktrace would use the proper libebl
ebl_set_initial_registers_sample function (if it were public).
* src/Makefile.am (stacktrace_LDADD): Add libebl.
* src/stacktrace.c (sample_registers_cb): New function,
though identical to pid_thread_state_registers_cb.
(sample_set_initial_registers): (XXX Invoke
ebl_set_initial_registers_sample instead of containing
platform-specific code directly. This is now commented out.
Patch12 in the series replaces with code in
libdwfl_stacktrace/dwflst_perf_frame.c.)
libebl [1/12]: api for perf register handling, start with x86_64
Changes for v6:
- Minor fixes as requested.
Changes for v2:
- Merged commit 11 into commit 1 (both sample_base_addr/sample_pc and
set_initial_registers_sample make sense to introduce in the same
commit).
- Added i386 backend and factored out common code.
* * *
First patch of a series that reworks eu-stacktrace functionality
into a library interface for other profiling tools.
* * *
As it happens, Linux perf_events and DWARF can prescribe completely
different layouts for the register file, requiring non-obvious code
for translation. This makes sense to put in libebl if we want
profilers to handle perf sample data with elfutils.
Start with the x86_64/i386 implementation taken from eu-stacktrace. The
code has been generalized to accept other perf register masks besides
the 'preferred' one.
* backends/Makefile.am (i386_SRCS): Add i386_initreg_sample.c.
(x86_64_SRCS): Add x86_64_initreg_sample.c.
(noinst_HEADERS): Add libebl_PERF_FLAGS.h,
linux-perf-regs.c, x86_initreg_sample.c.
* backends/libebl_PERF_FLAGS.h: New file.
* backends/linux-perf-regs.c: New file.
(perf_sample_find_reg): Index into a perf register array whose
contents are described by regs_mask. The target index is the index
of the register in the enum defined by
linux arch/N/include/uapi/asm/perf_regs.h on arch N.
* backends/x86_initreg_sample.c: New file, implements a generalized
version of the eu-stacktrace register shuffling for x86-64/i386.
* backends/x86_64_initreg_sample.c: New file, specializes
x86_initreg_sample.c for x86-64.
* backends/i386_initreg_sample.c: New file, specializes
i386_initreg_sample.c for i386.
* backends/x86_64_init.c (x86_64_init): Add initialization for
set_initial_registers_sample, perf_frame_regs_mask,
sample_base_addr, sample_pc.
* backends/i386_init.c (i386_init): Add initialization for
set_initial_registers_sample, perf_frame_regs_mask,
sample_base_addr, sample_pc.
* libebl/Makefile.am (libebl_a_SOURCES): Add eblinitreg_sample.c.
* libebl/ebl-hooks.h (set_initial_registers_sample): New hook.
(sample_base_addr): Ditto.
(sample_pc): Ditto.
* libebl/eblinitreg_sample.c: New file, implements ebl interface to
set_initial_registers_sample, sample_base_addr, sample_pc
backend hooks.
* libebl/libebl.h (ebl_set_initial_registers_sample): New function.
(ebl_perf_frame_regs_mask): New function.
(ebl_sample_base_addr): New function.
(ebl_sample_pc): New function.
* libebl/libeblP.h (struct ebl): Add perf_frame_regs_mask field
giving the preferred register mask.
Mark Wielaard [Wed, 2 Apr 2025 11:35:31 +0000 (13:35 +0200)]
libdw: Add Nim language and dwarf_srclang tests
DW_LANG_Nim and DW_LNAME_Nim were added to the DWARF languages.
While adding them, and the default lower bounds, I noticed DW_LANG_V
and DW_LANG_Algol68 where missing in srclang_to_language an internal
helper function.
Testing through the public api is not that easy since you need a
Dwarf_Die with the right attributes. So this patch adds a way to
compile an individual source file with an optional main function that
can directly access the internal/static functions.
Note that it is almost generic. But even though using .SECONDEXPANSION
I couldn't figure out how to create the equivalent of a rule starting
with %_check$(EXEEXT) target. So for now the rule has to repeated for
every new _check TEST. And there needs to be a line to tell make dist
to not expect the fake source: nodist_src_check_SOURCES = src_check.c
The new test pointed out that there were a few more bugs with
DW_LANG_Dylan and DW_LNAME_Mojo. Also fix those.
* libdw/dwarf.h: Add DW_LANG_Nim and DW_LNAME_Nim.
* libdw/Makefile.am: Add check_PROGRAMS and TESTS for
dwarf_srclang_check.
* libdw/dwarf_default_lower_bound.c
(dwarf_default_lower_bound): Add DW_LANG_Nim.
(dwarf_language_lower_bound): Add DW_LNAME_Nim.
* libdw/dwarf_srclang.c (srclang_to_language): Handle
DW_LANG_Dylan, DW_LANG_V, DW_LANG_Algol68 and DW_LANG_Nim.
(language_to_srclang): Fix DW_LNAME_Mojo. Add DW_LNAME_Nim.
(test_lang): New function guarded by MAIN_CHECK.
(main): Likewise.
Mark Wielaard [Fri, 4 Apr 2025 11:50:04 +0000 (13:50 +0200)]
lib: Prevent double inclusion of config.h through system.h
Files that include both <config.h> and "system.h" might include
config.h twice. Prevent that by adding a guard check in system.h
before including config.h.
* lib/crc32.c: Use #ifdef HAVE_CONFIG_H instead of #if.
* lib/error.h: Check HAVE_CONFIG_H before including config.h.
* lib/system.h: Check both HAVE_CONFIG_H and whether
EU_CONFIG_H is defined before including config.h.
Suggested-by: Dmitry V. Levin <ldv@strace.io> Signed-off-by: Mark Wielaard <mark@klomp.org>
Michael Trapp [Thu, 27 Mar 2025 16:06:35 +0000 (17:06 +0100)]
debuginfod: add --listen-address option
Use MHD_OPTION_SOCK_ADDR to bind the http listen socket to a single address.
The address should be an IPv4 or IPv6 address configured on the system:
--listen-address=127.0.0.1
--listen-address=::1
--listen-address='LOCAL_IPv4|IPv6_ADDRESS'
As debuginfod does not include any security features, a listen on the
localhost address is sufficient for a HTTP/HTTPS reverse-proxy setup.
Signed-off-by: Michael Trapp <michael.trapp@sap.com>
* configure.ac: Add --enable-helgrind option.
* tests/Makefile.am: If USE_HELGRIND is true, then include
--tool=helgrind in the valgrind command that tests are run
under.
Mark Wielaard [Sun, 9 Mar 2025 01:50:24 +0000 (02:50 +0100)]
libdw: Make dwarf_srclang forward compatible with DW_AT_language_name
For programs which use dwarf_srclang it might be useful to translate a
DW_LNAME (plus version) into a DW_LANG constant if there is no
DW_AT_language constant, but there is a DW_AT_language_name (and
DW_AT_language_version).
* libdw/dwarf_srclang.c (language_to_srclang): New function.
(dwarf_srclang): If there is no DW_AT_language, try
DW_AT_language_name and DW_AT_language_version, use
language_to_srclang.
Mark Wielaard [Thu, 30 Jan 2025 17:53:25 +0000 (18:53 +0100)]
libdw: Add dwarf_language and dwarf_language_lower_bound functions.
dwarf_language returns a DW_LNAME constant for a CU Die. If the CU Die
has a DW_AT_language_name attribute dwarf_language will return it and
the DW_AT_language_version attribute value. Otherwise dwarf_language
will lookup the (old style) DW_AT_language attribute and translate the
DW_LANG constant into a DW_LNAME constant and version using a new
static function srclang_to_language.
The dwarf_language_lower_bound function works just like the
dwarf_default_lower_bound function, but takes a DW_LNAME constant
instead of a DW_LANG constant.
Adds a new test to make sure dwarf_language_lower_bound handles all
known DW_LNAME constants.
* NEWS: Add new functions.
* libdw/libdw.map (ELFUTILS_0.193): New section with new functions.
* libdw/libdw.h (dwarf_srclang): Add comment explaining this
returns DW_LANG constants.
(dwarf_language): New function.
(dwarf_default_lower_bound): Add comment explaining this works on
DW_LANG constants.
(dwarf_language_lower_bound): New function.
* libdw/libdwP.h: INTDECL dwarf_language and
dwarf_language_lower_bound.
* libdw/dwarf_srclang.c (srclang_to_language): New function.
(dwarf_language): Likewise.
* libdw/dwarf_default_lower_bound.c (dwarf_language_lower_bound):
New function.
* libdw/dwarf_getfuncs.c (dwarf_getfuncs): Use dwarf_language and
dwarf_language_lower_bound.
* libdw/dwarf_aggregate_size.c (array_size): Likewise.
* tests/dwarf_language_lower_bound.c: New test.
* tests/Makefile.am (check_PROGRAMS): Add dwarf_language_lower_bound.
(TESTS): Likewise.
(dwarf_language_lower_bound_LDADD): New variable.
Mark Wielaard [Wed, 29 Jan 2025 14:35:06 +0000 (15:35 +0100)]
dwarf.h: Add DWARF v6 langauge attributes and DW_LNAME constants
https://dwarfstd.org/languages-v6.html defines the DWARF v6 language
attributes, DW_AT_language_name, DW_AT_language_version and DW_LNAME
constants as usable by pre-DWARF v6 producers and consumers.
Also add new DW_LANG_V and DW_LANG_Algol68 (DWARFv5) language constants.
* config/known-dwarf.awk: Handle DW_LNAME.
* libdw/dwarf.h: Add DW_AT_language_name and DW_AT_language_version
plus all currently defined DW_LNAME constants. Add DW_LANG_V and
DW_LANG_Algol68.
* libdw/dwarf_default_lower_bound.c: Add DW_LANG_V and
DW_LANG_Algol68.
PR31862: debuginfod: client to cache x-debuginfod-* headers
This feature allows the extra http headers from debuginfod to be saved
into the client cache, and also thus replayed to clients. This way
they can perform IMA verification again, if they like, or a federating
caching intermediate debuginfod server can replay the headers it
received previously from upstream to future downstream. The headers
are placed adjacent to the payload files .cache/debuginfod/BUILDID/PAYLOAD
as .cache/debuginfod/BUILDID/hdr-PAYLOAD. They are aged the same
atime-based way.
Testing via an extension of a previous small test, and hand-running both
client & server code under valgrind. No memcheck errors reported.
Mark Wielaard [Tue, 18 Feb 2025 22:24:48 +0000 (23:24 +0100)]
libelf: Rewrite elf_scnshndx, extended index table handling
elf_scnshndx is a elfutils extension to libelf that given a SHT_SYMTAB
section returns the index to the corresponding SHT_SYMTAB_SHNDX
section, if it exists. This is needed when there are more than 64K
sections and there are symbols that have to refer to a section with an
index larger than 64K, because the Elf Sym st_shndx field is only
16 bits.
This was implemented by adding an shndx_index field to the Elf_Scn
struct which is updated when reading the section headers. This takes
up space in every section and is hard to proof correct. In the case of
using ELF_C_READ_MMAP the shndx_index field was only updated when the
shdrs needed to be converted from file to memory order.
And the two places were this function was used in readelf.c and
elf-print-reloc-syms.c the wrong section was used to lookup the
extended index table. There were also no tests for this functionality.
Replace the elf_scnshndx implementation with a simpler lookup over all
sections. This sounds inefficient, but in practice the
SHT_SYMTAB_SHNDX section is the next section after the SHT_SYMTAB
section. elf_scnshndx only needs to be called when there are more than
SHN_LORESERVE (0xff00) sections. And normally a user would just lookup
the SHT_SYMTAB and SHT_SYMTAB_SHNDX sections at the same time (which
is what readelf does when showing the symbol table, as does nm,
objcopy and libdwfl).
Add a testfile manyfuncs.c that when compiled contains 64K symbols and
sections. Make sure to use -fasynchronous-unwind-tables so there is at
least one relocatable section that uses all function symbols (e.g. on
arm32 where there is no .eh_frame by default). This can then be used
to verify the readelf --relocs support. Add another test,
test-manyfuncs that explicitly goes through the symbol table and
associated extended index table and verify each function symbol
matches the section name.
There are For riscv there are local, notype, symbols at the start of each
executable section which relocations refer to instead of the section
symbol. Since all these local symbols are called ".L0" this isn't very
useful, so print the section name instead. For powerpc ELFv1 all
function symbols go through the .opd section. Allow this in the new
test-manyfuncs test.
* libelf/elf32_getshdr.c (load_shdr_wrlock): Remove handling
of shndx_index.
* libelf/elf_begin.c (file_read_elf): Likewise.
* libelf/elf_scnshndx.c (elf_scnshndx): Rewritten.
* libelf/libelf.h (elf_scnshndx): Added full documentation.
* libelf/libelfP.h (struct Elf_Scn): Remove shndx_index field.
(__elf_scnshndx_internal): Removed.
* src/readelf.c (handle_relocs_rel): Use symscn in call to
elf_scnshndx. Print section name for local start section label.
(handle_relocs_rela): Likewise.
* tests/Makefile.am (check_PROGRAMS): Add test-manyfuncs.
(manyfuncs.o): New target.
(check-local): New target, depends on manyfuncs.o.
(TESTS): Add run-readelf-r-manyfuncs.sh and
run-test-manyfuncs.sh.
(EXTRA_DIST): Add run-readelf-r-manyfuncs.sh,
run-test-manyfuncs.sh and manyfuncs.c.
(test_manyfuncs_LDADD): New variable.
(EXTRA_test_manyfuncs_DEPENDENCIES): New variable.
(CLEANFILES): Add manyfuncs.o.
* tests/elf-print-reloc-syms.c (print_reloc_symnames):
Use symscn in call to elf_scnshndx.
* tests/manyfuncs.c: New test file to generate 64K symbols and
sections.
* tests/run-readelf-r-manyfuncs.sh: New test wrapper.
* tests/run-test-manyfuncs.sh: Likewise.
* tests/test-manyfuncs.c: New test.
Mark Wielaard [Thu, 27 Feb 2025 20:22:49 +0000 (21:22 +0100)]
readelf: Use section_name instead of elf_strptr in print_debug_frame_section
All other print_debug_* functions use section_name (ebl, shdr) to get
the current section name. Be consistent and use the same method in
print_debug_frame_section to make static analyzers happy who might
think elf_strptr can return NULL in this case.
* src/readelf.c (print_debug_frame_section): Use section_name
instead of elf_strptr to get the section name.
Anton Moryakov [Thu, 13 Feb 2025 15:57:42 +0000 (18:57 +0300)]
readelf: check elf_getarhdr returns NULL in dump_archive_index
Report of the static analyzer:
DEREF_OF_NULL.RET Pointer, returned from function 'elf_getarhdr' at
readelf.c:13551, may be NULL and is dereferenced at readelf.c:13553.
Markus Mayer [Fri, 21 Feb 2025 19:19:34 +0000 (11:19 -0800)]
aarch64: define struct user_pac_mask if needed
On Aarch64, Linux is using Pointer Authentication Code (PAC) for pointer
authentication.[1] The struct "user_pac_mask" has been part of the Linux
kernel since version 5.0 as part of this feature. However, older kernels
do not define it.
Therefore, we want to check if the definition is present in the kernel
headers and provide one if it isn't. This ensures two things:
* elfutils will continue to compile against kernel headers from 4.x
* binaries built against older kernel headers will still be fully
functional if used on a newer system
For reference, the build error that is being avoided looks like this:
[...]
CC aarch64_initreg.o
aarch64_initreg.c: In function 'aarch64_set_initial_registers_tid':
aarch64_initreg.c:61:24: error: storage size of 'pac_mask' isn't known
struct user_pac_mask pac_mask;
^~~~~~~~
aarch64_initreg.c:61:24: warning: unused variable 'pac_mask' [-Wunused-variable]
make[4]: *** [Makefile:831: aarch64_initreg.o] Error 1
make[3]: *** [Makefile:547: all-recursive] Error 1
make[2]: *** [Makefile:463: all] Error 2
Frank Ch. Eigler [Wed, 19 Feb 2025 03:09:12 +0000 (22:09 -0500)]
debuginfod-client: correct invalid free() in failed ima path
debuginfod-find with a failed signature configuration was found on f41
glibc (2.40) to sometimes leave invalid addresses in a sscanf("%ms", &ptr)
pointer in case of error, leading to an invalid free() during cleanup.
We now clear that ptr manually, making valgrind happy. Unclear why this
hasn't shown up in testsuite testing.
==1098106== Invalid free() / delete / delete[] / realloc()
==1098106== at 0x4844B83: free (vg_replace_malloc.c:989)
==1098106== by 0x49647A2: ??? (in /usr/lib64/libdebuginfod-0.192.so)
==1098106== by 0x401528: main (debuginfod-find.c:209)
==1098106== Address 0xc8 is not stack'd, malloc'd or (recently) free'd
Mark Wielaard [Wed, 12 Feb 2025 23:02:32 +0000 (00:02 +0100)]
libelf: Handle elf_strptr on section without any data
In the unlikely situation that elf_strptr was called on a section with
sh_size already set, but that doesn't have any data yet we could crash
trying to verify the string to return.
This could happen for example when a new section was created with
elf_newscn, but no data having been added yet.
* libelf/elf_strptr.c (elf_strptr): Check strscn->rawdata_base
is not NULL.
Mark Wielaard [Thu, 13 Feb 2025 13:59:34 +0000 (14:59 +0100)]
strip: Verify symbol table is a real symbol table
We didn't check the symbol table referenced from the relocation table
was a real symbol table. This could cause a crash if that section
happened to be an SHT_NOBITS section without any data. Fix this by
adding an explicit check.
* src/strip.c (INTERNAL_ERROR_MSG): New macro that takes a
message string to display.
(INTERNAL_ERROR): Use INTERNAL_ERROR_MSG with elf_errmsg (-1).
(remove_debug_relocations): Check the sh_link referenced
section is real and isn't a SHT_NOBITS section.
Mark Wielaard [Sat, 8 Feb 2025 23:07:39 +0000 (00:07 +0100)]
readelf: Skip trying to uncompress sections without a name
When combining eu-readelf -z with -x or -p to dump the data or strings
in an (corrupted ELF) unnamed numbered section eu-readelf could crash
trying to check whether the section name starts with .zdebug. Fix this
by skipping sections without a name.
* src/readelf.c (dump_data_section): Don't try to gnu decompress a
section without a name.
(print_string_section): Likewise.
Mark Wielaard [Sat, 8 Feb 2025 23:07:13 +0000 (00:07 +0100)]
readelf: Handle NULL phdr in handle_dynamic_symtab
A corrupt ELF file can have broken program headers, in which case
gelf_getphdr returns NULL. This could crash handle_dynamic_symtab
while searching for the PT_DYNAMIC phdr. Fix this by checking whether
gelf_phdr returns NULL.
Mark Wielaard [Sat, 8 Feb 2025 20:44:56 +0000 (21:44 +0100)]
libelf, readelf: Use validate_str also to check dynamic symstr data
When dynsym/str was read through eu-readelf --dynamic by readelf
process_symtab the string data was not validated, possibly printing
unallocated memory past the end of the symstr data. Fix this by
turning the elf_strptr validate_str function into a generic
lib/system.h helper function and use it in readelf to validate the
strings before use.
* libelf/elf_strptr.c (validate_str): Remove to...
* lib/system.h (validate_str): ... here. Make inline, simplify
check and document.
* src/readelf.c (process_symtab): Use validate_str on symstr_data.
Mark Wielaard [Sat, 8 Feb 2025 19:00:12 +0000 (20:00 +0100)]
libdw: Simplify __libdw_getabbrev and fix dwarf_offabbrev issue
__libdw_getabbrev could crash on reading a bad abbrev by trying to
deallocate memory it didn't allocate itself. This could happen because
dwarf_offabbrev would supply its own memory when calling
__libdw_getabbrev. No other caller did this.
Simplify the __libdw_getabbrev common code by not taking external
memory to put the abbrev result in (this would also not work correctly
if the abbrev was already cached). And make dwarf_offabbrev explicitly
copy the result (if there was no error or end of abbrev).
* libdw/dwarf_getabbrev.c (__libdw_getabbrev): Don't take
Dwarf_Abbrev result argument. Always just allocate abb when
abbrev not found in cache.
(dwarf_getabbrev): Don't pass NULL as last argument to
__libdw_getabbrev.
* libdw/dwarf_tag.c (__libdw_findabbrev): Likewise.
* libdw/dwarf_offabbrev.c (dwarf_offabbrev): Likewise. And copy
abbrev into abbrevp on success.
* libdw/libdw.h (dwarf_offabbrev): Document return values.
* libdw/libdwP.h (__libdw_getabbrev): Don't take Dwarf_Abbrev
result argument.
Aaron Merey [Fri, 31 Jan 2025 01:16:11 +0000 (20:16 -0500)]
libdwfl/offline.c: Avoid closing invalid fd
process_archive may be called with an fd argument of -1, which
libelf interprets as "no file opened". However when closing
the fd process_archive does not check whether the fd is valid
and may attempt to close an fd of -1.
Anton Moryakov [Tue, 4 Feb 2025 21:50:23 +0000 (00:50 +0300)]
objdump: Handle elf_getarhdr returning NULL in handle_ar
Report of the static analyzer:
Pointer, returned from function 'elf_getarhdr' at objdump.c:314,
may be NULL and is dereferenced at objdump.c:317. (CWE-476, CWE-690)
Corrections explained:
When processing archive elements, the code could dereference a NULL
pointer if 'elf_getarhdr' returns NULL. This patch adds a check to
ensure 'arhdr' is not NULL before using it. The fix ensures that the
function safely handles cases where 'elf_getarhdr' fails, avoiding
potential crashes.
Triggers found by static analyzer Svace.
Signed-off-by: Anton Moryakov <ant.v.moryakov@gmail.com>
Alfred Wingate [Wed, 29 Jan 2025 07:32:49 +0000 (09:32 +0200)]
Declare HAVE_ERROR_H only if error symbol is available
Musl doesn't provide error.h. But there exists a standalone implementation
that installs /usr/include/error.h but requires explicit linkage. This
leads linkage errors later on due to the assumption that error
symbols are provided by the libc.
Bug: https://bugs.gentoo.org/948878
Bug: https://sourceware.org/bugzilla/show_bug.cgi?id=32548 Signed-off-by: Alfred Wingate <parona@protonmail.com>
Michael Pratt [Mon, 13 Jan 2025 23:56:38 +0000 (23:56 +0000)]
Replace usage of ar with stored library manifest files
The ar program is called to assemble a list of objects
within each archive to assist in building combined libraries,
however make already has this information
when processing the subdirectory for that respective library.
The list can be saved in a "manifest" file
instead of being generated whenever it is needed
for use with other subdirectories.
Even though the difference in time is insignificant,
a simple "echo" and "cat" is as much as 10 times faster
than a call to "ar t" for printing the archive members.
Since elfutils builds ar,
this also removes the awkward circular dependency
where an installation of ar is required
to build the libraries for ar.
Additionally, not every version of ar is equally portable,
as native versions of ar on macOS and other BSD-like distributions
may print out a special archive member like "__.SYMDEF"
which is not a compiled object but rather just metadata
from ranlib, leading to a build failure.
Avoid these limitations by removing usage of ar
and adding build and clean rules
for the usage of archive manifest files.
* .gitignore: exclude ".manifest" file extension.
* backends/Makefile.am: add manifest file build and clean rules.
* debuginfod/Makefile.am: Likewise.
* lib/Makefile.am: Likewise.
* libasm/Makefile.am: Likewise.
* libcpu/Makefile.am: Likewise.
* libdw/Makefile.am: Likewise,
and set object lists to manifest contents.
* libdwelf/Makefile.am: Likewise.
* libdwfl/Makefile.am: Likewise.
* libebl/Makefile.am: Likewise.
* libelf/Makefile.am: Likewise,
and set object lists to manifest contents.
* src/Makefile.am: Likewise.
Aaron Merey [Tue, 21 Jan 2025 00:20:12 +0000 (19:20 -0500)]
libdwfl/argp-std.c: Set offline_next_address for a.out default
If none of -e, -p or -k is given as a command line option, libdwfl
parse_opt will use '-e a.out' as the default.
When handling -e, parse_opt will set dwfl->offline_next_address
to 0. However when handling the default '-e a.out',
dwfl->offline_next_address is not set to 0. This can result in
eu-addr2line failing to correctly output the line information for a
given address.
Fix this by setting dwfl->offline_next_address to 0 when setting up
the default '-e a.out' dwfl.
Mark Wielaard [Mon, 18 Nov 2024 23:45:39 +0000 (00:45 +0100)]
Update minimum C compiler supported to C11 plus stdatomic.h
We were using a somewhat odd mix of gnu99 with a fallback atomics.h so
we could build with gcc 4.7. Lets requires a compiler that can do at
least C11 and has a stdatomic.h. So we can build against any C
standard >= C11 (including C17 and C23). That means GCC 4.9+.
GCC 4.9 was released in 2014.
The lowest compiler version we check in the buildbots is currently
GCC 8.3.0 (Debian old old stable, released in 2018).
Also update the minimum autoconf version to 2.69. We would really like
2.70+ so we don't need the gnulib gnu11.m4. But 2.69 still seems in
use on various stable systems. autoconf 2.70 was released end of 2020,
autoconf 2.69 in 2012.
* configure.ac (AC_PACKAGE_URL): Removed workaround for
autoconf < 2.64.
(AC_PREREQ): Update from 2.63 to 2.69.
(AC_PROG_CC): Test for ac_cv_prog_cc_c11.
(AC_CACHE_CHECK for stdatomic.h
* config/eu.am (AM_CFLAGS): Remove -std=gnu99.
* lib/Makefile.am (noinst_HEADERS): Remove atomics.h and
stdatomic-fbsd.h.
* lib/atomics.h: Removed.
* lib/stdatomic-fbsd.h: Removed.
* lib/dynamicsizehash_concurrent.h: Include stdatomic.h.
* libdw/libdw_alloc.c: Likewise.
* m4/.gitignore: Add !/std-gnu11.m4.
* m4/std-gnu11.m4: New file from gnulib to provide AC_PROG_CC
from autoconf 2.70.
i386_lex_CFLAGS is a make variable to tweak the i386_lex.c.o
specific compile flags. The are already some -W defined there, I'd
like to add one to avoid seeing false warning during build with -O3
compile flag.
GCC 14.2 (as well as 13.x) produce a false positive warning for the
i386 lexer, a bug is filled already against GCC for that, CLANG is
immune.
Mark Wielaard [Wed, 8 Jan 2025 11:37:35 +0000 (12:37 +0100)]
config: Put the -D option as first argument to INSTALL_DATA
INSTALL_DATA uses the install program to install files. The GNU
coreutils implementation of install takes options anywhere. Other
implementations of install might take options only before the source
and destination arguments.
* config/Makefile.am (all-local): Move -D option before
file and directory arguments.
Michael Pratt [Sun, 24 Nov 2024 19:03:58 +0000 (19:03 +0000)]
Consolidate list of custom phony targets
Having a target defined as phony within a condition
while another target is always defined as phony
causes an automake warning:
Makefile.am:67: warning: .PHONY was already defined in condition TRUE, which includes condition GCOV ...
config/eu.am:141: ... '.PHONY' previously defined here
Makefile.am:21: 'config/eu.am' included from here
tests/Makefile.am:895: warning: .PHONY was already defined in condition TRUE, which includes condition GCOV ...
config/eu.am:141: ... '.PHONY' previously defined here
tests/Makefile.am:19: 'config/eu.am' included from here
Instead, list all the custom targets that are phony
in the common definitions in the eu.am file.
Since it is all related to coverage at this moment,
the list can be grouped as it is instead of moved or split.
* Makefile.am: remove .PHONY list in conditional
* config/eu.am: add coverage target to .PHONY list
* tests/Makefile.am: remove .PHONY list in conditional
Michael Pratt [Sun, 24 Nov 2024 19:03:47 +0000 (19:03 +0000)]
Consolidate custom clean targets for coverage files
As reported by Mark:
Since b2f225d6b ("Consolidate and add files to clean target variables")
autoreconf (automake) produces these warnings:
debuginfod/Makefile.am:130: warning: CLEANFILES multiply defined in condition TRUE ...
config/eu.am:138: ... 'CLEANFILES' previously defined here
debuginfod/Makefile.am:32: 'config/eu.am' included from here
libasm/Makefile.am:91: warning: CLEANFILES multiply defined in condition TRUE ...
config/eu.am:138: ... 'CLEANFILES' previously defined here
libasm/Makefile.am:30: 'config/eu.am' included from here
libcpu/Makefile.am:105: warning: CLEANFILES multiply defined in condition TRUE ...
config/eu.am:138: ... 'CLEANFILES' previously defined here
libcpu/Makefile.am:30: 'config/eu.am' included from here
libdw/Makefile.am:156: warning: CLEANFILES multiply defined in condition TRUE ...
config/eu.am:138: ... 'CLEANFILES' previously defined here
libdw/Makefile.am:30: 'config/eu.am' included from here
libelf/Makefile.am:142: warning: CLEANFILES multiply defined in condition TRUE ...
config/eu.am:138: ... 'CLEANFILES' previously defined here
libelf/Makefile.am:30: 'config/eu.am' included from here
src/Makefile.am:47: warning: CLEANFILES multiply defined in condition TRUE ...
config/eu.am:138: ... 'CLEANFILES' previously defined here
src/Makefile.am:19: 'config/eu.am' included from here
tests/Makefile.am:891: warning: CLEANFILES multiply defined in condition TRUE ...
config/eu.am:138: ... 'CLEANFILES' previously defined here
tests/Makefile.am:19: 'config/eu.am' included from here
This is because config/eu.am defines a default "CLEANFILES".
However, the list is only for the removal of coverage files.
Since there is an exising custom clean rule for coverage in Makefile.am
because there is a directory involved as well as files,
further simplification of these rule definitions can resolve this.
Define cleaning targets for coverage files
in a single place and as a common definition
for all generated Makefiles using a custom rule.
This allows for avoiding the use of "CLEANFILES"
variables for anything related to removing coverage files,
and removing the usage of this variable where there is a conflict,
as it is no longer needed to be defined there.
* Makefile.am: remove variables and custom clean target.
* config/eu.am: remove clean variables, add variables
and custom clean target based on removed definitions.
Fixes: b2f225d6b ("Consolidate and add files to clean target variables") Reported-by: Mark Wielaard <mark@klomp.org> Signed-off-by: Michael Pratt <mcpratt@pm.me>
Michael Pratt [Sun, 24 Nov 2024 19:03:28 +0000 (19:03 +0000)]
Makefile.am: include common eu.am definitions
Some definitions in Makefile.am can be moved
to the common definitions in config/eu.am,
but for them to apply to the top-level Makefile after moving,
eu.am must be added as an include statement.
Except for "CLEANFILES", handled in the next commit,
there are no definitions in eu.am that are defined in Makefile.am,
so there are no other conflicts or overrides between the two.
Henning Meyer [Sat, 7 Dec 2024 20:01:54 +0000 (15:01 -0500)]
debuginfod: in --cors mode, add CORS response headers and OPTIONS method
CORS is the Cross-Origin-Resource-Sharing mechanism explained at
https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS 1. by default
JavaScript code from Website A cannot request arbitrary resources from
website B, these are called cross-origin-requests 2. The browser
performs what is called a preflight check, this the OPTIONS method 3.
the response allows website B fine-grained control over what the web
browser should allow 4. Setting "Access-Control-Allow-Origin: *" tells
the web browser to allow all access, e.g. the same behavior you get with
curl or debuginfod-find The website mentions that the corresponding spec
has been changed, such that preflight requests are no longer necessary,
but in the browsers I use today (Firefox 132 and Chromium 131) they are
still necessary.
I have confirmed that I can use debuginfod with this patch from my web
application at https://core-explorer.github.io/cdx-type/
FChE simplified the code and added a few quick "curl -i | grep" tests
to confirm the new headers are there.
* debuginfod/debuginfod.cxx (handle_options): New function.
(handler_cb): Call it for OPTIONS. Add ACAO header for all
successful requests.
(parse_opt): Parse --cors.
* tests/run-debuginfod-federation-metrics.sh,
tests/run-debuginfod-find-metadata.sh: Lightly test.
* doc/debuginfod.8: Document --cors option, default off.
Signed-off-by: Henning Meyer <hmeyer.eu@gmail.com> Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
Mark Wielaard [Fri, 22 Nov 2024 17:17:29 +0000 (18:17 +0100)]
libdw: Update DW_LANG codes
Pick up the language codes published after DWARF5 was published.
The are listed at https://dwarfstd.org/languages.html
Also adjust C language dectection in dwarf_getfuncs and update the
dwarf_default_lower_bound function to return the default lower bounds
for the new langauge codes.
There is one small change in dwarf_default_lower_bound. We now return
zero instead of an error when called for DW_LANG_Mips_Assembler. Since
there is now an "official" DW_LANG_Assembly which is explicitly
defined as having a default lower bound of zero. It seems better to do
the same for the vendor code too.
* libdw/dwarf.h: Add new DW_LANG codes.
* libdw/dwarf_default_lower_bound.c (dwarf_default_lower_bound):
Handle new language codes and remove a special case for
DW_LANG_Mips_Assembler.
* libdw/dwarf_getfuncs.c (dwarf_getfuncs): Check against
DW_LANG_C17 and DW_LANG_C23.
* tests/dwarf_default_lower_bound.c: Also check for
DW_LANG_Mips_Assembler.
Mark Wielaard [Mon, 18 Nov 2024 18:50:17 +0000 (19:50 +0100)]
libdw: Don't use ATOMIC_VAR_INIT
ATOMIC_VAR_INIT was introduced in C11, but not deemed necessary to
implement atomics. So deprecated in C17 and removed in C23. Normal
initialization syntax should be sufficient.
* libdw/libdw_alloc.c (next_id): Initialize to zero without
using ATOMIC_VAR_INIT.
Michael Pratt [Wed, 23 Oct 2024 06:33:18 +0000 (06:33 +0000)]
Consolidate and add files to clean target variables
To increase the consistency of how automatic clean targets run,
define the variables together without +=, default to MOSTLYCLEANFILES
when there is no need for different levels or
add more clean levels to match other subdirectories,
add more files that are built, remove duplication, and cleanup.
Do the same for EXTRA_DIST where it is equally messy.
* backends/Makefile.am: add more objects to clean, improve spacing.
* debuginfod/Makefile.am: Likewise, and remove duplicates.
* lib/Makefile.am: improve spacing.
* libasm/Makefile.am: add more objects to clean, split similar to debuginfod.
* libcpu/Makefile.am: use normal =, add more objects to clean.
* libdw/Makefile.am: add more objects to clean, split similar to debuginfod.
* libdwelf/Makefile.am: add more objects to clean, use lowest clean level.
* libdwfl/Makefile.am: Likewise.
* libebl/Makefile.am: add more objects to clean.
* libelf/Makefile.am: add more objects to clean, split similar to debuginfod.
* src/Makefile.am: consolidate including EXTRA_DIST, split clean levels,
define with normal =, define with variables.
* tests/Makefile.am: Likewise, but not including EXTRA_DIST.
Mark Wielaard [Tue, 5 Nov 2024 22:31:14 +0000 (23:31 +0100)]
libelf: Only fetch shdr once in elf_compress[_gnu]
Some compilers assume the second call to elf[32|64]_getshdr can fail
and produce error: potential null pointer dereference. Just store the
result of the first call and reuse (when not NULL).
* libelf/elf_compress.c (elf_compress): Store getshdr result in
a shdr union var.
* libelf/elf_compress_gnu.c (): Likewise
Aaron Merey [Tue, 5 Nov 2024 22:41:26 +0000 (17:41 -0500)]
strip: Ignore --reloc-debug-sections-only for non-ET_REL files.
strip --reloc-debug-sections-only is expected to be a no-op for
non-ET_REL files. This was not enforced in the code. Sections
were copied over to a new output file and normally its contents
would be identical to the input file.
However the output file is not identical to a non-ET_REL input
file if the linker organized sections such that the indices of
SHF_ALLOC sections are not in a contigous group.
In this case strip will modify sections in order to keep all SHF_ALLOC
sections in a contiguous group.
Fix this by ignoring --reloc-debug-sections-only for non-ET_REL files.
Mark Wielaard [Thu, 24 Oct 2024 09:34:11 +0000 (11:34 +0200)]
config: Enable Debuginfod RPM sig checking and eu-stacktrace in spec
For testing that the eu-stacktrace and debuginfod ima verification
code builds correctly explicitly add --enable-stacktrace and
--enable-debuginfod-ima-verification to configure.
* config/elfutils.spec.in (enable_stacktrace): New global,
depends on arch.
(BuildRequires): Add sysprof-capture-devel.
(configure): Add --enable-stacktrace and
--enable-debuginfod-ima-verification.
(files): Add eu-stacktrace.
Mark Wielaard [Thu, 24 Oct 2024 09:06:08 +0000 (11:06 +0200)]
stacktrace: Init elf_fd in sysprof_init_dwfl
When building with LTO gcc believes elf_fd can be used uninitialized:
In function ‘sysprof_init_dwfl’,
inlined from ‘sysprof_unwind_cb’ at stacktrace.c:1235:16:
stacktrace.c:1087:7: error: ‘elf_fd’ may be used uninitialized [-Werror=maybe-uninitialized]
1087 | close (elf_fd);
| ^
This code won't be reached because if find_procfile doesn't initialize
elf_fd, it will return an error. But help the compiler by initializing
elf_fd to -1.
* src/stacktrace.c (sysprof_init_dwfl): Init elf_fd to -1.
Mark Wielaard [Tue, 22 Oct 2024 16:19:09 +0000 (18:19 +0200)]
configure: better error message for [lib]debuginfod missing dependencies
When dependencies for libdebuginfod, debuginfod or ima verification are
missing and these features are explicitly enabled the user might not
immediately know which of the dependicies are missing. Move the checks
around a little so checks for dependencies are done immediately before
the enable error message. And add the possible reason to the error to
make things more clear.
* configure.ac: Move libcurl and json-c tests before libdebuginfod
check, move libmicrohttpd, sqlite3 and libarchive tests before
debuginfod check and move librpm, libcrypto and imaevm.h tests
before ima verification check.
Mark Wielaard [Tue, 22 Oct 2024 13:03:42 +0000 (15:03 +0200)]
libelf: Add libeu objects to libelf.a static archive
libelf might use some symbols from libeu.a, specifically the eu-search
wrappers. But we don't ship libeu.a separately. So include the libeu
objects in the libelf.a archive to facilitate static linking.
* libelf/Makefile.am (libeu_objects): New variable.
(libelf_a_LIBADD): New, add libeu_objects.
Mark Wielaard [Tue, 22 Oct 2024 11:48:22 +0000 (13:48 +0200)]
tests: run-debuginfod-seekable.sh depends on LZMA and DEBUGINFOD
The run-debuginfod-seekable.sh test would run (and fail) if lzma
support was enabled, but debuginfod wasn't build. Only run it when
both are available.
* tests/Makefile.am (TESTS): Add run-debuginfod-seekable.sh if
DEBUGINFOD and LZMA.
Mark Wielaard [Fri, 18 Oct 2024 14:52:49 +0000 (16:52 +0200)]
libelf: Treat elf_memory image as writable
There are use cases where the Elf image created by elf_memory is
manipulated, through the libelf interfaces, in place. This doesn't
work anymore since we changed elf_memory to assume the memory is
read-only in elfutils 0.191. commit cc44ac674 ('libelf: Treat
elf_memory as if using ELF_C_READ_MMAP').
The reason for that change was that if elf_memory was given a
read-only memory image then decompressing a section with elf_compress
would crash. Since it directly writes the updated Shdr size. If you do
want to use elf_compress on an Elf created by elf_memory you have make
sure the memory is writable. You can do this for example by using mmap
PROTE_WRITE and MAP_PRIVATE.
* libelf/elf_memory.c (elf_memory): Call
__libelf_read_mmaped_file with ELF_C_READ_MMAP_PRIVATE.
* tests/elfgetzdata.c (main): Use mmap PROT_WRITE and MAP_PRIVATE.
elf_compress would compress all (new) data chunks, but didn't reset
the section data_list. This would cause extra data to be returned
after decompression or create bad compressed data. Add a new testcase
for this and explicitly zap the scn->data_list before resetting the
elf section raw data after (de)compression.
* libelf/elf_compress.c (__libelf_reset_rawdata): Cleanup
scn->data_list.
* tests/newzdata.c: New testcase.
* tests/Makefile.am (check_PROGRAMS): Add newzdata.
(TESTS): Likewise.
(newzdata_LDADD): New variable.
Serhei Makarov [Sat, 12 Oct 2024 19:26:06 +0000 (15:26 -0400)]
eu-stacktrace [4/5]: src: add unwind origin diagnostics to eu-stack
Since we obtain diagnostics about unwind method, another logical place
to show them is in eu-stack.
* src/stack.c (show_unwound_source): New global variable.
(struct frame): Add unwound_source field.
(frame_callback): Copy over unwound_source from Dwfl_Frame.
(print_frame): Take unwound_source string and print it.
(print_inline_frames): Take unwound_source argument and pass it on,
except for subsequent frames where we pass the string "inline".
(print_frames): Take unwound_source field from struct frame and pass
it on.
(parse_opt): Add --cfi-type,-c option to set show_unwound_source.
(main): Ditto.
* tests/run-stack-i-test.sh: Add testcase for --cfi-type.
Track the method used to unwind each Dwfl_Frame using an enum field
unwound_source; provide access to the field. Then use that in
eu-stacktrace to display the unwind methods used for each process.
This is an important diagnostic to verify how many processes are
adequately covered by the eh_frame unwinder.
* libdw/libdw.map (ELFUTILS_0.192): Add dwfl_frame_unwound_source,
dwfl_unwound_source_str.
* libdwfl/libdwfl.h (Dwfl_Unwound_Source): New enum.
(dwfl_frame_unwound_source)
(dwfl_unwound_source_str): New functions.
* libdwfl/dwfl_frame.c (dwfl_frame_unwound_source)
(dwfl_unwound_source_str): New functions.
* libdwfl/libdwflP.h: Add INTDECL for dwfl_frame_unwound_source,
dwfl_unwound_source_str.
(struct Dwfl_Frame): Add unwound_source field.
* libdwfl/frame_unwind.c (__libdwfl_frame_unwind): Set
state->unwound_source depending on the unwind method used.
* src/stacktrace.c (struct sysprof_unwind_info): Add last_pid
field to provide access to the current sample's dwfltab entry.
(sysprof_unwind_frame_cb): Add unwound_source to the data displayed
with the show_frames option.
(sysprof_unwind_cb): Set last_pid when processing a sample.
(main): Add unwind_source to the data displayed in the final summary
table.
Serhei Makarov [Fri, 11 Oct 2024 21:14:41 +0000 (17:14 -0400)]
eu-stacktrace [2/5]: configure.ac: initial version (x86/sysprof only)
Due to the x86-specific code in the initial version the configury has
significant restrictions. If --enable-stacktrace is not explicitly
provided, then eu-stacktrace will be disabled by default.
The way we test for x86 is a bit unusual. What we actually care about
is that the register file provided by perf_events on the system is an
x86 register file; this is done by checking that <asm/perf_regs.h> is
Linux kernel arch/x86/include/uapi/asm/perf_regs.h.
Once eu-stacktrace is properly portable across architectures,
these grody checks can be simplified. Enablement of the feature
by default depends on a released Sysprof version we can point to
for the patches.
* configure.ac: Add configure checks and conditionals for stacktrace tool.
* src/Makefile.am: Add stacktrace tool conditional on ENABLE_STACKTRACE.
Serhei Makarov [Fri, 11 Oct 2024 21:05:54 +0000 (17:05 -0400)]
eu-stacktrace [1/5]: src: add eu-stacktrace tool
eu-stacktrace is a utility to process a stream of raw stack
samples (such as those obtained from the Linux kernel's
PERF_SAMPLE_STACK facility) into a stream of stack traces (such as
those obtained from PERF_SAMPLE_CALLCHAIN), freeing other profiling
utilities from having to implement their own backtracing logic.
eu-stacktrace accepts data from a profiling tool via a pipe or
fifo. The initial version of the tool works on x86 architectures and
accepts data from Sysprof [1]. For future work, it will make sense
to expand support to other profilers, in particular perf tool.
Further patches in this series provide configury, docs, and improved
diagnostics for tracking the method used to unwind each frame in the
stack trace.
[1]: The following patched version of Sysprof (upstream submission ETA
~very_soon) can produce data with stack samples:
Note that sysprof polkit actions must be installed systemwide
(e.g. installing the system sysprof package will provide these).
Otherwise, "Action org.gnome.sysprof3.profile is not registered"
error will result.
Michael Pratt [Wed, 16 Oct 2024 19:53:52 +0000 (19:53 +0000)]
Remove usage of "unlocked" variant of stdio print functions
These "unlocked" Linux Standard Base variants of standard functions
are not available on some systems that are still capable
of building Linux and ELFs.
The difference is negligible for simple printing to stdout.
POSIX also states for the similar putc_unlocked():
These functions can safely be used in a multi-threaded program
if and only if they are called while the invoking thread owns
the (FILE *) object, as is the case after a successful call
to the flockfile() or ftrylockfile() functions.
...
These unlocked versions can be safely used
only within explicitly locked program regions,
using exported locking primitives.
and these precautions were never done.
Use the standard forms of these print functions.
There is inconsistent use of fputc_unlocked() with putc_unlocked(),
so consistently use the safer fputc() instead.
projects / thirdparty / elfutils.git / log
