Aaron Merey [Wed, 8 Feb 2023 02:15:33 +0000 (21:15 -0500)]
debuginfod-client.c: Download section even if cached executable didn't contain it.
Before attempting to download a section, cache_find_section tries to
extract the section from existing files in the cache. If it's determined
that the section must not exist, cache_find_section returns -ENOENT to
indicate that the download should be skipped.
This patch fixes a bug where cache_find_section returns -ENOENT even
though the section exists. If the cache contains the executable but
not the debuginfo with the given build-id and the section only exists
in the debuginfo (such as any of the .debug_* sections), then
debuginfod_find_section returns -ENOENT even if the section could be
downloaded.
Fix this by having cache_find_section not return -ENOENT unless cached
debuginfo was able to be read.
Mark Wielaard [Mon, 6 Feb 2023 09:21:58 +0000 (10:21 +0100)]
debuginfod: Make sure there is only one typedef for debuginfod_client
Both debuginfod.h and libdwfl.h have a simple typedef for struct
debuginfod_client. Some compilers pedantically warn when including
both headers that such typedefs are only officially supported in
C11. So guard them with _ELFUTILS_DEBUGINFOD_CLIENT_TYPEDEF to
make them happy.
Mark Wielaard [Sat, 21 Jan 2023 23:31:57 +0000 (00:31 +0100)]
libdw: Search for abstract origin in the correct CU
With gcc -flto the abstract origin of an inlined subroutine
could be in a different CU. dwarf_getscopes might return an
empty scope if it cannot find the abstract origin scope. So
make sure to search in the
We also tried to add the origin match in pc_record directly
in the current inlined scope. This always failed, causing
to do a needless traversal, followed by the full CU scan in
dwarf_getscopes. Just always stop the pc_record search and
then do the CU origin_match in dwarf_getscopes.
Mark Wielaard [Sun, 22 Jan 2023 19:50:53 +0000 (20:50 +0100)]
addr2line: -C should not try to handle optional (ignored) argument
The --demangle option takes an option (ignored) argument. Since -C
is the short option of --demangle it also takes that optional
argument. But that means that something like -Cfi is like -C got
fi as argument, while the user expects -Cfi to be like -C -f -i.
Separate the --demangle and -C options. --demangle still takes
an optional (ignored) argument, but -C doesn't take any arguments
so -Cfi acts the same as -fiC.
Also fix --target, -b, so that it is properly ignored (and not
accidentially enables demangling).
Mark Wielaard [Thu, 19 Jan 2023 22:53:33 +0000 (23:53 +0100)]
tests: run-debuginfod-query-retry.sh /bin/sh might not have a build-id
If /bin/sh wasn't build with build-id the test fails because
debuginfod-find will not be able to even try to fetch something.
Use libdebuginfod.so.1 instead as test file because we just
build that and configure makes sure we always produce a build-id.
Mark Wielaard [Tue, 10 Jan 2023 17:18:05 +0000 (18:18 +0100)]
readelf: Check compression status of .debug section data
The various print_debug_*_section functions didn't get the section
data in the same way. Add a new get_debug_elf_data function that
gets the (possibly relocated) section data and that checks (and
warns) if the data might still be compressed in a way that we
cannot decompress.
Serhei Makarov [Tue, 10 Jan 2023 16:20:13 +0000 (11:20 -0500)]
debuginfod-find.1: note on how to find a comp_dir
doc/ChangeLog:
* debuginfod-find.1: add a note to DESCRIPTION section for the 'source'
subcommand, clarifying where to find the CU compilation-directory.
I'm looking at how to improve and document the workflow for using
debuginfod-find to study the behaviour of packaged software on a system, e.g.
in conjunction with a tracing tool like SystemTap. You can read the exact
compiled source code with debuginfod-find source and use that to figure out
which source code locations are interesting to trace. But that process has
several non-obvious or inconvenient steps.
This patch is a clarification to the debuginfod-find source man page pointing
out the eu-readelf command that can show the comp_dir in downloaded debuginfo.
Since debuginfod-find source could be picked up and used by a developer
unfamiliar with DWARF terminology, I think such a clarification saves time
for newbies figuring out what a CU compilation-directory is.
Shahab Vahedi [Wed, 21 Dec 2022 11:27:05 +0000 (12:27 +0100)]
Add support for Synopsys ARCv2 processors
There is no regression in tests for an x86_64 build, while the new
hello_arc_hs4.ko is added as well. This is the only meaningful
test that I could add at the moment, given the features supported
by this port.
Andrew Paprocki [Wed, 21 Dec 2022 16:15:00 +0000 (11:15 -0500)]
PR29926: debuginfod: Fix usage of deprecated CURLINFO_*
The `CURLINFO_SIZE_DOWNLOAD_T` and `CURLINFO_CONTENT_LENGTH_DOWNLOAD_T`
identifiers are `enum`s, not pre-processor definitions, so the current
`#ifdef` logic is not selecting the newer API. This results in the
older identifiers being used and they now generate errors when compiled
against Curl 7.87, which has silently deprecated them, causing GCC to
emit `-Werror=deprecated-declarations`.
Instead, the newer identifiers were added in Curl 7.55, so explicitly
check for `CURL_AT_LEAST_VERSION(7, 55, 0)` instead of the current
logic. This eliminates the error when compiling against Curl 7.87.
Ref: https://github.com/curl/curl/pull/1511
Signed-off-by: Andrew Paprocki <andrew@ishiboo.com>
Mark Wielaard [Tue, 20 Dec 2022 13:53:43 +0000 (14:53 +0100)]
Do not use relative include paths in library files.
Rely on include dirs being set up correctly. Setup libdw AM_CPPFLAGS
to include libebl directory. In libdwfl note that debuginfod.h is a
generated file in the builddir. Only include it in the one file
debuginfod-client.c that really needs it.
Gavin Li [Wed, 30 Nov 2022 17:26:19 +0000 (18:26 +0100)]
libdwfl: Read no more than required in dwfl_segment_report_module
Since read_portion and the standard dwfl_elf_phdr_memory_callback
functions make sure to read at least minread bytes there is no need
for dwfl_segment_report_module to check and adjust the data to the
actual buffer size read. Reading beyond the end of the expected data
size (if the buffer read is much larger) actually causes issues when
passing the data to elfXX_xlatetom() because it is possible that
src->d_size is not a multiple of recsize (for ELF_T_DYN, recsize is 16
while the minimum required alignment is 8), causing elfXX_xlatetom()
to return ELF_E_INVALID_DATA.
Signed-off-by: Gavin Li <gavin@matician.com> Signed-off-by: Mark Wielaard <mark@klomp.org>
Martin Liska [Mon, 28 Nov 2022 13:10:36 +0000 (14:10 +0100)]
Refactor elf_compare
src/ChangeLog:
* elfcompress.c (T_UNSET): Remove and use ch_type.
(T_DECOMPRESS): Likewise.
(T_COMPRESS_ZLIB): Likewise.
(T_COMPRESS_GNU): Likewise.
(enum ch_type): New.
(parse_opt): Parse to new ch_type.
(compress_section): Change gnu and compress arguments
to schtype (source compression) and dchtype (destination
compression).
(process_file): Use new enum type and adjusted compress_section.
Mark Wielaard [Tue, 15 Nov 2022 16:50:01 +0000 (17:50 +0100)]
debuginfod: Initialize response_data early in debuginfod-client query
On error going to out2, the response_data is freed. So initialize the
response_data to NULL immediately after allocation or when going back
to query_in_parallel.
Aaron Merey [Fri, 4 Nov 2022 21:22:40 +0000 (17:22 -0400)]
debuginfod_find_section: Always update rc with most recent error code
debuginfod_find_section may attempt to download both the debuginfo
and executable matching the given build-id. If neither of these
files can be found, update rc to ensure that we always return an
accurate error code in this case.
Mark Wielaard [Thu, 3 Nov 2022 12:38:45 +0000 (13:38 +0100)]
readelf: Check phdr != NULL or shdr != NULL in handle_dynamic.
The compiler doesn't know that when use_dynamic_segment is true,
then phdr should/will be non-NULL and otherwise shdr is non-NULL.
Add explicit checks to help the compiler out and in case an error
is made calling the handle_dynamic function.
Aaron Merey [Tue, 1 Nov 2022 03:44:23 +0000 (23:44 -0400)]
debuginfod: Support queries for ELF/DWARF sections
Add new function debuginfod_find_section which queries debuginfod
servers for the raw binary contents of the specified ELF/DWARF section
in a file matching the given build-id.
Extend the server webapi to support section queries. Section query
URLS have the following format: /buildid/BUILDID/section/SECTION
The server will attempt to extract the section from a debuginfo file
matching the given build-id. If the debuginfo file cannot be found
or the section has type SHT_NOBITS, the server will attempt to extract
the section from the executable file matching the build-id.
If the server is built without section query support, the client will
attempt to download the debuginfo matching the build-id and extract the
section. If the debuginfo file cannot be found or the section has type
SHT_NOBITS, the server will attempt to download the executable file
matching the build-id and extract the section.
These filenames are invalid on win32.
We don't want to include multiple .spec files for make rpm.
rpmbuild --sign is not supported anymore.
Also include hello3.specfile in EXTRA_DIST.
Escape some macros in the elfutils.spec.in file comments.
Signed-off-by: Yonggang Luo <luoyonggang@gmail.com> Signed-off-by: Mark Wielaard <mark@klomp.org>
This DEBUGINFOD_HEADERS_FILE environment variable names a file to supply
HTTP headers to outgoing requests. Notably, this allows for
Authorization headers to be added from a file under OS access control.
Signed-off-by: Daniel Thornburgh <dthorn@google.com>
Mark Wielaard [Sat, 22 Oct 2022 19:58:19 +0000 (21:58 +0200)]
configure.ac: Update AC_PROG_CC and AC_PROG_LEX for autoconf 2.70
With autoconf 2.70 we must use AC_PROG_CC (which will check for c11
and c99), for earlier versions we'll use AC_PROG_CC_C99. Also use
AC_PROG_LEX([noyywrap]), the extra argument is ignored with earlier
versions, but required for 2.70.
Mark Wielaard [Wed, 19 Oct 2022 22:02:39 +0000 (00:02 +0200)]
readelf: Handle DW_LLE_GNU_view_pair
DW_LLE_GNU_view_pair is used by gcc -gvariable-location-views=incompat5.
As described in http://www.fsfla.org/~lxoliva/papers/sfn/dwarf6-sfn-lvu.txt
and proposed for DWARF6 https://dwarfstd.org/ShowIssue.php?issue=170427.1
If we don't remove this macro, when try #include <system.h> in
libdw/memory-access.h wont' take effect because
"#define LIB_SYSTEM_H 1"
The compile error:
./../libdw/memory-access.h:390:12: error: implicit declaration of
function ‘bswap_32’ [-Werror=implicit-function-declaration]
Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Yonggang Luo [Tue, 20 Sep 2022 08:43:05 +0000 (16:43 +0800)]
Strip __ prefix from __BYTE_ORDER __LITTLE_ENDIAN and __BIG_ENDIAN
__BYTE_ORDER, __LITTLE_ENDIAN and __BIG_ENDIAN are defined by the
gcc/clang preprocessor. BYTE_ORDER, LITTLE_ENDIAN and BIG_ENDIAN are
defined in <endian.h>.
Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Yonggang Luo [Sat, 15 Oct 2022 16:36:21 +0000 (00:36 +0800)]
Move the #include <libintl.h> into eu-config.h
So we do not need include in each file.
And indeed the macro
#define _(Str) dgettext ("elfutils", Str)
access libintl function dgettext, so it's make more sense
#include <libintl.h> in file eu-config.h
Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Yonggang Luo [Sat, 15 Oct 2022 16:36:20 +0000 (00:36 +0800)]
move platform depended include into system.h of libelf
All of these files either #include <system.h> directly or #include "libelfP.h"
And now "libelfP.h also #include <system.h>, so the platform depended include
can be moved to system.h safely
Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
Mark Wielaard [Sun, 16 Oct 2022 16:02:46 +0000 (18:02 +0200)]
tests: Check lseek, read and malloc results with correct types in test.
When compiling dwfl-report-offline-memory.c on some systems (latest
gcc/glibc and --enable-sanitize-undefined) we might get:
In file included from /usr/include/features.h:490,
from /usr/include/assert.h:35,
from dwfl-report-offline-memory.c:18:
In function ‘read’,
inlined from ‘main’ at dwfl-report-offline-memory.c:68:23:
/usr/include/bits/unistd.h:38:10: error: ‘__read_alias’ specified size 18446744073709551615
exceeds maximum object size 9223372036854775807 [-Werror=stringop-overflow=]
38 | return __glibc_fortify (read, __nbytes, sizeof (char),
| ^~~~~~~~~~~~~~~
/usr/include/bits/unistd.h: In function ‘main’:
/usr/include/bits/unistd.h:26:16: note: in a call to function ‘__read_alias’ declared with
attribute ‘access (write_only, 2, 3)’
26 | extern ssize_t __REDIRECT (__read_alias, (int __fd, void *__buf,
| ^~~~~~~~~~
cc1: all warnings being treated as errors
make[2]: *** [Makefile:2461: dwfl-report-offline-memory.o] Error 1
Fix by using the correct types and checking all return values.
Aleksei Vetrov [Tue, 20 Sep 2022 13:36:37 +0000 (13:36 +0000)]
libdwfl: add dwfl_report_offline_memory
This method allows to read and report ELF from memory instead of opening
a file. That way arbitrary memory can be worked with, e.g. when coming
from a stream without the need to persist.
Another useful application is for fuzzing, because fuzzers might be able
to track accesses to the memory and change the fuzzer input to cover
more edge cases through more targeted input. Hence, add a new function
along with a test case.
A local error() implementation is used when libc does not provide it,
therefore link in libeu.a which contains this function in tests needing
error() API
due to tests/debuginfod-subr.sh:archive_test failing on
"test $fileame -ot `pwd`", with LHS file having a bogus timestamp.
LHS is generated by debuginfo-find as a cache file.
I'm not sure the fix is the correct way, but
since -D_TIME_BITS=64 mandates -D_FILE_OFFSET_BITS=64 in glibc,
this does work on glibc environment.
PR28284: add tweaks on previous debuginfod x-debuginfod* header forwarding work
Embrace case-independent headers, more fully document, handle HTTP \r.
In addition to test case, hand-tested against fedora debuginfod
instances, running federated servers under valgrind.
Debuginfod and debuginfod clients are now equipped to send
and receive http headers prefixed with X-DEBUGINFOD and
print them in verbose mode for more context
debuginfod: Use auto-sized connection pool when -C is not given with arg
Since commit 4b42d9ad, libmicrohttpd's epoll event loop is used when
available in which case we must disable its setting for spawning a thread
per request. This contradicts the debuginfod doc's description of '-C',
which indicates that if this command line option is not given then the
thread pool size is unbounded.
Fix this by using an auto-sized thread pool when '-C' is not given, just
as we do when it's given with no argument. Update the doc's description
of '-C'.
Also use a fixed-size pool even if epoll is not supported. The unbounded
pool config cannot be considered entirely reliable as it appears to cause
random fails in the run-debuginfod-webapi-concurrency test.
Martin Liska [Wed, 17 Aug 2022 07:38:17 +0000 (09:38 +0200)]
debuginfod: fix http_requests_total{type="debuginfo"} when dwz is used
When dwarf_extract_source_paths is called, it can call handle_buildid
when a rpm file used dwz. Ignore such internal request in
http_requests_total statistics.
Martin Liska [Tue, 16 Aug 2022 13:14:26 +0000 (15:14 +0200)]
Support nullglob in profile.*.in files
In openSUSE we have the following RPM checker that uses
shopt -s nullglob:
https://github.com/openSUSE/post-build-checks/blob/master/checks/50-check-libtool-deps#L31
The script loads all /etc/profile.d/*.sh files via source $FILE which
can end up by stuck cat (with no arguments):
Frank Ch. Eigler [Mon, 15 Aug 2022 10:20:10 +0000 (06:20 -0400)]
PR29474: debuginfod
Previous code sometimes confused debuginfod with concurrent queries
targeting the same RPM. One thread fetching & prefetching, the other
thread misinterpreted the sudden presence of its target file in the
fdcache as a mere unnecessary prefetch duplicate. But if it was the
other thread's target file, previous code would -skip- it completely,
resulting in a 404 error. New code allows the other thread to also
decompress the target file and return it, and still continue to its
own prefetching process for other files.
There's a performance trade-off here. Another option would be for the
other thread to check the fdcache regularly and abort its own
prefetch/fetch/prefetch loop early in case of a sudden target file
hit. That'd save CPU probably but it'd stop prefetching of later
segments of the input archive, which could save future time.
Automated testing is too time/load sensitive to attempt. Confirmed
working with Martin's stress tester.
Reported-By: Martin Liška <mliska@suse.cz> Signed-Off-By: Frank Ch. Eigler <fche@redhat.com>
Mark Wielaard [Fri, 5 Aug 2022 17:09:36 +0000 (19:09 +0200)]
po: standardize Project-Id-Version to just elfutils
The po/pl.po file already just said Project-Id-Version: elfutils\n
Do the same for the other po files which had somewhat odd names.
The generated pot file will still include the version number too.
Mark Wielaard [Mon, 1 Aug 2022 10:18:30 +0000 (12:18 +0200)]
tests: Add initial scan wait_ready in run-debuginfod-percent-escape.sh
Otherwise wait_ready for thread_work_total{role="traverse"} after the
kill -USR1 can be either zero, one or two. We want to see it change to
one first, then after the kill -USR1 it should change to two to be sure
the scan happened after the new binary was created.
Di Chen [Thu, 28 Jul 2022 08:31:05 +0000 (16:31 +0800)]
libdwfl: Add new function dwfl_frame_reg
Dwfl has most of the infrastructure to keep the full unwind state,
including the state of unwound registers per frame using
Dwfl_Thread_Callbacks. But there is no public API to access the state,
except for the PC (dwfl_frame_pc).
This commit adds a new function dwfl_frame_reg to get the value of the
DWARF register number in the given frame.
Mark Wielaard [Thu, 28 Jul 2022 13:31:12 +0000 (15:31 +0200)]
libdwfl: Rewrite reading of ar_size in elf_begin_rand
With GCC 12.1.1, glibc 2.35, -fsanitize=undefined and
-D_FORTIFY_SOURCE=3 we get the following error message:
In file included from /usr/include/ar.h:22,
from ../libelf/libelfP.h:33,
from core-file.c:31:
In function ‘pread’,
inlined from ‘pread_retry’ at ../lib/system.h:188:21,
inlined from ‘elf_begin_rand’ at core-file.c:86:16,
inlined from ‘core_file_read_eagerly’ at core-file.c:205:15:
/usr/include/bits/unistd.h:74:10: error: ‘__pread_alias’ writing 58 or more bytes into a region of size 10 overflows the destination [-Werror=stringop-overflow=]
74 | return __glibc_fortify (pread, __nbytes, sizeof (char),
| ^~~~~~~~~~~~~~~
/usr/include/ar.h: In function ‘core_file_read_eagerly’:
/usr/include/ar.h:41:10: note: destination object ‘ar_size’ of size 10
41 | char ar_size[10]; /* File size, in ASCII decimal. */
| ^~~~~~~
/usr/include/bits/unistd.h:50:16: note: in a call to function ‘__pread_alias’ declared with attribute ‘access (write_only, 2, 3)’
50 | extern ssize_t __REDIRECT (__pread_alias,
| ^~~~~~~~~~
cc1: all warnings being treated as errors
The warning disappears when dropping either -fsanitize=undefined
or when using -D_FORTIFY_SOURCE=2. It looks like a false positive.
But I haven't figured out how/why it happens.
The code is a little tricky to proof correct though. The ar_size
field is a not-zero terminated string ASCII decimal, right-padded
with spaces. Which is then converted with strtoll. Relying on the
fact that the struct ar_hdr is zero initialized, so there will be
a zero byte after the ar_size field.
Rewrite the code to just use a zero byte terminated char array.
Which is much easier to reason about. As a bonus the error disappears.
Josef Cejka [Fri, 29 Jul 2022 16:05:05 +0000 (12:05 -0400)]
debuginfod: create indexes to speed up grooming
Create indexes on _r_de and _f_de tables
to speed up delete operations called by groom() function.
Primary keys of those tables are optimalized to search rows
by buildids so delete by file and mtime attributes
has to scan the whole table. On large database can single
delete query take minutes and grooming will be aborted
before completion by time limit.
New indexes are compatible with current sqlite schema.
dwfl_get_debuginfod_client: add dummy parameter for --disable-libdebuginfod
Since the stub version of "dwfl_get_debuginfod_client" doesn't name its
parameter, building elfuitls fails on a system with gcc 10.2.1:
-----------------------------------------------------------------------------
$ ./configure ... --disable-libdebuginfod
$ make
Making all in libdwfl
CC debuginfod-client.o
/src/libdwfl/debuginfod-client.c: In function 'dwfl_get_debuginfod_client':
/src/libdwfl/debuginfod-client.c:145:29: error: parameter name omitted
145 | dwfl_get_debuginfod_client (Dwfl *)
| ^~~~~~
make[2]: *** [Makefile:707: debuginfod-client.o] Error 1
-----------------------------------------------------------------------------
This fixes the issue by providing a name for the unused parameter.
Dwfl can use debuginfod internally, which was so far totally opaque
to the outside. While the functionality is great for users of the
dwfl API, the long wait times induced by downloading of data over
debuginfod lead to complaints by endusers. To offer them a bit more
insight into the internal ongoings, one can now use e.g.
`debuginfod_set_progressfn` on the handle returned by
`dwfl_get_debuginfod_client` to report download progress.
Rename get_client to dwfl_get_debuginfod_client and make it public.
Unconditionally compile debuginfod-client.c and stub the new public
function and always return NULL when debuginfod integration was
disabled.
Ulrich Drepper [Wed, 1 Jun 2022 11:14:27 +0000 (13:14 +0200)]
Arm Ehdr flag printing
Arm needs to decode flags and I modeled it after the binutils code.
The same messages are printed. Given the requirement of the interface
and the ABIs the current version of the callback function isn't
sufficient unless one wants to create a stateful interface. The
problem is that most flags need to be interpreted in the context of
the ABI version. So I changed the API to also pass the original flag
value. This shouldn't be a problem because there are no users yet.
There is also a bug in ebl_machine_flag_name. When copying the string
provided by the callback cp is moved past the NUL byte. It should
move to the NUL byte. Otherwise one cannot anything but the first
added flag description. Finally some cosmetic changes (space after
each comma in the output).