DJ Delorie [Fri, 17 Jan 2025 22:34:02 +0000 (17:34 -0500)]
manual: Update signal descriptions
Based on auditing all the signals and source trees for Hurd and
Linux...
SIGSYS - This is not used for a bad system call (ENOSYS is used
for that). This is used by SECCOMP and some cases where an invalid
sub-function was requested.
SIGSTKFLT - Note it used to be a coprocessor stack fault but is now
obsolete and available for general user use.
SIGLOST - Hurd only now; note that its original purpose as an NFS
lock lost signal is obsolete.
SIGPWR - Note this is for power lost *and* power restored, and is
more a user-mode signal than a kernel-generated signal.
DJ Delorie [Tue, 10 Dec 2024 21:57:21 +0000 (16:57 -0500)]
manual: make @manpageurl more specific to each output
Tweak the @manpageurl macro to customize the output for
each of html, info, and pdf output. HTML and PDF (at
least, these days) support clicking on the link title,
whereas info does not. Add text to the intro section
explaining which man pages are normative and which
aren't.
Florian Weimer [Sun, 2 Feb 2025 19:10:09 +0000 (20:10 +0100)]
elf: Merge __dl_libc_freemem into __rtld_libc_freeres
The functions serve very similar purposes. The advantage of
__rtld_libc_freeres is that it is located within ld.so, so it is
more natural to poke at link map internals there.
This slightly regresses cleanup capabilities for statically linked
binaries. If that becomes a problem, we should start calling
__rtld_libc_freeres from __libc_freeres (perhaps after renaming it).
gfleury [Sat, 1 Feb 2025 08:02:02 +0000 (10:02 +0200)]
htl: move pthread_setcancelstate into libc.
sysdeps/pthread/sem_open.c: call pthread_setcancelstate directely
since forward declaration is gone on hurd too
Message-ID: <20250201080202.494671-1-gfleury@disroot.org>
Arjun Shankar [Tue, 14 Jan 2025 01:52:10 +0000 (02:52 +0100)]
manual: Add links to POSIX Semaphores man-pages documentation
The POSIX Semaphores functions are currently undocumented in our info
pages. This commit adds links to the man-pages documentation for all
the `sem_*' functions (except `sem_clockwait') so that they refer to
some useful documentation instead of just being stubs. `sem_clockwait'
isn't documented by man-pages but thankfully already has a small useful
blurb in our own docs.
Arjun Shankar [Tue, 14 Jan 2025 01:52:09 +0000 (02:52 +0100)]
manual: Consolidate POSIX Semaphores docs in Threads chapter
This commit moves the `sem_*' family of functions from the IPC chapter,
replacing them with a reference to their new location in the Threads
chapter. `sem_clockwait' is also moved out of the Non-POSIX Extensions
subsection since it is now included in the standard since Issue 8:
https://pubs.opengroup.org/onlinepubs/9799919799/functions/sem_clockwait.html
Petr Malat [Tue, 28 Jan 2025 10:08:20 +0000 (11:08 +0100)]
ld.so: Decorate BSS mappings
Decorate BSS mappings with [anon: glibc: .bss <file>], for example
[anon: glibc: .bss /lib/libc.so.6]. The string ".bss" is already used
by bionic so use the same, but add the filename as well. If the name
would be longer than what the kernel allows, drop the directory part
of the path.
Refactor glibc.mem.decorate_maps check to a separate function and use
it to avoid assembling a name, which would not be used later.
Signed-off-by: Petr Malat <oss@malat.biz> Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
nptl: Add support for setup guard pages with MADV_GUARD_INSTALL
Linux 6.13 (662df3e5c3766) added a lightweight way to define guard areas
through madvise syscall. Instead of PROT_NONE the guard region through
mprotect, userland can madvise the same area with a special flag, and
the kernel ensures that accessing the area will trigger a SIGSEGV (as for
PROT_NONE mapping).
The madvise way has the advantage of less kernel memory consumption for
the process page-table (one less VMA per guard area), and slightly less
contention on kernel (also due to the fewer VMA areas being tracked).
The pthread_create allocates a new thread stack in two ways: if a guard
area is set (the default) it allocates the memory range required using
PROT_NONE and then mprotect the usable stack area. Otherwise, if a
guard page is not set it allocates the region with the required flags.
For the MADV_GUARD_INSTALL support, the stack area region is allocated
with required flags and then the guard region is installed. If the
kernel does not support it, the usual way is used instead (and
MADV_GUARD_INSTALL is disabled for future stack creations).
The stack allocation strategy is recorded on the pthread struct, and it
is used in case the guard region needs to be resized. To avoid needing
an extra field, the 'user_stack' is repurposed and renamed to 'stack_mode'.
This patch also adds a proper test for the pthread guard.
I checked on x86_64, aarch64, powerpc64le, and hppa with kernel 6.13.0-rc7.
nptl: Correct stack size attribute when stack grows up [BZ #32574]
Set stack size attribute to the size of the mmap'd region only
when the size of the remaining stack space is less than the size
of the mmap'd region.
This was reversed. As a result, the initial stack size was only
135168 bytes. On architectures where the stack grows down, the
initial stack size is approximately 8384512 bytes with the default
rlimit settings. The small main stack size on hppa broke
applications like ruby that check for stack overflows.
Signed-off-by: John David Anglin <dave.anglin@bell.net>
Florian Weimer [Wed, 29 Jan 2025 09:28:10 +0000 (10:28 +0100)]
manual: Update compatibility note on flushing of line-oriented files
Operation systems which represent text files in a line-oriented
fashion (and not as byte streams with a character sequence reserved
for line termination) logically cannot flush a buffer without
also creating a terminated line.
Update this portability note and move it to the Binary Streams
section. Add another related compatibility concern, too.
I haven't exposed _pthread_mutex_lock, _pthread_mutex_trylock and
_pthread_mutex_unlock in GLIBC_PRIVATE since there aren't used in any
code in libpthread
Message-ID: <20250103103750.870897-3-gfleury@disroot.org>
Joseph Myers [Tue, 28 Jan 2025 23:39:12 +0000 (23:39 +0000)]
Add test of input file flushing / offset issues
Having fixed several bugs relating to flushing of FILE* streams (with
fflush and other operations) and their offsets (both the file position
indicator in the FILE*, and the offset in the underlying open file
description), especially after ungetc but not limited to that case,
add a test that more systematically covers different combinations of
cases for such issues, with 57220 separate scenarios tested (which
include examples of all the five separate fixed bugs), all of which
pass given the five previous bug fixes.
Joseph Myers [Tue, 28 Jan 2025 23:20:08 +0000 (23:20 +0000)]
Fix fflush handling for mmap files after ungetc (bug 32535)
As discussed in bug 32535, fflush fails on files opened for reading
using mmap after ungetc. Fix the logic to handle this case and still
compute the file offset correctly.
Joseph Myers [Tue, 28 Jan 2025 22:35:21 +0000 (22:35 +0000)]
Fix fseek handling for mmap files after ungetc or fflush (bug 32529)
As discussed in bug 32529, fseek fails on files opened for reading
using mmap after ungetc. The implementation of fseek for such files
has an offset computation that's also incorrect after fflush. A
combined fix addresses both problems (with tests for both included as
well) and it seems reasonable to consider them a single bug.
Joseph Myers [Tue, 28 Jan 2025 21:53:49 +0000 (21:53 +0000)]
Make fflush (NULL) flush input files (bug 32369)
As discussed in bug 32369 and required by POSIX, the POSIX feature
fflush (NULL) should flush input files, not just output files. The
POSIX requirement is that "fflush() shall perform this flushing action
on all streams for which the behavior is defined above", and the
definition for input files is for "a stream open for reading with an
underlying file description, if the file is not already at EOF, and
the file is one capable of seeking".
Implement this requirement in glibc. (The underlying flushing
implementation is what deals with avoiding errors for seeking on an
unseekable file.)
Joseph Myers [Tue, 28 Jan 2025 20:22:56 +0000 (20:22 +0000)]
Make fclose seek input file to right offset (bug 12724)
As discussed in bug 12724 and required by POSIX, before an input file
(based on an underlying seekable file descriptor) is closed, fclose is
sometimes required to seek that file descriptor to the correct offset,
so that any other file descriptors sharing the underlying open file
description are left at that offset (as a motivating example, a script
could call a sequence of commands each of which processes some data
from (seekable) stdin using stdio; fclose needs to do this so that
each successive command can read exactly the data not handled by
previous commands), but glibc fails to do this.
The precise POSIX wording has changed a few times; in the 2024 edition
it's "If the file is not already at EOF, and the file is one capable
of seeking, the file offset of the underlying open file description
shall be set to the file position of the stream if the stream is the
active handle to the underlying file description.".
Add appropriate logic to _IO_new_file_close_it to handle this case. I
haven't made any attempt to test or change things in this area for the
"old" functions.
Note that there was a previous attempt to fix bug 12724, reverted in
commit eb6cbd249f4465b01f428057bf6ab61f5f0c07e3. The fix version here
addresses the original test in that bug report without breaking the
one given in a subsequent comment in that bug report (which works with
glibc before the patch, but maybe was broken by the original fix that
was reverted).
The logic here tries to take care not to seek the file, even to its
newly computed current offset, if at EOF / possibly not the active
handle; even seeking to the current offset would be problematic
because of a potential race (fclose computes the current offset,
another thread or process with the active handle does its own seek,
fclose does a seek (not permitted by POSIX in this case) that loses
the effect of the seek on the active handle in another thread or
process). There are tests included for various cases of being or not
being the active handle, though there aren't tests for the potential
race condition.
Joseph Myers [Tue, 28 Jan 2025 19:38:27 +0000 (19:38 +0000)]
Fix fflush after ungetc on input file (bug 5994)
As discussed in bug 5994 (plus duplicates), POSIX requires fflush
after ungetc to discard pushed-back characters but preserve the file
position indicator. For this purpose, each ungetc decrements the file
position indicator by 1; it is unspecified after ungetc at the start
of the file, and after ungetwc, so no special handling is needed for
either of those cases.
This is fixed with appropriate logic in _IO_new_file_sync. I haven't
made any attempt to test or change things in this area for the "old"
functions; the case of files using mmap is addressed in a subsequent
patch (and there seem to be no problems in this area with files opened
with fmemopen).
libio: Start to return errors when flushing fwrite's buffer [BZ #29459]
When an error happens, fwrite is expected to return a value that is less
than nmemb. If this error happens while flushing its internal buffer,
fwrite is in a complex scenario: all the data might have been written to
the buffer, indicating a successful copy, but the buffer is expected to
be flushed and it was not.
POSIX.1-2024 states the following about errors on fwrite:
If an error occurs, the resulting value of the file-position indicator
for the stream is unspecified.
The fwrite() function shall return the number of elements successfully
written, which may be less than nitems if a write error is encountered.
With that in mind, this commit modifies _IO_new_file_write in order to
return the total number of bytes written via the file pointer. It also
modifies fwrite in order to use the new information and return the
correct number of bytes written even when sputn returns EOF.
Add 2 tests:
1. tst-fwrite-bz29459: This test is based on the reproducer attached to
bug 29459. In order to work, it requires to pipe stdout to another
process making it hard to reuse test-driver.c. This code is more
specific to the issue reported.
2. tst-fwrite-pipe: Recreates the issue by creating a pipe that is shared
with a child process. Reuses test-driver.c. Evaluates a more generic
scenario.
Co-authored-by: Florian Weimer <fweimer@redhat.com> Reviewed-by: DJ Delorie <dj@redhat.com>
Martin Coufal [Thu, 23 Jan 2025 12:04:06 +0000 (13:04 +0100)]
Add new tests for fopen
Adding some basic tests for fopen, testing different modes, stream
positioning and concurrent read/write operation on files. Reviewed-by: DJ Delorie <dj@redhat.com>
Xi Ruoyao [Sat, 25 Jan 2025 16:48:27 +0000 (00:48 +0800)]
testsuite: Make stdio-common/tst-printf-format-*-mem UNSUPPORTED if the mtrace output does not exist
When gawk was not built with MPFR, there's no mtrace output and those
tests FAIL. But we should make them UNSUPPORTED like other
tst-printf-format-* tests in the case.
Signed-off-by: Xi Ruoyao <xry111@xry111.site> Reviewed-by: Sam James <sam@gentoo.org> Reviewed-by: Andreas K Hüttel <dilfridge@gentoo.org>
Florian Weimer [Fri, 24 Jan 2025 09:40:28 +0000 (10:40 +0100)]
stdlib: Re-implement free (environ) compatibility kludge for setenv
For the originally failing application (userhelper from usermode),
it is not actually necessary to call realloc on the environ
pointer. Yes, there will be a memory leak because the application
assigns a heap-allocated pointer to environ that it never frees,
but this leak was always there: the old realloc-based setenv had
a hidden internal variable, last_environ, that was used in a similar
way to __environ_array_list. The application is not impacted by
the leak anyway because the relevant operations do not happen in
a loop.
The change here just uses a separte heap allocation and points
environ to that. This means that if an application calls
free (environ) and restores the environ pointer to the value
at process start, and does not modify the environment further,
nothing bad happens.
This change should not invalidate any previous testing that went into
the original getenv thread safety change, commit 7a61e7f557a97ab597d6
("stdlib: Make getenv thread-safe in more cases").
The new test cases are modeled in part on the env -i use case from
bug 32588 (with !DO_MALLOC && !DO_EARLY_SETENV), and the previous
stdlib/tst-setenv-malloc test. The DO_MALLOC && !DO_EARLY_SETENV
case in the new test should approximate what userhelper from the
usermode package does.
Florian Weimer [Wed, 22 Jan 2025 12:48:56 +0000 (13:48 +0100)]
stdlib: Support malloc-managed environ arrays for compatibility
Some applications set environ to a heap-allocated pointer, call
setenv (expecting it to call realloc), free environ, and then
restore the original environ pointer. This breaks after
commit 7a61e7f557a97ab597d6fca5e2d1f13f65685c61 ("stdlib: Make
getenv thread-safe in more cases") because after the setenv call,
the environ pointer does not point to the start of a heap allocation.
Instead, setenv creates a separate allocation and changes environ
to point into that. This means that the free call in the application
results in heap corruption.
The interim approach was more compatible with other libcs because
it does not assume that the incoming environ pointer is allocated
as if by malloc (if it was written by the application). However,
it seems to be more important to stay compatible with previous
glibc version: assume the incoming pointer is heap allocated,
and preserve this property after setenv calls.
Florian Weimer [Mon, 20 Jan 2025 20:06:59 +0000 (21:06 +0100)]
stdlib: Test for expected sequence of random numbers from rand
As the test comment explains, this test is not quite valid, but
preserving the exact sequences helps distributions to port to
newer glibc versions. We can remove this test if we ever switch
to a different implementation.
Florian Weimer [Mon, 20 Jan 2025 20:06:43 +0000 (21:06 +0100)]
stdlib: Fix unintended change to the random_r implementation
Commit d5bceac99d24af1131b90027dab267e437b65cd1 changed the sequence
of random numbers. This was completely unintended. The statistical
properties of the new sequences are unclear, so restore the old
behavior.
Yury Khrustalev [Tue, 26 Nov 2024 11:38:30 +0000 (11:38 +0000)]
aarch64: Fix tests not compatible with targets supporting GCS
- Add GCS marking to some of the tests when target supports GCS
- Fix tst-ro-dynamic-mod.map linker script to avoid removing
GNU properties
- Add header with macros for GNU properties
Szabolcs Nagy [Wed, 10 Jan 2024 16:20:24 +0000 (16:20 +0000)]
aarch64: Ignore GCS property of ld.so
check_gcs is called for each dependency of a DSO, but the GNU property
of the ld.so is not processed so ldso->l_mach.gcs may not be correct.
Just assume ld.so is GCS compatible independently of the ELF marking.
Szabolcs Nagy [Fri, 14 Jul 2023 14:49:11 +0000 (15:49 +0100)]
aarch64: Enable GCS in dynamic linked exe
Use the dynamic linker start code to enable GCS in the dynamic linked
case after _dl_start returns and before _dl_start_user which marks
the point after which user code may run.
Like in the static linked case this ensures that GCS is enabled on a
top level stack frame.
Szabolcs Nagy [Thu, 13 Jul 2023 06:22:44 +0000 (07:22 +0100)]
aarch64: Enable GCS in static linked exe
Use the ARCH_SETUP_TLS hook to enable GCS in the static linked case.
The system call must be inlined and then GCS is enabled on a top
level stack frame that does not return and has no exception handlers
above it.
Szabolcs Nagy [Mon, 17 Jul 2023 07:31:05 +0000 (08:31 +0100)]
aarch64: Add glibc.cpu.aarch64_gcs tunable
This tunable controls Guarded Control Stack (GCS) for the process.
0 = disabled: do not enable GCS
1 = enforced: check markings and fail if any binary is not marked
2 = optional: check markings but keep GCS off if a binary is unmarked
3 = override: enable GCS, markings are ignored
By default it is 0, so GCS is disabled, value 1 will enable GCS.
The status is stored into GL(dl_aarch64_gcs) early and only applied
later, since enabling GCS is tricky: it must happen on a top level
stack frame. Using GL instead of GLRO because it may need updates
depending on loaded libraries that happen after readonly protection
is applied, however library marking based GCS setting is not yet
implemented.
Szabolcs Nagy [Tue, 4 Apr 2023 09:42:21 +0000 (10:42 +0100)]
aarch64: Add GCS support for makecontext
Changed the makecontext logic: previously the first setcontext jumped
straight to the user callback function and the return address is set
to __startcontext. This does not work when GCS is enabled as the
integrity of the return address is protected, so instead the context
is setup such that setcontext jumps to __startcontext which calls the
user callback (passed in x20).
The map_shadow_stack syscall is used to allocate a suitably sized GCS
(which includes some reserved area to account for altstack signal
handlers and otherwise supports maximum number of 16 byte aligned
stack frames on the given stack) however the GCS is never freed as
the lifetime of ucontext and related stack is user managed.
Szabolcs Nagy [Fri, 24 Feb 2023 16:29:32 +0000 (16:29 +0000)]
aarch64: Add GCS support for setcontext
Userspace ucontext needs to store GCSPR, it does not have to be
compatible with the kernel ucontext. For now we use the linux
struct gcs_context layout but only use the gcspr field from it.
Similar implementation to the longjmp code, supports switching GCS
if the target GCS is capped, and unwinding a continuous GCS to a
previous state.
Reviewed-by: Wilco Dijkstra <Wilco.Dijkstra@arm.com> Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Szabolcs Nagy [Thu, 23 Feb 2023 08:54:04 +0000 (08:54 +0000)]
aarch64: Add GCS support to longjmp
This implementations ensures that longjmp across different stacks
works: it scans for GCS cap token and switches GCS if necessary
then the target GCSPR is restored with a GCSPOPM loop once the
current GCSPR is on the same GCS.
This makes longjmp linear time in the number of jumped over stack
frames when GCS is enabled.
Reviewed-by: Carlos O'Donell <carlos@redhat.com> Reviewed-by: Wilco Dijkstra <Wilco.Dijkstra@arm.com>
Szabolcs Nagy [Fri, 25 Nov 2022 18:16:07 +0000 (18:16 +0000)]
aarch64: Define jmp_buf offset for GCS
The target specific internal __longjmp is called with a __jmp_buf
argument which has its size exposed in the ABI. On aarch64 this has
no space left, so GCSPR cannot be restored in longjmp in the usual
way, which is needed for the Guarded Control Stack (GCS) extension.
setjmp is implemented via __sigsetjmp which has a jmp_buf argument
however it is also called with __pthread_unwind_buf_t argument cast
to jmp_buf (in cancellation cleanup code built with -fno-exception).
The two types, jmp_buf and __pthread_unwind_buf_t, have common bits
beyond the __jmp_buf field and there is unused space there which we
can use for saving GCSPR.
For this to work some bits of those two generic types have to be
reserved for target specific use and the generic code in glibc has
to ensure that __longjmp is always called with a __jmp_buf that is
embedded into one of those two types. Morally __longjmp should be
changed to take jmp_buf as argument, but that is an intrusive change
across targets.
Note: longjmp is never called with __pthread_unwind_buf_t from user
code, only the internal __libc_longjmp is called with that type and
thus the two types could have separate longjmp implementations on a
target. We don't rely on this now (but might in the future given that
cancellation unwind does not need to restore GCSPR).
Given the above this patch finds an unused slot for GCSPR. This
placement is not exposed in the ABI so it may change in the future.
This is also very target ABI specific so the generic types cannot
be easily changed to clearly mark the reserved fields.
Reviewed-by: Carlos O'Donell <carlos@redhat.com> Reviewed-by: Wilco Dijkstra <Wilco.Dijkstra@arm.com>
Szabolcs Nagy [Wed, 22 Feb 2023 14:35:00 +0000 (14:35 +0000)]
aarch64: Add asm helpers for GCS
The Guarded Control Stack instructions can be present even if the
hardware does not support the extension (runtime checked feature),
so the asm code should be backward compatible with old assemblers.
Reviewed-by: Carlos O'Donell <carlos@redhat.com> Reviewed-by: Wilco Dijkstra <Wilco.Dijkstra@arm.com>
projects / thirdparty / glibc.git / log
