Carlos O'Donell [Fri, 7 Jul 2023 15:27:08 +0000 (11:27 -0400)]
Translations: Add new ro support and update others.
This brings in the new Romanian language translations, and updates
nine other translations. Important translations in this update
include the Italian and Japanese translations for ESTALE which
remove the mention of "NFS" from the error message translation.
realloc: Limit chunk reuse to only growing requests [BZ #30579]
The trim_threshold is too aggressive a heuristic to decide if chunk
reuse is OK for reallocated memory; for repeated small, shrinking
allocations it leads to internal fragmentation and for repeated larger
allocations that fragmentation may blow up even worse due to the dynamic
nature of the threshold.
Limit reuse only when it is within the alignment padding, which is 2 *
size_t for heap allocations and a page size for mmapped allocations.
There's the added wrinkle of THP, but this fix ignores it for now,
pessimizing that case in favor of keeping fragmentation low.
Some locales define a list of mapping pairs of alternate digits and
separators for input digits (to_inpunct). This require the scanf
to create a list of all possible inputs for the optional type
modifier 'I'.
Checked on x86_64-linux-gnu.
Reviewed-by: Joe Simmons-Talbott <josimmon@redhat.com>
fileops: Don't process ,ccs= as individual mode flags (BZ#18906)
In processing the first 7 individual characters of the mode for fopen
if ,ccs= is used those characters will be processed as well. Stop
processing individual mode flags once a comma is encountered. This has
the effect of requiring ,ccs= to be the last mode flag in the mode
string. Add a testcase to check that the ,ccs= mode flag is not
processed as individual mode flags.
Frédéric Bérat [Wed, 28 Jun 2023 07:07:26 +0000 (09:07 +0200)]
libio/bits/stdio2.h: Clearly separate declaration from definitions
Move declarations from libio/bits/stdio.h to existing
libio/bits/stdio2-decl.h. This will enable future use of
__REDIRECT_FORTIFY in place of some __REDIRECT.
misc/bits/syslog.h: Clearly separate declaration from definition
This allows to include bits/syslog-decl.h in include/sys/syslog.h and
therefore be able to create the libc_hidden_builtin_proto (__syslog_chk)
prototype.
misc/bits/select2.h: Clearly separate declaration from definitions
The __fdelt_chk declaration needs to be available so that
libc_hidden_proto can be used while not redefining __FD_ELT.
Thus, misc/bits/select-decl.h is created to hold the corresponding
prototypes.
posix/bits/unistd.h: Clearly separate declaration from definitions
This change is similar to what was done for bits/wchar2.h.
Routines declaration are moved into a dedicated bits/unistd-decl.h file
which is then included into the bits/unistd.h file.
This will allow to adapt the files so that PLT entries are not created when
_FORTIFY_SOURCE is enabled.
misc/sys/cdefs.h: Create FORTIFY redirects for internal calls
The __REDIRECT* macros are creating aliases which may lead to unwanted
PLT entries when fortification is enabled.
To prevent these entries, the REDIRECT alias should be set to point to the
existing __GI_* aliases.
This is done transparently by creating a __REDIRECT_FORTIFY* version of
these macros, that can be overwritten internally when necessary.
stdio: Ensure *_chk routines have their hidden builtin definition available
If libc_hidden_builtin_{def,proto} isn't properly set for *_chk routines,
there are unwanted PLT entries in libc.so.
There is a special case with __asprintf_chk:
If ldbl_* macros are used for asprintf, ABI gets broken on s390x,
if it isn't, ppc64le isn't building due to multiple asm redirections.
This is due to the inclusion of bits/stdio-lbdl.h for ppc64le whereas it
isn't for s390x. This header creates redirections, which are not
compatible with the ones generated using libc_hidden_def.
Yet, we can't use libc_hidden_ldbl_proto on s390x since it will not
create a simple strong alias (e.g. as done on x86_64), but a versioned
alias, leading to ABI breakage.
This results in errors on s390x:
/usr/bin/ld: glibc/iconv/../libio/bits/stdio2.h:137: undefined reference
to `__asprintf_chk'
Frédéric Bérat [Fri, 16 Jun 2023 14:53:29 +0000 (16:53 +0200)]
sysdeps: Ensure ieee128*_chk routines to be properly named
The *_chk routines naming doesn't match the name that would be generated
using libc_hidden_ldbl_proto. Since the macro is needed for some of
these *_chk functions for _FORTIFY_SOURCE to be enabled, that needed to
be fixed.
While at it, all the *_chk function get renamed appropriately for
consistency, even if not strictly necessary.
Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org> Reviewed-by: Paul E. Murphy <murphyp@linux.ibm.com>
Frédéric Bérat [Fri, 17 Mar 2023 09:17:28 +0000 (10:17 +0100)]
Exclude routines from fortification
Since the _FORTIFY_SOURCE feature uses some routines of Glibc, they need to
be excluded from the fortification.
On top of that:
- some tests explicitly verify that some level of fortification works
appropriately, we therefore shouldn't modify the level set for them.
- some objects need to be build with optimization disabled, which
prevents _FORTIFY_SOURCE to be used for them.
Assembler files that implement architecture specific versions of the
fortified routines were not excluded from _FORTIFY_SOURCE as there is no
C header included that would impact their behavior.
Frédéric Bérat [Fri, 17 Mar 2023 09:14:50 +0000 (10:14 +0100)]
Allow glibc to be built with _FORTIFY_SOURCE
Add --enable-fortify-source option.
It is now possible to enable fortification through a configure option.
The level may be given as parameter, if none is provided, the configure
script will determine what is the highest level possible that can be set
considering GCC built-ins availability and set it.
If level is explicitly set to 3, configure checks if the compiler
supports the built-in function necessary for it or raise an error if it
isn't.
If the configure option isn't explicitly enabled, it _FORTIFY_SOURCE is
forcibly undefined (and therefore disabled).
The result of the configure checks are new variables, ${fortify_source}
and ${no_fortify_source} that can be used to appropriately populate
CFLAGS.
A dedicated patch will follow to make use of this variable in Makefiles
when necessary.
Updated NEWS and INSTALL.
Adding dedicated x86_64 variant that enables the configuration.
manual: Update documentation of strerror and related functions
The current implementation of strerror is thread-safe, but this
has implications for the lifetime of the return string.
Describe the strerror_l function. Describe both variants of the
strerror_r function. Mention the lifetime of the returned string
for strerrorname_np and strerrordesc_np. Clarify that perror
output depends on the current locale.
manual: Enhance documentation of the <ctype.h> functions
Describe the problems with signed characters, and the glibc extension
to deal with most of them. Mention that the is* functions return
zero for the special argument EOF.
Andreas Schwab [Tue, 30 Jan 2018 09:16:00 +0000 (10:16 +0100)]
Always do locking when accessing streams (bug 15142, bug 14697)
Now that abort no longer calls fflush there is no reason to avoid locking
the stdio streams anywhere. This fixes a conformance issue and potential
heap corruption during exit.
Sergey Bugaev [Sun, 25 Jun 2023 23:17:51 +0000 (02:17 +0300)]
hurd: Implement MAP_EXCL
MAP_FIXED is defined to silently replace any existing mappings at the
address range being mapped over. This, however, is a dangerous, and only
rarely desired behavior.
Various Unix systems provide replacements or additions to MAP_FIXED:
* SerenityOS and Linux provide MAP_FIXED_NOREPLACE. If the address space
already contains a mapping in the requested range, Linux returns
EEXIST. SerenityOS returns ENOMEM, however that is a bug, as the
MAP_FIXED_NOREPLACE implementation is intended to be compatible with
Linux.
* FreeBSD provides the MAP_EXCL flag that has to be used in combination
with MAP_FIXED. It returns EINVAL if the requested range already
contains existing mappings. This is directly analogous to the O_EXCL
flag in the open () call.
* DragonFly BSD, NetBSD, and OpenBSD provide MAP_TRYFIXED, but with
different semantics. DragonFly BSD returns ENOMEM if the requested
range already contains existing mappings. NetBSD does not return an
error, but instead creates the mapping at a different address if the
requested range contains mappings. OpenBSD behaves the same, but also
notes that this is the default behavior even without MAP_TRYFIXED
(which is the case on the Hurd too).
Since the Hurd leans closer to the BSD side, add MAP_EXCL as the primary
API to request the behavior of not replacing existing mappings. Declare
MAP_FIXED_NOREPLACE and MAP_TRYFIXED as aliases of (MAP_FIXED|MAP_EXCL),
so any existing software that checks for either of those macros will
pick them up automatically. For compatibility with Linux, return EEXIST
if a mapping already exists.
Sergey Bugaev [Sun, 25 Jun 2023 23:17:50 +0000 (02:17 +0300)]
hurd: Fix mapping at address 0 with MAP_FIXED
Zero address passed to mmap () typically means the caller doesn't have
any specific preferred address. Not so if MAP_FIXED is passed: in this
case 0 means literal 0. Fix this case to pass anywhere = 0 into vm_map.
Sergey Bugaev [Sun, 25 Jun 2023 23:17:48 +0000 (02:17 +0300)]
hurd: Map brk non-executable
The rest of the heap (backed by individual pages) is already mapped RW.
Mapping these pages RWX presents a security hazard.
Also, in another branch memory gets allocated using vm_allocate, which
sets memory protection to VM_PROT_DEFAULT (which is RW). The mismatch
between protections prevents Mach from coalescing the VM map entries.
Sergey Bugaev [Sun, 25 Jun 2023 23:17:47 +0000 (02:17 +0300)]
htl: Let Mach place thread stacks
Instead of trying to allocate a thread stack at a specific address,
looping over the address space, just set the ANYWHERE flag in
vm_allocate (). The previous behavior:
- defeats ASLR (for Mach versions that support ASLR),
- is particularly slow if the lower 4 GB of the address space are mapped
inaccessible, as we're planning to do on 64-bit Hurd,
- is just silly.
Samuel Thibault [Sun, 2 Jul 2023 11:27:51 +0000 (11:27 +0000)]
mach: strerror must not return NULL (bug 30555)
This follows 1d44530a5be2 ("string: strerror must not return NULL (bug 30555)"):
«
For strerror, this fixes commit 28aff047818eb1726394296d27b ("string:
Implement strerror in terms of strerror_l"). This commit avoids
returning NULL for strerror_l as well, although POSIX allows this
behavior for strerror_l.
»
support: Build with exceptions and asynchronous unwind tables [BZ #30587]
Changing tst-cleanup4.c to use xread instead of read caused
the nptl/tst-cleanupx4 test to fail. The routines in libsupport.a
need to be built with exception handling and asynchronous unwind
table support.
v2: Use "CFLAGS-.oS" instead of "override CFLAGS".
H.J. Lu [Thu, 22 Jun 2023 21:30:31 +0000 (14:30 -0700)]
ld.so: Always use MAP_COPY to map the first segment [BZ #30452]
The first segment in a shared library may be read-only, not executable.
To support LD_PREFER_MAP_32BIT_EXEC on such shared libraries, we also
check MAP_DENYWRITE to decide if MAP_32BIT should be passed to mmap.
Normally the first segment is mapped with MAP_COPY, which is defined
as (MAP_PRIVATE | MAP_DENYWRITE). But if the segment alignment is
greater than the page size, MAP_COPY isn't used to allocate enough
space to ensure that the segment can be properly aligned. Map the
first segment with MAP_COPY in this case to fix BZ #30452.
Martin Coufal [Mon, 19 Jun 2023 14:05:21 +0000 (16:05 +0200)]
Add checks for wday, yday and new date formats
tm time struct contains tm_wday and tm_yday that were previously not
checked in this test. Also added new test cases for date formats
containing %D, %R or %h. Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Joe Ramsay [Wed, 28 Jun 2023 11:19:39 +0000 (12:19 +0100)]
aarch64: Add vector implementations of exp routines
Optimised implementations for single and double precision, Advanced
SIMD and SVE, copied from Arm Optimized Routines.
As previously, data tables are used via a barrier to prevent
overly aggressive constant inlining. Special-case handlers are
marked NOINLINE to avoid incurring the penalty of switching call
standards unnecessarily.
Reviewed-by: Szabolcs Nagy <szabolcs.nagy@arm.com>
Joe Ramsay [Wed, 28 Jun 2023 11:19:38 +0000 (12:19 +0100)]
aarch64: Add vector implementations of log routines
Optimised implementations for single and double precision, Advanced
SIMD and SVE, copied from Arm Optimized Routines. Log lookup table
added as HIDDEN symbol to allow it to be shared between AdvSIMD and
SVE variants.
As previously, data tables are used via a barrier to prevent
overly aggressive constant inlining. Special-case handlers are
marked NOINLINE to avoid incurring the penalty of switching call
standards unnecessarily.
Reviewed-by: Szabolcs Nagy <szabolcs.nagy@arm.com>
Joe Ramsay [Wed, 28 Jun 2023 11:19:37 +0000 (12:19 +0100)]
aarch64: Add vector implementations of sin routines
Optimised implementations for single and double precision, Advanced
SIMD and SVE, copied from Arm Optimized Routines.
As previously, data tables are used via a barrier to prevent
overly aggressive constant inlining. Special-case handlers are
marked NOINLINE to avoid incurring the penalty of switching call
standards unnecessarily.
Reviewed-by: Szabolcs Nagy <szabolcs.nagy@arm.com>
Joe Ramsay [Wed, 28 Jun 2023 11:19:36 +0000 (12:19 +0100)]
aarch64: Add vector implementations of cos routines
Replace the loop-over-scalar placeholder routines with optimised
implementations from Arm Optimized Routines (AOR).
Also add some headers containing utilities for aarch64 libmvec
routines, and update libm-test-ulps.
Data tables for new routines are used via a pointer with a
barrier on it, in order to prevent overly aggressive constant
inlining in GCC. This allows a single adrp, combined with offset
loads, to be used for every constant in the table.
Special-case handlers are marked NOINLINE in order to confine the
save/restore overhead of switching from vector to normal calling
standard. This way we only incur the extra memory access in the
exceptional cases. NOINLINE definitions have been moved to
math_private.h in order to reduce duplication.
AOR exposes a config option, WANT_SIMD_EXCEPT, to enable
selective masking (and later fixing up) of invalid lanes, in
order to trigger fp exceptions correctly (AdvSIMD only). This is
tested and maintained in AOR, however it is configured off at
source level here for performance reasons. We keep the
WANT_SIMD_EXCEPT blocks in routine sources to greatly simplify
the upstreaming process from AOR to glibc.
Reviewed-by: Szabolcs Nagy <szabolcs.nagy@arm.com>
Joseph Myers [Wed, 28 Jun 2023 21:22:14 +0000 (21:22 +0000)]
Update syscall lists for Linux 6.4
Linux 6.4 adds the riscv_hwprobe syscall on riscv and enables
memfd_secret on s390. Update syscall-names.list and regenerate the
arch-syscall.h headers with build-many-glibcs.py update-syscalls.
linux: Return unsupported if procfs can not be mount on tst-ttyname-namespace
Trying to mount procfs can fail due multiples reasons: proc is locked
due the container configuration, mount syscall is filtered by a
Linux Secuirty Module, or any other security or hardening mechanism
that Linux might eventually add.
The tests does require a new procfs without binding to parent, and
to fully fix it would require to change how the container was created
(which is out of the scope of the test itself). Instead of trying to
foresee any possible scenario, if procfs can not be mount fail with
unsupported.
Checked on aarch64-linux-gnu. Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
The tst-ttyname-direct.c checks the ttyname with procfs mounted in
bind mode (MS_BIND|MS_REC), while tst-ttyname-namespace.c checks
with procfs mount with MS_NOSUID|MS_NOEXEC|MS_NODEV in a new
namespace.
Checked on x86_64-linux-gnu and aarch64-linux-gnu. Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
Maxim Kuvyrkov [Thu, 15 Jun 2023 15:25:47 +0000 (15:25 +0000)]
Fix tests-clean Makefile target (bug 30545)
This patch improves tests-clean Makefile target to reliably clean
test artifacts from a build directory. Before this patch tests-clean
missed around 3k (out of total 9k) .out and .test-result files.
Sergey Bugaev [Sat, 17 Jun 2023 16:40:25 +0000 (19:40 +0300)]
elf: Port ldconfig away from stack-allocated paths
ldconfig was allocating PATH_MAX bytes on the stack for the library file
name. The issues with PATH_MAX usage are well documented [0][1]; even if
a program does not rely on paths being limited to PATH_MAX bytes,
allocating 4096 bytes on the stack for paths that are typically rather
short (strlen ("/lib64/libc.so.6") is 16) is wasteful and dangerous.
Paul Eggert [Thu, 22 Jun 2023 20:44:50 +0000 (13:44 -0700)]
Call "CST" a time zone abbreviation, not a name
In documentation, call strings like "CST" time zone abbreviations, not
time zone names. This terminology is more precise, and is what tzdb uses.
A string like "CST" is ambiguous and does not fully name a time zone.
Frederic Berat [Tue, 20 Jun 2023 18:19:03 +0000 (20:19 +0200)]
debug/readlink{, at}_chk.c: Harmonize declaration and definition
The declaration and definition of these routines aren't consistent.
Make the definition of __readlink_chk and __readlinkat_chk match the
declaration of the routines they fortify. While there are no problems
today this avoids any future potential problems related to the mismatch.
Frederic Berat [Tue, 20 Jun 2023 18:18:59 +0000 (20:18 +0200)]
wcsmbs/bits/wchar2{, -decl}.h: Clearly separate declaration from definitions
This will enable __REDIRECT_FORTIFY* macros to be used when _FORTIFY_SOURCE
is set.
Routine declarations that were in bits/wchar2.h are moved into the
bits/wchar2-decl.h file.
The file is now included into include/wchar.h irrespectively from
fortification.
There is a potential memory leak for large writes due to writev being a
"shall occur" cancellation point. Add back the cleanup handler removed
in cf30aa43a5917f441c9438aaee201c53c8e1d76b.
Checked on i686-gnu and x86_64-linux-gnu.
Message-Id: <20230619143842.2901522-1-josimmon@redhat.com>
Joseph Myers [Mon, 19 Jun 2023 19:40:34 +0000 (19:40 +0000)]
C2x scanf %b support
ISO C2x defines scanf %b for input of binary integers (with an
optional 0b or 0B prefix). Implement such support, along with the
corresponding SCNb* macros in <inttypes.h>. Unlike the support for
binary integers with 0b or 0B prefix with scanf %i, this is supported
in all versions of scanf (independent of the standards mode used for
compilation), because there are no backwards compatibility concerns
(%b wasn't previously a supported format) the way there were for %i.
Joseph Myers [Mon, 19 Jun 2023 18:52:12 +0000 (18:52 +0000)]
C2x printf %wN, %wfN support (bug 24466)
ISO C2x defines printf length modifiers wN (for intN_t / int_leastN_t
/ uintN_t / uint_leastN_t) and wfN (for int_fastN_t / uint_fastN_t).
Add support for those length modifiers (such a feature was previously
requested in bug 24466). scanf support is to be added separately.
GCC 13 has format checking support for these modifiers.
When used with the support for registering format specifiers, these
modifiers are translated to existing flags in struct printf_info,
rather than trying to add some way of distinguishing them without
breaking the printf_info ABI. C2x requires an error to be returned
for unsupported values of N; this is implemented for printf-family
functions, but the parse_printf_format interface doesn't support error
returns, so such an error gets discarded by that function.
Frédéric Bérat [Wed, 14 Jun 2023 08:52:07 +0000 (10:52 +0200)]
tests: replace system by xsystem
With fortification enabled, system calls return result needs to be checked,
has it gets the __wur macro enabled. Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
Frédéric Bérat [Wed, 14 Jun 2023 08:52:06 +0000 (10:52 +0200)]
tests: replace read by xread
With fortification enabled, read calls return result needs to be checked,
has it gets the __wur macro enabled.
Note on read call removal from sysdeps/pthread/tst-cancel20.c and
sysdeps/pthread/tst-cancel21.c:
It is assumed that this second read call was there to overcome the race
condition between pipe closure and thread cancellation that could happen
in the original code. Since this race condition got fixed by d0e3ffb7a58854248f1d5e737610d50cd0a60f46 the second call seems
superfluous. Hence, instead of checking for the return value of read, it
looks reasonable to simply remove it. Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
Florian Weimer [Thu, 15 Jun 2023 10:08:22 +0000 (12:08 +0200)]
string: strerror must not return NULL (bug 30555)
For strerror, this fixes commit 28aff047818eb1726394296d27b ("string:
Implement strerror in terms of strerror_l"). This commit avoids
returning NULL for strerror_l as well, although POSIX allows this
behavior for strerror_l.
Florian Weimer [Wed, 14 Jun 2023 16:10:08 +0000 (18:10 +0200)]
Implement strlcpy and strlcat [BZ #178]
These functions are about to be added to POSIX, under Austin Group
issue 986.
The fortified strlcat implementation does not raise SIGABRT if the
destination buffer does not contain a null terminator, it just
inherits the non-failing regular strlcat behavior.
Frederic Berat [Mon, 12 Jun 2023 15:18:21 +0000 (17:18 +0200)]
tests: replace fgets by xfgets
With fortification enabled, fgets calls return result needs to be checked,
has it gets the __wur macro enabled. Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
Frederic Berat [Mon, 12 Jun 2023 15:18:20 +0000 (17:18 +0200)]
tests: replace fread by xfread
With fortification enabled, fread calls return result needs to be checked,
has it gets the __wur macro enabled. Reviewed-by: Siddhesh Poyarekar <siddhesh@sourceware.org>
The tst-mallocfork2 and tst-mallocfork3 create large number of
subprocesss, around 11k for former and 20k for latter, to check
for malloc async-signal-safeness on both fork and _Fork. However
they do not really exercise allocation patterns different than
other tests fro malloc itself, and the spawned process just exit
without any extra computation.
The tst-malloc-tcache-leak is similar, but creates 100k threads
and already checks the resulting with mallinfo.
These tests are also very sensitive to system load (since they
estresss heavy the kernel resource allocation), and adding them
on THP tunable and mcheck tests increase the pressure even more.
For THP the fork tests do not add any more coverage than other
tests. The mcheck is also not enable for tst-malloc-tcache-leak.
Checked on x86_64-linux-gnu. Reviewed-by: Carlos O'Donell <carlos@redhat.com>
There is no fork detection on current arc4random implementation, so
use lower subprocess on fork tests. The tests now run on 0.1s
instead of 8s on a Ryzen9 5900X.
Checked on x86_64-linux-gnu. Reviewed-by: Carlos O'Donell <carlos@redhat.com>
tst-getdate: Improve testcase flexibility and add test.
The getdate testcases all expect successful results. Add support for
negative testcases and testcases where a full date and time are not
supplied by skipping the tm checks in the test. Add a testcase that
would catch a use-after-free that was recently found.
Noah Goldstein [Wed, 7 Jun 2023 18:18:03 +0000 (13:18 -0500)]
x86: Make the divisor in setting `non_temporal_threshold` cpu specific
Different systems prefer a different divisors.
From benchmarks[1] so far the following divisors have been found:
ICX : 2
SKX : 2
BWD : 8
For Intel, we are generalizing that BWD and older prefers 8 as a
divisor, and SKL and newer prefers 2. This number can be further tuned
as benchmarks are run.
[1]: https://github.com/goldsteinn/memcpy-nt-benchmarks Reviewed-by: DJ Delorie <dj@redhat.com>
Noah Goldstein [Wed, 7 Jun 2023 18:18:02 +0000 (13:18 -0500)]
x86: Refactor Intel `init_cpu_features`
This patch should have no affect on existing functionality.
The current code, which has a single switch for model detection and
setting prefered features, is difficult to follow/extend. The cases
use magic numbers and many microarchitectures are missing. This makes
it difficult to reason about what is implemented so far and/or
how/where to add support for new features.
This patch splits the model detection and preference setting stages so
that CPU preferences can be set based on a complete list of available
microarchitectures, rather than based on model magic numbers. Reviewed-by: DJ Delorie <dj@redhat.com>
Noah Goldstein [Wed, 7 Jun 2023 18:18:01 +0000 (13:18 -0500)]
x86: Increase `non_temporal_threshold` to roughly `sizeof_L3 / 4`
Current `non_temporal_threshold` set to roughly '3/4 * sizeof_L3 /
ncores_per_socket'. This patch updates that value to roughly
'sizeof_L3 / 4`
The original value (specifically dividing the `ncores_per_socket`) was
done to limit the amount of other threads' data a `memcpy`/`memset`
could evict.
Dividing by 'ncores_per_socket', however leads to exceedingly low
non-temporal thresholds and leads to using non-temporal stores in
cases where REP MOVSB is multiple times faster.
Furthermore, non-temporal stores are written directly to main memory
so using it at a size much smaller than L3 can place soon to be
accessed data much further away than it otherwise could be. As well,
modern machines are able to detect streaming patterns (especially if
REP MOVSB is used) and provide LRU hints to the memory subsystem. This
in affect caps the total amount of eviction at 1/cache_associativity,
far below meaningfully thrashing the entire cache.
As best I can tell, the benchmarks that lead this small threshold
where done comparing non-temporal stores versus standard cacheable
stores. A better comparison (linked below) is to be REP MOVSB which,
on the measure systems, is nearly 2x faster than non-temporal stores
at the low-end of the previous threshold, and within 10% for over
100MB copies (well past even the current threshold). In cases with a
low number of threads competing for bandwidth, REP MOVSB is ~2x faster
up to `sizeof_L3`.
The divisor of `4` is a somewhat arbitrary value. From benchmarks it
seems Skylake and Icelake both prefer a divisor of `2`, but older CPUs
such as Broadwell prefer something closer to `8`. This patch is meant
to be followed up by another one to make the divisor cpu-specific, but
in the meantime (and for easier backporting), this patch settles on
`4` as a middle-ground.
Benchmarks comparing non-temporal stores, REP MOVSB, and cacheable
stores where done using:
https://github.com/goldsteinn/memcpy-nt-benchmarks
Sheets results (also available in pdf on the github):
https://docs.google.com/spreadsheets/d/e/2PACX-1vS183r0rW_jRX6tG_E90m9qVuFiMbRIJvi5VAE8yYOvEOIEEc3aSNuEsrFbuXw5c3nGboxMmrupZD7K/pubhtml Reviewed-by: DJ Delorie <dj@redhat.com> Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Martin Coufal [Fri, 9 Jun 2023 11:58:04 +0000 (13:58 +0200)]
Remove unused DATEMSK file for tst-getdate
tst-getdate used to rely on an in-tree datemsk file that was
subsequently replaced by a file created during test execution. This
commit removes the unused file and corresponding env-var and uses a more
appropriate name for the temp file. Reviewed-by: Carlos O'Donell <carlos@redhat.com>
Arjun Shankar [Tue, 6 Jun 2023 17:20:31 +0000 (19:20 +0200)]
time: Fix use-after-free in getdate
getdate would free the buffer pointed to by the result of its call to
strptime, then reference the same buffer later on -- leading to a
use-after-free. This commit fixes that.
Reported-by: Martin Coufal <mcoufal@redhat.com> Reviewed-by: Adhemerval Zanella <adhemerval.zanella@linaro.org>
Frédéric Bérat [Fri, 2 Jun 2023 15:28:12 +0000 (17:28 +0200)]
Move {read,write}_all functions to a dedicated header
Since these functions are used in both catgets/gencat.c and
malloc/memusage{,stat}.c, it make sense to move them into a dedicated
header where they can be inlined.
linux: Fail as unsupported if personality call is filtered
Container management default seccomp filter [1] only accepts
personality(2) with PER_LINUX, (0x0), UNAME26 (0x20000),
PER_LINUX32 (0x8), UNAME26 | PER_LINUX32, and 0xffffffff (to query
current personality)
Although the documentation only state it is blocked to prevent
'enabling BSD emulation' (PER_BSD, not implemented by Linux), checking
on repository log the real reason is to block ASLR disable flag
(ADDR_NO_RANDOMIZE) and other poorly support emulations.
So handle EPERM and fail as UNSUPPORTED if we can really check for
BZ#19408.
Joseph Myers [Mon, 5 Jun 2023 14:35:25 +0000 (14:35 +0000)]
Remove MAP_VARIABLE from hppa bits/mman.h
As suggested in
<https://sourceware.org/pipermail/libc-alpha/2023-February/145890.html>,
remove the MAP_VARIABLE define from the hppa bits/mman.h, for
consistency with Linux 6.2 which removed the define there.
Tested with build-many-glibcs.py for hppa-linux-gnu.
Since the area of the user's stack we use for the registers dump (and
otherwise as __sigreturn2's stack) can and does overlap the sigcontext,
we have to be very careful about the order of loads and stores that we
do. In particular we have to load sc_reply_port before we start
clobbering the sigcontext.
Carlos O'Donell [Thu, 18 May 2023 20:18:22 +0000 (16:18 -0400)]
Add lint-makefiles Makefile linting test.
We add a 'make check' test that lints all Makefiles in the source
directory of the glibc build. This linting test ensures that the
lines in all Makefiles will be sorted correctly as developers
creates patches. It is added to 'make check' because it is
light-weight and supports the existing developer workflow
projects / thirdparty / glibc.git / log
